Page MenuHomePhabricator

Analyze qualitative user satisfaction data for search on-wiki
Closed, DeclinedPublic10 Estimated Story Points

Description

We have a way of measuring how long users stay on pages they searched for and we have QuickSurveys for getting user feedback. After we deploy a survey in T118800 and obtain some data, we need to figure out if & how their length of stay correlates to the self-reported satisfaction with the search results.

Event Timeline

mpopov created this task.Sep 21 2015, 3:59 PM
mpopov raised the priority of this task from to Needs Triage.
mpopov updated the task description. (Show Details)
mpopov added subscribers: mpopov, Deskana, Ironholds and 2 others.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 21 2015, 3:59 PM

P.S. We also need to figure out alternative metrics we can use temporarily in the case that this one takes too long to validate.

Oliver and I briefly discussed the possibility of using the median lethal dose ("LD50") which is the point at which we've lost half of our population. So if that number goes up, it means the users are staying on the pages longer. This is NOT necessarily indicative of the quality of our results since we may make the system yield worse results and so LD50 goes up because suddenly users are spending more time on pages looking for content they want, figuring out why the heck we gave them that page. So if we DO end up going on with this or something similar, we would need to build in safeguards against such situations.

mpopov set Security to None.
mpopov updated the task description. (Show Details)
Ironholds triaged this task as Unbreak Now! priority.Sep 21 2015, 4:51 PM
Ironholds edited a custom field.Sep 21 2015, 4:57 PM
mpopov added a comment.EditedSep 21 2015, 9:11 PM

Results of the meeting: we have agreed to not hack together a solution that will get us in trouble with the community. We are waiting until we have a surveying solution in place. One possible solution is Readership's survey system.

Deskana lowered the priority of this task from Unbreak Now! to High.Sep 23 2015, 4:42 AM
Deskana moved this task from Needs triage to On Sprint Board on the Discovery board.

Lowering priority somewhat to reflect reality.

Moving out of the sprint since it's dependent on unscheduled, unspecified work from other teams.

Deskana renamed this task from Obtain qualitative user satisfaction data to Obtain qualitative user satisfaction data for search on-wiki.Jan 19 2016, 8:23 PM
debt renamed this task from Obtain qualitative user satisfaction data for search on-wiki to QuickSurveys: Obtain qualitative user satisfaction data for search on-wiki.Jan 28 2016, 4:48 PM
debt added a subscriber: debt.

Moving to the portal backlog board until such time as we have time to do this.

mpopov renamed this task from QuickSurveys: Obtain qualitative user satisfaction data for search on-wiki to Analyze qualitative user satisfaction data for search on-wiki.Apr 14 2016, 9:34 PM
mpopov claimed this task.
mpopov updated the task description. (Show Details)

Repurposed this task because it sounded too similar to T118800 as it was.

Deskana changed the task status from Open to Stalled.Apr 19 2016, 8:11 PM

Stalled based on the status of the epic.

Restricted Application added a subscriber: TerraCodes. · View Herald TranscriptApr 19 2016, 8:11 PM
debt moved this task from Needs triage to Later on the Discovery-Analysis board.May 31 2016, 8:57 PM
debt closed this task as Declined.Sep 20 2016, 8:24 PM

Declining this for now, as we won't be doing the quick surveys any time soon.