Page MenuHomePhabricator
Create Task
Maniphest T129362

[EPIC} Create Survey to find out why some visitors don't interact with the Wikipedia.org Portal
Closed, ResolvedPublic
Actions

Description

As a Product Manager, I want to know why a large percentage of the visitors to the wikipedia.org page do not take any action on the page.

As of March 6, the "no action" by our visitors to the portal page was over 43%, we really want to change that behavior. We'd like to add in a type of survey (Qualtrics or something similar) that will display after a period of time (ie: one minute) after the visitor arrives at the wikipedia.org portal page but does nothing.

The exact wording of this survey is yet TBD, but I wanted to investigate how best, technically, to do this.

Update: see comments for most recent update for wording

Related Objects
Search...

Event Timeline

debt created this task.Mar 9 2016, 4:46 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 9 2016, 4:46 PM
debt triaged this task as Normal priority.Mar 9 2016, 4:46 PM
debt moved this task from Backlog to To Discuss on the Discovery-Portal-Sprint board.
debt renamed this task from Add Qualtrics survey on Wikipedia.org Portal page to find out why users don't interact to Find out why some visitors don't interact with the Wikipedia.org Portal.Mar 14 2016, 8:54 PM
debt updated the task description. (Show Details)
debt moved this task from Needs triage to UX on the Discovery board.Mar 15 2016, 4:48 PM
Yair_rand added a subscriber: Yair_rand.Mar 30 2016, 7:44 PM

Have you considered the possibility that maybe some users just have www.wikipedia.org as their homepage? If that were the case, I would expect most of their (rather frequent) visits to end up in them clicking a bookmark or going to the address bar.

(I don't know how the 43% number was measured, but if it's unique visitors, then obviously my hypothesis is incorrect.)

debt added a comment.Mar 31 2016, 7:57 PM

Hi @Yair_rand - yup, we definitely have considered the possiblity that the wikipedia.org portal page is set as the home page for many browsers. Unfortunately, we have no way to determine how many browsers are setup in this manner.

This ticket is to meant to create (or otherwise use) a type of survey tool to see if we can capture some feedback from the users that "see" the portal page but don't actually use it.

debt renamed this task from Find out why some visitors don't interact with the Wikipedia.org Portal to Create Survey to find out why some visitors don't interact with the Wikipedia.org Portal.Apr 1 2016, 6:10 AM
debt edited projects, added Discovery-Portal-Backlog; removed Discovery-Portal-Sprint.
debt moved this task from needs triage to product backlog epics on the Discovery-Portal-Backlog board.
debt claimed this task.Apr 15 2016, 6:28 PM
debt edited projects, added Discovery-Portal-Sprint; removed Discovery-Portal-Backlog.

Access to Qualtrics has been achieved, have a meeting next week to go over survey contents and talk about when to launch.

debt added a comment.Apr 19 2016, 2:39 PM

Drafted sample survey to be used on Qualtrics using these (sample) questions:

  1. Why did you decide to come to Wikipedia.org?
  2. Did you find what you were looking for when you came to Wikipedia.org?
  3. What language(s) do you prefer to read or search in?
  4. Is there something you'd like to see on this page that would be helpful to you?

There will also be a thank you note, privacy policy links (WMF and Qualtrics) and a section where a visitor can give us their name and email to talk further about the portal page (if they desire).

Restricted Application added a subscriber: TerraCodes. · View Herald TranscriptApr 19 2016, 2:39 PM
JGirault added a comment.Apr 19 2016, 3:16 PM
In T129362#2218037, @debt wrote:

Drafted sample survey to be used on Qualtrics using these (sample) questions:

  1. Why did you decide to come to Wikipedia.org?
  2. Did you find what you were looking for when you came to Wikipedia.org?
  3. What language(s) do you prefer to read or search in?
  4. Is there something you'd like to see on this page that would be helpful to you?

There will also be a thank you note, privacy policy links (WMF and Qualtrics) and a section where a visitor can give us their name and email to talk further about the portal page (if they desire).

Did you draft the list of answers too?
In my experience the choice of answers is at least as important as the question itself.

debt added a comment.Apr 19 2016, 4:10 PM

Yes, here's the full (draft) questions and answers via screenshots. Please let me know if there are questions, comments, suggestions...


CKoerner_WMF added a comment.Apr 19 2016, 5:50 PM

These questions (and answers) are quick, to the point, and offer up the option to continue the conversation. 👍

debt added a comment.Apr 27 2016, 11:58 PM

Updated survey to be much shorter and pointed in answers, sent to Legal for review and privacy policy generation.

debt added a comment.Apr 29 2016, 5:08 PM

updated banner mock with privacy link and legal language

debt renamed this task from Create Survey to find out why some visitors don't interact with the Wikipedia.org Portal to [EPIC} Create Survey to find out why some visitors don't interact with the Wikipedia.org Portal.May 5 2016, 5:52 PM
debt added a project: Epic.
debt added a comment.May 6 2016, 1:20 PM

Legal approval has been achieved and privacy policy has been created. We'll create the banner and link to the WMF privacy policy and the Qualtrics privacy policy as well as the button that will go directly to the survey.

The goal is to get approximately 5,000 to 10,000 responses with this survey. To do this, we will put this banner in front of 1 in every 500 visitors to the site, using the existing EventLogging testing code logic.

We will be able to control the actual access to the survey within Qualtrics, so that we don't exceed the limits of the amount of responses that we have available.

We anticipate that this survey and banner will go live next week - we'll keep it on for at least a day (the banner). Once we're at our desired response rate, we'll turn off the survey from the Qualtrics website and then remove the banner from the portal page shortly thereafter, during a normal SWAT deployment window.

debt updated the task description. (Show Details)May 9 2016, 9:56 PM
debt added a comment.May 10 2016, 4:14 PM

Updated the banner in the ticket description to be on two lines and added the word 'statement' .

debt added a comment.May 17 2016, 4:11 PM

The banner went live on Tuesday, May 10 during morning swat for 1 in every 500 visitors. We updated the script later on that day to hit 1 in every 50 visitors, due to very slow rate of responses.

We had 318 visitors open up the survey as of Tuesday morning (9am MT) on May 17th and decided to change up the algorithm a bit to show to 1 in 30 visitors. This change was done during morning swat and we will monitor it throughout the day.

debt added a comment.May 20 2016, 8:38 PM

The survey was turned off on May 17, 2016 and the results compiled here: https://commons.wikimedia.org/wiki/File:Wikipedia_Portal_Survey_-_May_2016.pdf

debt closed this task as Resolved.May 20 2016, 8:56 PM
debt closed subtask T135236: Wikipedia.org Portal Survey: publish results as Resolved.
debt moved this task from Done to Completed on the Discovery-Portal-Sprint board.May 20 2016, 8:59 PM
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL