Run Reader Survey 1 in Spanish (mobile + desktop combined)
Closed, ResolvedPublic1 Estimated Story Points
Actions

Description

Deploy QuickSurveys to Spanish Wikipedia

~~[] Ensure T123696 is fixed and deployed to Spanish Wikipedia~~

Deploy config change for Spanish survey

Repeat T113443 in Spanish, with the exception of the platform choice: we should run this survey on both Mobile and Desktop concurrently.

Plan to collect 2k responses.

array(
		"@description"=> "description of the survey",
		"description" => "Reader-segmentation-1-description",

		"@link"=> "external link to the survey (must be https)",
		"link"=> "Reader-segmentation-1-link",


		"@question"=> "survey question message key",
		"question"=> "Reader-segmentation-1-message",

		"@privacyPolicy"=> "message key for privacy policy. May contain links.",
		"privacyPolicy"=> "Reader-segmentation-1-privacy",

		"@coverage"=> "percentage of users that will see the survey",
		"coverage"=> "0.000625", // 1 out of 1600,

		"@platform"=> "for each platform (desktop, mobile), which version of it is targeted (stable, beta, alpha)",
		"platform"=> array(
			"desktop"=> ["stable"],
			"mobile"=> ["stable"]
		)
	)

Details

Subject	Repo	Branch	Lines +/-
Stop the first survey in fawiki and eswiki	operations/mediawiki-config	master	+0 -40
Bump up the QuickSurveys sampling rates for es and fa wikis	operations/mediawiki-config	master	+4 -2
Enable QuickSurveys on eswiki	operations/mediawiki-config	master	+19 -0

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	leila	T121727 [Epic] Reader segmentation research (behavioral analysis)
Resolved	• ellery	T123770 Run Reader Survey 1 in Spanish (mobile + desktop combined)
Duplicate	None	T124488 Deploy updated QuickSurveys extension
Resolved	dr0ptp4kt	T123696 Add event logging for eligibility, impression, and tap through for Quick Surveys funnel
Resolved	dr0ptp4kt	T125097 [Release] Check EventLogging of QuickSurveys in production environment
Resolved	• ellery	T124683 Inform the eswiki community about the reader survey going out

Event Timeline

leila created this task.Jan 15 2016, 7:12 PM

leila claimed this task.

leila raised the priority of this task from to Needs Triage.

leila updated the task description. (Show Details)

leila added projects: Research-and-Data-2016-Q3, Reading-Admin, Improving-access, Research.

leila added subscribers: Jdlrobson, Cervisiarius, Aklapper and 4 others.

@ellery I'm going to work on the Japanese/Persian version of the survey. If you work on the Spanish one, that'd be great. I just assigned it to you.

leila triaged this task as Medium priority.Jan 15 2016, 11:57 PM

leila added a project: QuickSurveys.Jan 18 2016, 5:59 PM

leila mentioned this in T123772: Run Reader Survey 3 in Spanish.Jan 19 2016, 5:23 PM

Does this survey need to be run separately on mobile and desktop? If so can you add T119367 as a blocking task (currently platform is still not working)

It doesn't need to run separately, @Jdlrobson. I'll update the description and title.

leila renamed this task from Repeat the first reader survey in Spanish to Repeat the first and second reader survey in Spanish, combined.Jan 20 2016, 7:20 PM

leila updated the task description. (Show Details)

leila added a comment.Jan 22 2016, 6:41 PM

This comment was removed by leila.

@dr0ptp4kt is it possible to run this survey from 2016-01-28 to 2016-02-01? We have the translations ready, and we don't need extra Legal and Security approval as this is no different than what we did last quarter.

• ellery renamed this task from Repeat the first and second reader survey in Spanish, combined to Run Reader Survey 1 in Spanish (mobile + desktop combined).Jan 22 2016, 6:48 PM

• ellery moved this task from Backlog to Staged (ready to execute) on the Research board.

• ellery moved this task from Staged (ready to execute) to In Progress on the Research board.Jan 22 2016, 6:51 PM

• ellery updated the task description. (Show Details)Jan 22 2016, 6:58 PM

• ellery updated the task description. (Show Details)

dr0ptp4kt added a subtask: T124488: Deploy updated QuickSurveys extension.Jan 22 2016, 11:53 PM

@leila, which of survey 1 or survey 3 would run with this date range?

If basic funnel analysis isn't needed (impression ratio, tap through ratio), I think that's a SWAT deploy and MediaWiki messages. @jhobs, you have a sense of how much work that is and if the start=2016-01-28 and end=2016-02-01 is feasible (morning SWATs I'm assuming)?

@leila, in case funnel analysis is needed, I've added T124488: Deploy updated QuickSurveys extension as a blocking task for this, which in turn depends on T123696: Add event logging for eligibility, impression, and tap through for Quick Surveys funnel, which is pretty close now. Lemme know if I should remove them as blocking tasks (or feel free to do so yourself).

@jhobs and @bmansurov (CC @phuedx, @Jdlrobson, @Jhernandez), do you guys have a sense for when the code update could be merged / version rev'd / deployed (possibly SWAT?) in case funnel analysis is needed for one of these Spanish surveys? @leila, the timing of the merge / version rev / deploy would best determine when to initiate the survey in case you feel funneling is necessary.

dr0ptp4kt added a project: Reading-Web-Sprint-65-Game of Phones.Jan 23 2016, 12:15 AM

@dr0ptp4kt and everyone else. Sorry for the confusion on my end. I had copy pasted a text by mistake in this task. I deleted it.

We do not need any instrumentation beyond what we had last quarter for T113443. No funnel token or extra data collection is needed.

Note to self: this is no longer blocked by the deployment task, remove tag (hitting bug on mobile device trying to remove blocked-by task).

@dr0ptp4kt, hopefully today or tomorrow. I don't see any -1s in the patch anymore.

leila updated the task description. (Show Details)Jan 25 2016, 5:08 PM

Jdlrobson updated the task description. (Show Details)Jan 25 2016, 5:44 PM

Jdlrobson edited a custom field.

Jdlrobson moved this task from Needs Analysis to To Do on the Reading-Web-Sprint-65-Game of Phones board.

Jdlrobson updated the task description. (Show Details)Jan 25 2016, 5:47 PM

Jdlrobson added a subtask: T123696: Add event logging for eligibility, impression, and tap through for Quick Surveys funnel.

Jdlrobson merged a task: T124488: Deploy updated QuickSurveys extension.Jan 25 2016, 5:49 PM

@Jdlrobson T123696 should not be a blocker for this task unless I'm missing something.

leila added a subtask: T124683: Inform the eswiki community about the reader survey going out.Jan 25 2016, 6:44 PM

leila raised the priority of this task from Medium to High.Jan 25 2016, 6:46 PM

leila updated the task description. (Show Details)Jan 25 2016, 8:38 PM

leila updated the task description. (Show Details)Jan 25 2016, 8:45 PM

leila updated the task description. (Show Details)Jan 25 2016, 11:37 PM

• ellery updated the task description. (Show Details)Jan 26 2016, 1:07 AM

leila updated the task description. (Show Details)Jan 26 2016, 11:00 PM

I'll remove T123696 as a dependency per Leila's comment above.

• bmansurov updated the task description. (Show Details)Jan 27 2016, 12:22 AM

• bmansurov removed a subtask: T123696: Add event logging for eligibility, impression, and tap through for Quick Surveys funnel.

bd808 moved this task from Backlog to Bugs on the QuickSurveys board.Jan 27 2016, 5:06 PM

@bmansurov this survey is ready to be deployed on our end. Could you have it deployed tomorrow after 1pm PST unless you hear from us on this ticket before that?

OK, I'll try to do so.

Change 266957 had a related patch set uploaded (by Bmansurov):
Enable QuickSurveys in eswiki

https://gerrit.wikimedia.org/r/266957

gerritbot added a project: Patch-For-Review.Jan 28 2016, 12:45 AM

@leila, can you also create the following pages?

https://es.wikipedia.org/wiki/MediaWiki:Reader-segmentation-1-name
https://es.wikipedia.org/wiki/MediaWiki:Reader-segmentation-1-link

Thanks!

leila updated the task description. (Show Details)Jan 28 2016, 1:09 AM

done, @bmansurov.

Thanks! The patch will be SWAT deployed during the 08:00–09:00 PST window today.

Change 266957 merged by jenkins-bot:
Enable QuickSurveys on eswiki

https://gerrit.wikimedia.org/r/266957

@leila, you can test the survey at https://es.wikipedia.org/wiki/Diego_Gim%C3%A9nez?quicksurvey=true

@bmansurov I see the the survey in Chrome, not in Firefox. No response is registered so far.

@bmansurov Adblock Plus is blocking the widget in Firefox. Not sure if this is a feature or a bug. Something your team may want to be aware of. Otherwise, it works in Firefox.

@leila, what version of Firefox are you using? I suppose you're using the latest Mac OS X too? It works for me with Adblock Plus enabled.

• bmansurov edited projects, added Reading-Web-Sprint-64-Five Sprints at Freddy's; removed Reading-Web-Sprint-65-Game of Phones.Jan 28 2016, 6:02 PM

@bmansurov I'm on Linux, Firefox 43.0.4.

@bmansurov we are two orders of magnitude off in our sampling, we have only 1 response in Spanish so far which is not expected. We had this issue in the previous surveys as well. I /think/ what's happening is that the way EL is sampling is multiplying whatever number we provide by 0.01. Can you multiply the sampling rate by 100?

~~For Espanol only? Or is this happening in Farsi too?~~ I see your other post now.

Change 267071 had a related patch set uploaded (by Bmansurov):
Bump up the QuickSurveys sampling rates for es and fa wikis

https://gerrit.wikimedia.org/r/267071

Jdlrobson moved this task from Needs Analysis to Code Review on the Reading-Web-Sprint-64-Five Sprints at Freddy's board.Jan 28 2016, 10:48 PM

In T123770#1978599, @leila wrote:

@bmansurov I'm on Linux, Firefox 43.0.4.

I've check the site on Firefox 44.0 on Arch Linux and I was able to see the survey. Do you think it's that specific Firefox version or maybe your linux distro?

Change 267071 merged by jenkins-bot:
Bump up the QuickSurveys sampling rates for es and fa wikis

https://gerrit.wikimedia.org/r/267071

I don't know how to make sense of it. In the same Linux distribution, with the same browser, the fawiki test link works, the eswiki doesn't. :-\

Jdlrobson removed a project: Patch-For-Review.Jan 29 2016, 1:28 AM

Jdlrobson moved this task from Code Review to Doing on the Reading-Web-Sprint-64-Five Sprints at Freddy's board.

@bmansurov, we have enough responses for this survey. Could you stop it? I'm sure you know this already but just in case I checked with @greg and he approves stopping the survey on Friday (not starting one :-)