Page MenuHomePhabricator

Setup Welcome Survey for Arabic Wikipedia
Closed, ResolvedPublicJul 15 2019

Description

Target date: July 15

We will need to deploy the Welcome Survey for Arabic Wikipedia. This task is blocked by T226039. We should turn off free text responses before deploying to Arabic so that we are not capturing a small set of free text that we don't need.

The survey should be deployed in A/B test in which half of newcomer accounts get Var A and half do not get a survey.

The parent task (T225738) has the set of translations relevant for the survey.

  • Test survey in Test Wikipedia in Arabic to make sure all translations look right and that all links go to correct privacy policy and survey privacy statement.
  • Test survey on desktop and mobile by creating new accounts in production.
  • Check that data is being recorded correctly after deployment.

Event Timeline

@Dyolf77_WMF -- the welcome survey is ready for you to test in Beta wiki. You can try out the survey with these links:

Please let us know whether the translations all look okay, whether the links go to the right places, and if you find any issues with the interface.

MMiller_WMF updated the task description. (Show Details)
Restricted Application changed the subtype of this task from "Task" to "Deadline". · View Herald TranscriptJul 9 2019, 10:11 PM

@Dyolf77_WMF -- the welcome survey is ready for you to test in Beta wiki. You can try out the survey with these links:

Please let us know whether the translations all look okay, whether the links go to the right places, and if you find any issues with the interface.

Both tested.
Made some changes to the translations after viewing the context of the sentences. Now it seems okay for me.
For the links, they go to the right pages in Arabic Wikipedia.
Privacy page is in English, please see my comment above.

Privacy page is in English, please see my comment above.

Yes, we need to set wgWelcomeSurveyPrivacyPolicyUrl in the configuration repository. The other wikis have links to the privacy policy on foundation.wikimedia.org, so once we have the Arabic translation on foundation.wikimedia.org we can update the configuration.

I will ask Legal to move Habib's translations to the correct location on foundation.wikimedia.org. I will let you know when it is there.

@Dyolf77_WMF -- the legal team has moved your translation of the Survey Privacy Statement to foundation.wikimedia.org. Could you please check to make sure there are no errors and that everything was copied correctly?

https://foundation.wikimedia.org/wiki/%D8%A8%D9%8A%D8%A7%D9%86_%D8%AE%D8%B5%D9%88%D8%B5%D9%8A%D8%A9_%D8%A7%D9%84%D9%85%D8%B3%D8%AA%D8%AE%D8%AF%D9%85_%D8%A7%D9%84%D8%AC%D8%AF%D9%8A%D8%AF

In the last paragraph, the email address of Benoît should be in the end not in the beginning of the sentence. If not possible, it would be better to show it in a separate line.

Change 523202 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[operations/mediawiki-config@master] GrowthExperiments: Enable WelcomeSurvey for arwiki

https://gerrit.wikimedia.org/r/523202

Change 523202 merged by jenkins-bot:
[operations/mediawiki-config@master] GrowthExperiments: Enable WelcomeSurvey A/B test for arwiki

https://gerrit.wikimedia.org/r/523202

Mentioned in SAL (#wikimedia-operations) [2019-07-15T18:11:11Z] <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[:gerrit:523202|GrowthExperiments: Enable WelcomeSurvey A/B test for arwiki]] (T226221) (duration: 01m 02s)

Testing both desktop and mobile with 4 new accounts.
Getting the survey for 2 of them.
I noticed that I got the older version of the translation for the survey not the updated one. It was working fine on beta wiki.

I noticed that I got the older version of the translation for the survey not the updated one. It was working fine on beta wiki.

When were the updates to the translations made? They probably did not make it into wmf.13 and so won't be in production until Thursday when wmf.14 is deployed.

When were the updates to the translations made? They probably did not make it into wmf.13 and so won't be in production until Thursday when wmf.14 is deployed.

Okay, it was on July 10th.

MMiller_WMF moved this task from In Progress to QA on the Growth-Team (Sprint 0 (Growth Team)) board.

@Dyolf77_WMF told me that the translation issues are minor, and we can just let them roll in with the next train.

Assigning this now to @nettrom_WMF because the last step is checking the incoming data. I'm also about to make a subtask to analyze the results of the experiment after a month.

I've looked at the data and find no reason to be concerned. Treatment/control group assignment is where it should be (on or about 50/50), response rate after a week is good (71.5%), users are primarily answering all questions (87.4% do), abandonment rate in the survey group is low, and there are no free text responses in the data.