Page MenuHomePhabricator

Deploy survey (Community Feedback on Core Metrics Reports) on Meta-Wiki
Closed, ResolvedPublic

Description

The Hypothesis 2.3.1 team wants to deploy a survey on a specific Meta-Wiki page that will contain a Core Metrics report, in order to solicit feedback about the report.

Objective: Collect feedback from Movement members about the accessibility, transparency, and usefulness of the report. Feedback will be used to improve the next report.
When: Nov 28, 2023.
Duration: 6 weeks.
Audience: Viewers of the report page on Meta-Wiki; restrict QuickSurvey to users that are logged-in
Coverage: If possible, deploy with 100% coverage. A relatively small number of people will be visiting the page, so we would like everyone who visits the page to see the QuickSurvey solicitation.

Additional information:

Desired QuickSurvey appearance and content:

Would you like to provide feedback 
about this report?

Complete our brief, anonymous survey 
to help us better understand how useful, 
transparent, and accessible the report 
is to members of the Wikimedia Movement. 

Your feedback can help improve next
quarter's Core Metrics report. 

[] Yes. Take me to the survey.
[] No thanks.

Messages

  • question: Would you like to provide feedback about this report?

Complete our brief, anonymous survey to help us better understand how useful, transparent, and accessible the report is to members of the Wikimedia Movement.
Your feedback can help improve next quarter's Core Metrics report.

  • yesMsg: Yes. Take me to the survey.
  • confirmMsg: Thanks for the feedback! We are using it to make reports on Meta even better!

Tests

Event Timeline

Change 975059 had a related patch set uploaded (by DDesouza; author: DDesouza):

[operations/mediawiki-config@master] Pre-deploy Annual Plan Core Metrics survey

https://gerrit.wikimedia.org/r/975059

Gonna deploy a survey with zero coverage today to test messages and speed up deployment later.

I will set up a small experiment on beta prior to deployment just to confirm that the survey will target only users that are logged-in AND viewing an specific page.

Change 975059 merged by jenkins-bot:

[operations/mediawiki-config@master] Pre-deploy Annual Plan Core Metrics survey

https://gerrit.wikimedia.org/r/975059

Mentioned in SAL (#wikimedia-operations) [2023-11-16T21:07:16Z] <dr0ptp4kt@deploy2002> Started scap: Backport for [[gerrit:975059|Pre-deploy Annual Plan Core Metrics survey (T351353)]]

Mentioned in SAL (#wikimedia-operations) [2023-11-16T21:08:36Z] <dr0ptp4kt@deploy2002> dr0ptp4kt and dani: Backport for [[gerrit:975059|Pre-deploy Annual Plan Core Metrics survey (T351353)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2023-11-16T21:18:29Z] <dr0ptp4kt@deploy2002> Finished scap: Backport for [[gerrit:975059|Pre-deploy Annual Plan Core Metrics survey (T351353)]] (duration: 11m 12s)

@CMyrick-WMF

This is how the survey looks:

Screenshot from 2023-11-16 18-42-44.png (332×314 px, 28 KB)

Screenshot from 2023-11-16 18-42-58.png (107×309 px, 9 KB)

I linked the privacy policy message to the foundation privacy policy.

Thanks, @DDeSouza! This looks great.

In terms of the "Thanks for.." message, would it be possible to change to "Thanks for the feedback! We are using it to make reports on Meta even better!" ? If that's too lengthy, or if the generic message is generally preferred, I think that's fine too.

Updated desired date of deployment: Nov 28. Will that be possible, @DDeSouza?

Change 976839 had a related patch set uploaded (by DDesouza; author: DDesouza):

[operations/mediawiki-config@master] Update Annual Plan Core Metrics survey

https://gerrit.wikimedia.org/r/976839

I've added the URL for the future location of the report page, where the QuickSurvey should appear (https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2023-2024/Reports/Core_Metrics_Q1)

Change 976842 had a related patch set uploaded (by DDesouza; author: DDesouza):

[operations/mediawiki-config@master] [beta] Deploy Annual Plan Core Metrics survey

https://gerrit.wikimedia.org/r/976842

Change 976844 had a related patch set uploaded (by DDesouza; author: DDesouza):

[operations/mediawiki-config@master] Deploy Annual Plan Core Metrics survey

https://gerrit.wikimedia.org/r/976844

Change 976839 merged by jenkins-bot:

[operations/mediawiki-config@master] Update Annual Plan Core Metrics survey

https://gerrit.wikimedia.org/r/976839

Mentioned in SAL (#wikimedia-operations) [2023-11-22T21:28:10Z] <catrope@deploy2002> Started scap: Backport for [[gerrit:976839|Update Annual Plan Core Metrics survey (T351353)]]

Mentioned in SAL (#wikimedia-operations) [2023-11-22T21:29:28Z] <catrope@deploy2002> catrope and dani: Backport for [[gerrit:976839|Update Annual Plan Core Metrics survey (T351353)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2023-11-22T21:37:15Z] <catrope@deploy2002> Finished scap: Backport for [[gerrit:976839|Update Annual Plan Core Metrics survey (T351353)]] (duration: 09m 04s)

Change 976842 merged by jenkins-bot:

[operations/mediawiki-config@master] [beta] Deploy Annual Plan Core Metrics survey

https://gerrit.wikimedia.org/r/976842

DDeSouza changed the task status from Open to In Progress.Nov 28 2023, 12:08 AM
DDeSouza triaged this task as Medium priority.

This is how the survey looks:

Screenshot from 2023-11-16 18-42-44.png (332×314 px, 28 KB)

If I understand correctly, this is problematic as it seems impossible to get aware of and read the Privacy Statement before I'd take the survey.
(Also, minor issue: "See our privacy policy" should be a link without any "here".)

This is how the survey looks:

Screenshot from 2023-11-16 18-42-44.png (332×314 px, 28 KB)

If I understand correctly, this is problematic as it seems impossible to get aware of and read the Privacy Statement before I'd take the survey.
(Also, minor issue: "See our privacy policy" should be a link without any "here".)

On the QuickSurvey box, can we change "See our privacy policy here" to "Read our privacy statement" and link to https://foundation.wikimedia.org/wiki/Legal:Community_Feedback_Survey_on_Annual_Plan_FY_23-24_Core_Metrics_Reports_Privacy_Statement there?

On the QuickSurvey box, can we change "See our privacy policy here" to "Read our privacy statement" and link to https://foundation.wikimedia.org/wiki/Legal:Community_Feedback_Survey_on_Annual_Plan_FY_23-24_Core_Metrics_Reports_Privacy_Statement there?

Done.

Change 976844 merged by jenkins-bot:

[operations/mediawiki-config@master] Deploy Annual Plan Core Metrics survey

https://gerrit.wikimedia.org/r/976844

Mentioned in SAL (#wikimedia-operations) [2023-11-29T21:20:27Z] <tgr@deploy2002> Started scap: Backport for [[gerrit:976844|Deploy Annual Plan Core Metrics survey (T351353)]]

Mentioned in SAL (#wikimedia-operations) [2023-11-29T21:21:48Z] <tgr@deploy2002> tgr and dani: Backport for [[gerrit:976844|Deploy Annual Plan Core Metrics survey (T351353)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2023-11-29T21:34:14Z] <tgr@deploy2002> Finished scap: Backport for [[gerrit:976844|Deploy Annual Plan Core Metrics survey (T351353)]] (duration: 13m 47s)

Change 983962 had a related patch set uploaded (by DDesouza; author: DDesouza):

[operations/mediawiki-config@master] Undeploy Annual Plan Core Metrics survey

https://gerrit.wikimedia.org/r/983962

@DDeSouza Survey no longer needed. Please remove the survey today. Thank you!

Change 983962 merged by jenkins-bot:

[operations/mediawiki-config@master] Undeploy Annual Plan Core Metrics survey

https://gerrit.wikimedia.org/r/983962

Mentioned in SAL (#wikimedia-operations) [2023-12-19T21:16:08Z] <kharlan@deploy2002> Started scap: Backport for [[gerrit:983962|Undeploy Annual Plan Core Metrics survey (T351353)]]

Mentioned in SAL (#wikimedia-operations) [2023-12-19T21:17:34Z] <kharlan@deploy2002> kharlan and dani: Backport for [[gerrit:983962|Undeploy Annual Plan Core Metrics survey (T351353)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2023-12-19T21:26:09Z] <kharlan@deploy2002> Finished scap: Backport for [[gerrit:983962|Undeploy Annual Plan Core Metrics survey (T351353)]] (duration: 10m 00s)

Change 984287 had a related patch set uploaded (by DDesouza; author: DDesouza):

[operations/mediawiki-config@master] Undeploy Annual Plan Core Metrics beta survey

https://gerrit.wikimedia.org/r/984287

Change #984287 abandoned by DDesouza:

[operations/mediawiki-config@master] Undeploy Annual Plan Core Metrics beta survey

Reason:

Old and I think no longer necessary.

https://gerrit.wikimedia.org/r/984287