Page MenuHomePhabricator

Create survey for TechWish prototype announcements on dewiki and metawiki
Closed, ResolvedPublic0 Estimated Story Points

Description

Surveys to be added to the template prototype announcement pages:
TW metawiki
TW on dewiki

We agreed to make a custom tool which can be embedded into the page sections related to the survey questions. It might use aspects of the QuickSurvey extension but will function a bit differently.

The survey should:

  • be shown to all users who visit these only these two pages
  • allow for checkbox responses where users can select multiple answers
  • allow for multiple surveys on one page, with different page sections having separate questions

Ideally, after answers are selected and submitted, users would get a short thank you for your feedback message.

Note: we should clearly communicate that this is not a vote but that we are collecting feedback on perceived usefulness of different proposals.

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 19 2020, 10:09 AM

Implementation notes: we can collect responses using EventLogging. Since we'll depend on Javascript, the default style should hide the form, it will be shown if JS is active and Do No Track is false. Once a user responds to the survey, we might want to either hide it, or show an indication that they've already responded. To deduplicate responses, some kind of user or session identifier can be included (take the final response).

Make sure that pageviews data is available for any page with a survey on it, so we can estimate the proportion of viewers who responded.

Lena_WMDE set the point value for this task to 3.May 20 2020, 7:51 AM

@ecohen

Ideally, after answers are selected and submitted, users would get a short thank you for your feedback message.

This will be JS-only, so we can show a popup with the thank-you. Or were you imagining redirecting to a separate page? Or maybe even replacing the survey content with a thank-you message?

We also need to make decisions about how to handle users without JS, and those with an "already responded" cookie. My preference so far is that the embedded survey disappears for both of these groups. Alternatively, the content would be replaced with some kind of placeholder, dimmed with an overlap explaining why it's not activated... that kind of thing.

  1. Yes I was thinking about replacing the content with a thank-you message. I can put something quickly together for what that could look like.
  1. I think having the placeholder might be the easiest way to make sure that the feedback section still makes sense if it's referencing the survey but the survey isn't visible. For no-JS, it could show a message saying JS must be activated in order to use the survey. For those with the cookie, it could still display the "Thank you for sharing your input" message. What do you think?

A few minor questions after looking at the code:

  • Do we agree to keep the "shuffle" feature, which shows the answers in random order to each user? There's probably a lot of bias without this.
  • Geolocated country code is bundled in the response. That should be removed, I don't think we want to collect it?
awight removed awight as the assignee of this task.May 20 2020, 12:24 PM
awight updated the task description. (Show Details)
awight added a subscriber: awight.

Try the prototype live:
Install this userscript on your beta dewiki User:<user>/common.js page, for example:

mw.loader.load('//de.wikipedia.beta.wmflabs.org/w/index.php?title=Benutzer:Adamw/survey.js&action=raw&ctype=text/javascript');

Visit this page, which includes the injectable <div>s,
https://de.wikipedia.beta.wmflabs.org/wiki/Benutzer:Adamw/Leichter_mit_Vorlagen_arbeiten#Prototyp_1:_Suchfilter_f%C3%BCr_ausgezeichnete_Vorlagen

Completed:

  • Suppressed with custom message when Do Not Track is enabled.
  • URL query parameter "forcesurvey=true" overrides other tests.
  • Hide with thank-you message when the user already submitted a response, per survey.
  • Multiple surveys are supported per page
  • Injection replaces an existing <div> by ID, for each survey.
  • Responses sent to existing QuickSurveyResponses topic in EventLogging.
  • Page impressions sent to existing QuickSurveyInitiation topic in EventLogging.

Remaining work:

  • Include injection divs where we want surveys.
  • Store the Javascript as a publicly accessible, permissioned module. Will require site interface admin rights.
  • Include the Javascript from pages where surveys appear.
  • Write survey configuration for each survey, decide where this should live. Note this may also require site interface admin rights.
  • Populate all i18n messages for en and de, on each site or globally:
    • wmde-tw-survey-thank-you-notice
    • wmde-tw-template-survey-dnt-notice
    • All questions, description, and answers for individual surveys, e.g. wmde-tw-template-survey-prototype1-question
awight added a comment.EditedMay 20 2020, 3:18 PM

Mildly legal thing, we should probably link to our own privacy policy and not WMF's.

@Lena_WMDE @Tobi_WMDE_SW Is there a standard text snippet for this?

awight added a comment.EditedMay 28 2020, 6:48 AM

Currently, I'm imagining we create a new MediaWiki-namespace page for this and transclude it directly from our pages, but I'd like to take a more careful look at precedents.

It would probably be best for one of our team members to request temporary Interface admin rights, in case we need to rapidly fix bugs and so on.

I wonder why "do not track" shows up here? As far as I understand, we don't want to track users, but collect anonymized answers, don't we? Furthermore, we do not collect any data when the user does not interact with the survey. It's up to the user to decide if they want to give us data or not. As it is described, "do not track" does not apply in such a situation.

The main reason for my concern is that certain web browsers have – as far as I know – "do not track" enabled by default nowadays, which means we might exclude a lot of users, if not the majority.

That's a really good point. I think it was a carry over from the QuickSurvey extension, but if we are not tracking them or collecting any information then maybe it's not necessary. I agree getting it in front of as many users as possible should be the goal and this could limit it significantly.

awight added a comment.Jun 2 2020, 7:46 AM

Embedding JS

Here's the main precedent for adding JS to specific pages: https://www.mediawiki.org/wiki/Adding_JavaScript_to_Wiki_Pages . This suggests that we add code linked from common.js, which will inject itself only when a matching element is present on the page. I didn't find any examples of a page transcluding directly from JS in the MediaWiki namespace, it's probably impossible now that I think about it. The JS would be rendered as text, and any HTML escaped.

QuickSurveys integration

After discussing my plan in a preliminary technical review, we decided that it's better to deploy the code via an existing extension rather than as ad-hoc Javascript on a wiki page. I made the unilateral decision to take this logic a step further, and have attempted to integrate into QuickSurveys itself rather than shoehorning the script into a random, unrelated extension. This may have been overly ambitious, but the code is ready to go--remaining obstacles will mostly be social and organizational, since we haven't communicated with the QS maintainers ahead of time.

awight added a comment.EditedJun 2 2020, 8:35 AM

Do not track

I wonder why "do not track" shows up here? As far as I understand, we don't want to track users, but collect anonymized answers, don't we? Furthermore, we do not collect any data when the user does not interact with the survey. It's up to the user to decide if they want to give us data or not. As it is described, "do not track" does not apply in such a situation.

+1 @thiemowmde and @ecohen, DNT behavior was carried over from QuickSurveys, where they suppress everything when the header is present. My personal interpretation is that we would be following the spirit of DNT to suppress the initial impression tracking, but allow the survey to show and collect data if voluntarily submitted. My guess about why QuickSurveys defaults to suppress everything is that random sampling relies on having an accurate measure for the response rate, which becomes more difficult if an unknown number of users don't report the impression. But there could have also been a sense that showing random surveys is against the spirit of DNT.

There are some precedents in code, but they're mixed:

  • CentralNotice will show banners to DNT users, but won't track the history of banners seen.
  • QuickSurveys suppresses everything.
  • WikiLovesMonuments disables cookies.
  • EventLogging recently (yesterday) stopped honoring the header. See below...

Importantly, the WMF privacy policy states that DNT will not change server behavior, because we use discretion for all users, so there's no privacy invasion to prevent. There was a lengthy discussion in T252438, which resulted in a decision to *not* modulate any behaviors based on DNT, because we're "almost" compliant with the spirit of DNT for all users, so opt-out wouldn't add anything more.

This is a bit disappointing to me, but I think we should follow this precedent, especially in our case where the embedded survey is presented 100% of the time (no targeting) and participation is purely opt-in.

However... now that I've integrated with QuickSurveys we're stuck following the decisions already made in that codebase, and making a DNT change would probably be a significant barrier to merging our form. What I'll do is to disable DNT in a separate patch, and try to isolate the discussions. What it means for us is that we might launch the survey with DNT users excluded, and then at a later date will start to include them, granted the QS maintainers agree with this logic.

awight added a comment.Jun 2 2020, 8:52 AM

However... now that I've integrated with QuickSurveys we're stuck following the decisions already made in that codebase, and making a DNT change would probably be a significant barrier to merging our form. What I'll do is to disable DNT in a separate patch, and try to isolate the discussions.

This discussion has been split out to T254224: Remove Do Not Track support for QuickSurveys.

Change 601696 had a related patch set uploaded (by Awight; owner: Awight):
[mediawiki/extensions/QuickSurveys@master] Gracefully degrade when survey configuration is invalid

https://gerrit.wikimedia.org/r/601696

awight triaged this task as High priority.Jun 2 2020, 11:37 AM
awight changed the point value for this task from 3 to 8.
awight added a comment.Jun 3 2020, 7:52 AM

@ecohen I keep forgetting to mention a major drawback of the implementation I've chosen: it will be difficult to support multiple surveys per page. Is it feasible and not too terrible to split our pages up so that only one survey appears on each?

ecohen added a comment.Jun 3 2020, 7:58 AM

@awight Oh, that is a major drawback. Not really, it would need to be multiple surveys on the same page. Is it because of the attachment to the QuickSurvey ext?

@Lena_WMDE What do you think? Should I bring it up with Michi about splitting the page? It is a lot of content on one page, but I think it's kind of too late. And also seems important that all of the feedback related to our template prototypes is on one single talk page.

I also think in order for the effort being putting now to be most useful in the future, this would be an important feature to have.

awight added a comment.Jun 3 2020, 8:24 AM

Is it because of the attachment to the QuickSurvey ext?

Exactly, the extension is currently designed to deliver zero or one random surveys. I'll start looking at a code fix for this, it should be possible to change the selection code without affecting anything else...

Change 602283 had a related patch set uploaded (by Awight; owner: Awight):
[mediawiki/extensions/QuickSurveys@master] Split SurveyFactory into validation and builder

https://gerrit.wikimedia.org/r/602283

Change 599809 had a related patch set uploaded (by Awight; owner: Awight):
[mediawiki/extensions/QuickSurveys@master] [WIP] Add a new survey configuration for embedding in a page

https://gerrit.wikimedia.org/r/599809

Change 602372 had a related patch set uploaded (by Awight; owner: Awight):
[mediawiki/extensions/QuickSurveys@master] [WIP] Multiselect answer layout

https://gerrit.wikimedia.org/r/602372

Change 602283 merged by jenkins-bot:
[mediawiki/extensions/QuickSurveys@master] Split SurveyFactory into validation and builder

https://gerrit.wikimedia.org/r/602283

Change 601696 merged by jenkins-bot:
[mediawiki/extensions/QuickSurveys@master] Disable and log surveys with invalid configuration

https://gerrit.wikimedia.org/r/601696

awight changed the point value for this task from 8 to 0.

Change 605616 had a related patch set uploaded (by Awight; owner: Awight):
[operations/mediawiki-config@master] [beta] Don't shuffle answers for demo survey

https://gerrit.wikimedia.org/r/605616

Change 605616 merged by jenkins-bot:
[operations/mediawiki-config@master] [beta] Don't shuffle answers for demo survey

https://gerrit.wikimedia.org/r/605616

Change 605619 had a related patch set uploaded (by Awight; owner: Awight):
[operations/mediawiki-config@master] [beta] Fix enum value

https://gerrit.wikimedia.org/r/605619

Change 605619 merged by jenkins-bot:
[operations/mediawiki-config@master] [beta] Fix enum value

https://gerrit.wikimedia.org/r/605619

Change 605854 had a related patch set uploaded (by Awight; owner: Awight):
[operations/mediawiki-config@master] [beta] Update survey with real questions and answers

https://gerrit.wikimedia.org/r/605854

Change 605854 merged by jenkins-bot:
[operations/mediawiki-config@master] [beta] Update survey with real questions and answers

https://gerrit.wikimedia.org/r/605854

Now with a generic privacy policy link.

Hm the links are working for me? I'm not seeing a survey, even in an incognito tab so I don't think it's cookies?

@ecohen Can you go into browser localstorage and delete the survey entries?

Change 607762 had a related patch set uploaded (by Awight; owner: Awight):
[operations/mediawiki-config@master] [beta] Enable mobile view for dewiki survey

https://gerrit.wikimedia.org/r/607762

Change 607762 merged by jenkins-bot:
[operations/mediawiki-config@master] [beta] Enable mobile view for dewiki survey

https://gerrit.wikimedia.org/r/607762

Change 607763 had a related patch set uploaded (by Awight; owner: Awight):
[operations/mediawiki-config@master] Enable WMDE Tech Wishes survey configuration

https://gerrit.wikimedia.org/r/607763

Change 607763 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable WMDE Tech Wishes survey configuration

https://gerrit.wikimedia.org/r/607763

Mentioned in SAL (#wikimedia-operations) [2020-06-25T11:38:54Z] <awight@deploy1001> Synchronized wmf-config/InitialiseSettings.php: BACON: [[gerrit:607763|Enable WMDE Tech Wishes survey configuration (T253112)]] (duration: 01m 09s)

Change 607767 had a related patch set uploaded (by Awight; owner: Awight):
[operations/mediawiki-config@master] Enable QuickSurveys on metawiki

https://gerrit.wikimedia.org/r/607767

Change 607767 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable QuickSurveys on metawiki

https://gerrit.wikimedia.org/r/607767

Mentioned in SAL (#wikimedia-operations) [2020-06-25T11:49:59Z] <awight@deploy1001> Synchronized wmf-config/InitialiseSettings.php: BACON: [[gerrit:607767|Enable QuickSurveys on metawiki (T253112)]] (duration: 01m 05s)

Change 607773 had a related patch set uploaded (by Awight; owner: Awight):
[operations/mediawiki-config@master] Enable TechWishes survey for dewiki

https://gerrit.wikimedia.org/r/607773

awight updated the task description. (Show Details)

Change 608278 had a related patch set uploaded (by Awight; owner: Awight):
[operations/mediawiki-config@master] Configure TeWü survey on dewiki

https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/ /608278

Change 608278 merged by jenkins-bot:
[operations/mediawiki-config@master] Configure TeWü survey on dewiki

https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/ /608278

Change 608236 had a related patch set uploaded (by Awight; owner: Awight):
[operations/mediawiki-config@master] Revert "Configure TeWü survey on dewiki"

https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/ /608236

Change 608236 merged by jenkins-bot:
[operations/mediawiki-config@master] Revert "Configure TeWü survey on dewiki"

https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/ /608236

Mentioned in SAL (#wikimedia-operations) [2020-06-30T11:32:40Z] <awight@deploy1001> Synchronized wmf-config/InitialiseSettings.php: BACON: [[gerrit:608478|Configure TeWü survey on dewiki (take 2) (T253112)]] (duration: 00m 58s)

Change 607773 abandoned by Awight:
[operations/mediawiki-config@master] Enable TechWishes survey for dewiki

Reason:
done in another patch

https://gerrit.wikimedia.org/r/607773

Lena_WMDE closed this task as Resolved.Tue, Jul 7, 11:40 AM
Lena_WMDE claimed this task.
Lena_WMDE moved this task from Demo to Done on the WMDE-QWERTY-Sprint-2020-06-24 board.