Page MenuHomePhabricator

Deploy the cawiki test safety survey to beta cluster
Closed, ResolvedPublic


User Story

Before deploying to production, we want to deploy the survey exactly as it will be in production to beta cluster.

Technical information

In the configuration, since the survey is the same but only the coverage value changes, it would be nice to have the survey configured with coverage 0 in default, and then override the coverage with +cawiki to the defined numbers in T296652: Define initial coverage % for the fawiki and cawiki test safety survey

Testing and QA steps

  • TBD e.g. beta cluster

Acceptance Criteria

Event Timeline

Change 742763 had a related patch set uploaded (by Eigyan; author: Eigyan):

[operations/mediawiki-config@master] WIP: Deploy GDI sur=vey to cawiki and fawiki

Change 742763 merged by jenkins-bot:

[operations/mediawiki-config@master] [beta] Deploy GDI survey to cawiki and fawiki

jsn.sherman renamed this task from Deploy the fawiki and cawiki test safety survey to beta cluster to Deploy the cawiki test safety survey to beta cluster.Dec 13 2021, 5:46 PM
jsn.sherman updated the task description. (Show Details)

@nshahquinn-wmf we have deployed this survey (quicksurvey=internal-gdi-safety-survey) to cawiki on beta:
and were hoping to get a sanity check on data collection. I don't even know where to start on getting eventlog data out of mediawiki projects, and the team said that you could probably check for us.
Could you let us know if responses are getting logged for this?

@jsn.sherman sure! I believe EventLogging works differently on the Beta Cluster, so I'll have to figure out how before I can check the data. I'll do that today.

Looking at this after the break, I believe we moved this into review while we were trying to figure out the what was going on with the coverage
T297623: [SPIKE] QuickSurveys-Coverage - Need to better understand how coverage is handled [4H]+[4H]

and event logging?

Neil Shah-Quinn 24 days ago
I guess I should join the party here!
There's a web UI for monitoring events on the Beta Cluster; I was able to use it to generate a couple test events: Everything there looks accurate.
If any of y'all want to use it for further testing, here's how:
Follow this link to open the UI with the two Quick Survey data streams preselected:,eventlogging_QuickSurveysResponses
Click the green "Stream" button to start monitoring.
Generate the events you're interested in (by answering the survey).
The corresponding events should appear in the web UI within a minute or two.
If you're sharing the results somewhere, note that your IP will appear in the results. You'll probably want to redact that.

Is that right, @Madalina? If so, I believe this is probably ready for us to go back to QA with the cookie clearing steps identified in the spike.

That is correct, it looks like we're good to move it to QA

It looks like we did already set the audience to have minimum edits, which will make QA with a low coverage more difficult.
Coverage uses a cookie, and clearing cookies causes a logout, meaning that the process would be:

1 time:

  • login to ca beta
  • make 5 edits


  • login to ca beta with the qualifying account
  • navigate to a random page
  • if the survey doesn't appear, clear cookies

since the coverage is a probability, this would likely need to be repeated for some multiple of the probability of being selected. Eg, if coverage is 0.1, It's possible that loading a page 10 times with an account that qualifies might not result in it being displayed even if it's working correctly

This is why @eigyan suggested that we just bump the coverage up to 1 (he can correct me if I'm wrong).