⚓ T200793 Disable page issues A/B test

	Subject	Repo	Branch	Lines +/-
	Disable page issues A/B test	operations/mediawiki-config	master	+0 -5

Status	Assigned	Task
Resolved	Jdlrobson	T147641 [EPIC] Improve top of the article user experience - mobile
Resolved	Jdlrobson	T143535 [EPIC] Improve article notes
Resolved	ovasileva	T159262 [EPIC] Improve page issues
Resolved	• alexhollender_WMF	T210553 Deploy page issues to all wikipedias (except enwiki)
Duplicate	None	T201975 Make page issue and hatnote classes names configurable so we are not English-Wikipedia centric
Resolved	• Niedzielski	T211257 Split pageIssues.js into smaller functions
Resolved	ovasileva	T210554 Deploy page issues to enwiki and all remaining projects
Resolved	ovasileva	T200794 Analyze results of page issues A/B test
Resolved	phuedx	T200793 Disable page issues A/B test
Resolved	• Tbayer	T200792 [EPIC] Run A/B test on page issues (Farsi, Japanese, Russian, English)
Resolved	ovasileva	T191532 Mobile page issues - instrument page issues
Resolved	• Niedzielski	T201124 Provide standard/reproducible way to access a PageToken
Resolved	• Tbayer	T202098 Log sectionNumbers in PageIssues schema
Resolved	• Tbayer	T202651 Section number in issues is reported incorrectly.
Resolved	• Tbayer	T202940 [subtask] Various page issues events sending NaN for sectionNumbers in old treatment
Resolved	• Tbayer	T203050 sectionNumbers and issuesSeverity should always be consistent in length
Resolved	ovasileva	T202786 [subtask] Edits via the main edit icon are not triggering Schema:Edit or Schema:PageIssues events
Resolved	• Tbayer	T203725 Icon correct but issuesSeverity in instrumentation is wrong
Resolved	phuedx	T193584 Prepare feature flagging gateway for mobile issues work
Resolved	ovasileva	T198113 QA mobile page issues modal
Resolved	Jdlrobson	T203802 Latvian Wikipedia amboxes should be compatible with Minerva's
Resolved	Jdlrobson	T204144 ReadingDepth sometimes initialises before PageIssues leading to incorrect or missing ReadingDepth events
Resolved	• Tbayer	T204143 ReadingDepth events are not being sent in browsers where navigator.sendBeacon should be supported but in practice isn't
Resolved	• Tbayer	T204609 Turn on page issues A/B test for Latvian Wikipedia, and conduct data checks
Resolved	• Niedzielski	T205355 A/B config flag should be subject to ResourceLoader caching rules not HTML caching rules
Declined	None	T204613 Turn A/B test for page issues on for test projects at low rates to validate instrumentation
Resolved	ovasileva	T204090 QA page issues feature

ovasileva triaged this task as Medium priority.Jul 31 2018, 1:16 PM

ovasileva created this task.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 31 2018, 1:16 PM

ovasileva changed the task status from Open to Stalled.Jul 31 2018, 1:17 PM

ovasileva updated the task description. (Show Details)

ovasileva added a subtask: T200792: [EPIC] Run A/B test on page issues (Farsi, Japanese, Russian, English).

ovasileva added a parent task: T200794: Analyze results of page issues A/B test.Jul 31 2018, 2:26 PM

Jdlrobson moved this task from Incoming to Needs Prioritization on the Web-Team-Backlog board.Aug 16 2018, 9:36 PM

ovasileva changed the task status from Stalled to Open.Oct 16 2018, 3:56 PM

ovasileva updated the task description. (Show Details)Oct 16 2018, 4:11 PM

ovasileva set the point value for this task to 1.

ovasileva moved this task from Needs Prioritization to Upcoming on the Web-Team-Backlog board.

@ovasileva, @Jdlrobson, @Tbayer, I believe this is now scheduled for next week but I'm not sure if anyone will be free to work on it prior to deployment.

In T200793#4692160, @Niedzielski wrote:

@ovasileva, @Jdlrobson, @Tbayer, I believe this is now scheduled for next week but I'm not sure if anyone will be free to work on it prior to deployment.

Yes (see also T200792#4707989) - I updated this and other tasks.

matmarex subscribed.Oct 30 2018, 11:01 PM

Jdlrobson edited projects, added Web-Team-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q2); removed Web-Team-Backlog.Oct 30 2018, 11:26 PM

• nray claimed this task.Oct 31 2018, 3:38 PM

• nray moved this task from To Do to Doing on the Web-Team-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q2) board.

Change 470868 had a related patch set uploaded (by Nray; owner: Nick Ray):
[operations/mediawiki-config@master] Disable page issues A/B test

https://gerrit.wikimedia.org/r/470868

gerritbot added a project: Patch-For-Review.Oct 31 2018, 4:42 PM

@ovasileva

On the patch, Jon stated:

"We need to double check the date of disabling with Olga (says 2nd November but not sure if we can do this on 1st - the last deployment window of the week)."

Can you please clarify?

In T200793#4710976, @nray wrote:

@ovasileva

On the patch, Jon stated:

"We need to double check the date of disabling with Olga (says 2nd November but not sure if we can do this on 1st - the last deployment window of the week)."

Can you please clarify?

In that case next week would be better. As explained at T200792#4707989, the idea is now to have four weeks of data at the current sampling ratios. Those were put in place on Oct 3 (T200792#4640570 ), and allowing 1 day for caching (actually a bit on the low side) I have been using the data from Oct 5 on in the analyses so far. That means the four weeks include tomorrow, Nov 1, and the test should be deactivated after that.