Page MenuHomePhabricator

Stop page previews A/B test on enwiki and dewiki
Closed, ResolvedPublic1 Story Points

Description

As discussed earlier this week, we plan to use a SWAT deploy on Monday the 18th to stop this test for now, now that the planned two weeks are complete.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 16 2017, 11:48 PM

@Tbayer, @phuedx - are we sure we're ready for this given what we've seen in T175918: EventLogging subscriber module in ready state but not sending tracked events - or is Firefox safe from that bug? (in which case, let's go ahead as planned)

Change 378686 had a related patch set uploaded (by Phuedx; owner: Phuedx):
[operations/mediawiki-config@master] pagePreviews: Stop A/B test on enwiki and dewiki

https://gerrit.wikimedia.org/r/378686

Change 378686 merged by jenkins-bot:
[operations/mediawiki-config@master] pagePreviews: Stop A/B test on enwiki and dewiki

https://gerrit.wikimedia.org/r/378686

Mentioned in SAL (#wikimedia-operations) [2017-09-18T13:18:52Z] <hashar@tin> Synchronized wmf-config/InitialiseSettings.php: pagePreviews: Stop A/B test on enwiki and dewiki - T176068 (duration: 00m 45s)

I can confirm that:

  • The A/B test is no longer running on dewiki and enwiki.
  • The Page Previews beta feature has been enabled on dewiki and enwiki.
  • The Page Previews EventLogging instrumentation has been disabled on all wikis.

@Tbayer, @phuedx - are we sure we're ready for this given what we've seen in T175918: EventLogging subscriber module in ready state but not sending tracked events - or is Firefox safe from that bug? (in which case, let's go ahead as planned)

Given the additional strain that we've put on the EventLogging pipeline during the last two weeks (a peak rate of 19.32K events/min!!1), I felt that we stop logging events at the pre-agreed time. If we conclude that T175918: EventLogging subscriber module in ready state but not sending tracked events has impacted the data that we've collected, then we should re-run the A/B test with a fix in place.

ovasileva triaged this task as High priority.Sep 18 2017, 2:25 PM
phuedx set the point value for this task to 1.Sep 18 2017, 2:31 PM

1 as it was a SWAT deploy.

phuedx closed this task as Resolved.EditedSep 18 2017, 2:32 PM

@Tbayer, @phuedx - are we sure we're ready for this given what we've seen in T175918: EventLogging subscriber module in ready state but not sending tracked events - or is Firefox safe from that bug? (in which case, let's go ahead as planned)

Just to add to @phuedx's explanation: T175918 unfortunately affects Firefox too according to the task description there; and on the other hand, the test was already planned to end this weekend before that bug was discovered. As he said, it look like we may need to fix T175918 first.