Deploy is scheduled for Monday 9th April

Sorry, is this a typo of some sort? It's not possible to do a full consultation and gather consensus from the community to enable the page previews feature in a single week. (Related: comments on T182317 .)

Also, could someone please clarify exactly what the relationship is between this task and its two parent tasks? Thanks.

@Yair_rand - this deployment only refers to behavior for logged-out users. We will be starting a consultation for the behavior for editors and other logged-in users tomorrow but are not planning on implementing any decisions from the consultation as a part of this task.

could someone please clarify exactly what the relationship is between this task and its two parent tasks? Thanks.

This is the task to actually make the config change (Wikimedia-Site-requests tag). The other tasks are the goal and epic cards.

@ABorbaWMF did you mean to paste that here on the deploy task? If so could you please add some context by what "language tests" means ? Thanks!

Premature testing, sorry about that

> Verify that Page previews is showing on all wikipedias - I suggest testing Spanish, German, Hebrew and English to verify

I was testing this one. Just figured I would test the top ten.

In T191101#4097920, @ovasileva wrote:

@Yair_rand - this deployment only refers to behavior for logged-out users.

A week isn't long enough to gather consensus for enabling it for logged-out users either. There isn't a separate shorter process for different classes of users, and limiting it to logged-out users doesn't make it okay to defy consensus.

There's a few ways this could go:

• The deployment to logged-out users happens against consensus. An admin quickly implements the already-made community decision to not have Page Previews, by adding .mwe-popups { display: none !important; } to Common.css. Either the staff decides not to take action against that, or they do and the board overrides that decision, as many of its members were elected/selected specifically on the platform that they would stop the WMF from doing such things in the aftermath of the superprotect crisis. The CSS solution makes Page Previews no longer available for anyone, even those who want to opt in. The relationship between the community and the WMF is heavily damaged.
• The deployment happens against consensus and the board does not veto a new superprotect. The community's ability to manage local affairs is damaged from all JS/CSS/gadgets becoming uneditable, the software develops loads of new problems coming out of the inevitable fundamental issues with trying to prevent admins from doing things on Mediawiki. Trust is gone, and the WMF and community get stuck in a conflict as they spend much of their time trying to stop the other from doing any more damage. Board elections come around, and the community either succeeds or fails in fixing both the board and the WMF as a whole.
• The deployment happens against consensus, the community has an immediate shift of opinion on whether it should be deployed, and a consensus that supports the deployment for not-logged-in users. No action is taken against the deployment, and we have the feature running several weeks earlier than we would have had if we waited for consensus first. Also, trust in the WMF goes down because they defied consensus as it was at the time.
• Consensus changes before deployment happens, as a result of the discussion on whether to enable for logged-in users. Similar results.

These sound like a terrible set of outcomes. Please reconsider this.

Hi @Yair_rand: Thanks for raising the alam. You're right, we don't want any of those outcomes, and there's no superprotect coming, on this or any other product release.

Olga's going to post tomorrow on the village pump about the work that the team's done to improve the feature, and our plans to continue deploying the feature. Our intention is to continue the conversation with the community. Chris posted a message on VP technical last month:

https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(technical)/Archive_164#Improvements_to_Page_Previews

This didn't get any response, so we're hoping that tomorrow's post will get more people involved.

Olga's going to include a list of the problems that people reported last time we talked, and how those problems were addressed and fixed. We want to make sure that we've cleared up the things that people were concerned about.

Once that conversation starts, we want to know what people think, especially if there's a problem. I'm glad that you care about this -- we could use some help making sure people see our post. We don't usually start RfCs, but if there is one, then obviously we'll want to participate and be a part of the discussion. Do you have any advice?

Olga's posted on Village pump miscellaneous:

https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(miscellaneous)#Page_Previews

German version of Olga's post: https://de.wikipedia.org/wiki/Wikipedia:Fragen_zur_Wikipedia#Seitenvorschaubilder_(Page_Previews)

Change 425588 had a related patch set uploaded (by Pmiazga; owner: Pmiazga):
[operations/mediawiki-config@master] Enable Page Previews for 10% enwiki anon users

https://gerrit.wikimedia.org/r/425588

^ Above patch is just to check we can cope with the added load in EventLogging. Further deploys will be dependent on the community consensus (see blockers)

Change 427082 had a related patch set uploaded (by Pmiazga; owner: Pmiazga):
[operations/mediawiki-config@master] Enable PagePreviews for 25% anon users on enwiki

https://gerrit.wikimedia.org/r/427082

Change 427082 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable PagePreviews for 25% anon users on enwiki

https://gerrit.wikimedia.org/r/427082

Mentioned in SAL (#wikimedia-operations) [2018-04-17T12:04:31Z] <mobrovac@tin> Synchronized wmf-config/InitialiseSettings.php: Enable page previews for 25% of annons on enwiki - T191101 (duration: 01m 03s)

Mentioned in SAL (#wikimedia-operations) [2018-04-17T12:31:44Z] <mobrovac@tin> Synchronized wmf-config/InitialiseSettings.php: Enable page previews for 25% of annons on enwiki, take #2 - T191101 (duration: 00m 58s)

Change 427103 had a related patch set uploaded (by Pmiazga; owner: Pmiazga):
[operations/mediawiki-config@master] Enable PagePreviews for 50% anon users on en wiki

https://gerrit.wikimedia.org/r/427103

Communities have been informed prior to deployment. T182317

The community discussion is still open and we’ll continue having the conversation if any new comments come in, but it’s been quiet for the past couple of days. So far, we haven’t heard any direct concerns around the changes proposed above.

Change 427103 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable PagePreviews for 50% anon users on en wiki

https://gerrit.wikimedia.org/r/427103

Mentioned in SAL (#wikimedia-operations) [2018-04-17T14:30:43Z] <mobrovac@tin> Synchronized wmf-config/InitialiseSettings.php: Enable page previews for 50% of anons for enwiki - T191101 (duration: 00m 58s)

Change 427137 had a related patch set uploaded (by Pmiazga; owner: Pmiazga):
[operations/mediawiki-config@master] Enable PagePreviews for 75% anon users on en wiki

https://gerrit.wikimedia.org/r/427137

Did not see any remaining blockers from my perspective. (Notice though that the work on measuring the usage of this new feature is still in progress, see T186728 etc.)

Change 427137 merged by Mobrovac:
[operations/mediawiki-config@master] Enable PagePreviews for 75% anon users on en wiki

https://gerrit.wikimedia.org/r/427137

Mentioned in SAL (#wikimedia-operations) [2018-04-17T16:02:22Z] <mobrovac@tin> Synchronized wmf-config/InitialiseSettings.php: Enable page previews for 75% of anons for enwiki - T191101 (duration: 00m 58s)

Change 427178 had a related patch set uploaded (by Pmiazga; owner: Pmiazga):
[operations/mediawiki-config@master] Enable PagePreviews to all anonymous users

https://gerrit.wikimedia.org/r/427178

Change 427178 merged by Mobrovac:
[operations/mediawiki-config@master] Enable PagePreviews to all anonymous users

https://gerrit.wikimedia.org/r/427178

Mentioned in SAL (#wikimedia-operations) [2018-04-17T17:16:49Z] <mobrovac@tin> Synchronized wmf-config/InitialiseSettings.php: Enable page previews for 100% of anons for enwiki - T191101 (duration: 00m 59s)

There have been some issues today.

We have been deploying progressively using the A/B testing frontend code, and the wgPopupsAnonsExperimentalGroupSize. As such, we have been mistaken in the amount of users we have deployed the feature to. See in the code, how experimentGroupSize is divided by 2, and that will be the % used for the on group.

Basically, if wgPopupsAnonsExperimentalGroupSize is at 0.5, then 25% of the anon users will get Popups enabled, so we have been deploying to half the users we were expecting through the day.

At the "75%" deploy we noticed the numbers didn't check out and did the investigation.

The maximum rollout we could do with this mechanism is 50% on.

Given the window and the train schedule, Nuria, Marco, Piotr, Olga and myself decided to go 100% (disabling the A/B test) and checking the logs and stats before the train, to have the chance to roll back if needed.

There are a few lessons in the story, one of them being: do not use A/B testing code to do progressive rollouts, and if you really do, test and double check your progressive rollouts.

Given the comment above we think the ratio of events (discounting lauch traffic do to "artifical" usage of feature) is goinmg to be about 1200 reqs/sec

Error rate still at 0

Smoke test results thus far on Chrome, Safari, Edge, and Firefox:
https://docs.google.com/spreadsheets/d/1CdHhFDoTOfbLDCuwVViAWSFlduNE88h3mCbSKIRbKcs/edit?usp=sharing

Sheet is called "Smoke Test - Previews - April 17"

On the back-end side things are looking good. After the initial increase in number of requests that caused Varnish misses, we are back to normal levels, indicating that the majority of requests are served from cache. In terms of errors things are looking good too: no noticeable changes in error rates in parsing/serving summaries.

Request ratio peak is about this time, looks like things are all good.

removing " Parent tasks- both goal and epic are resolved." from the acceptance criteria. Let's go ahead an resolve the epic, but keep T191888: Change page previews configuration for opt-in accounts under the goal card just so that everything is in one place.

@Jdlrobson - do we still need an epic for post-deploy cleanup?

^ Twitter went a bit crazy so we can consider that checkbox done :)

In T191101#4142016, @ovasileva wrote:

@Jdlrobson - do we still need an epic for post-deploy cleanup?

I've opened up T192622 to continue this journey!