Deploy page previews to 90% of users on all wikis but English and German
Closed, ResolvedPublic3 Story Points

Description

Blockers

Plan (YMMV):

Current status

Thursday, 3rd August: The first 50 wikis (in alphabetical order) from the list were deployed to ๐ŸŽ‰๐ŸŽ‰๐ŸŽ‰

Monday, 21st August: Page Previews is currently deployed to all wikis but enwiki and dewiki ๐ŸŽ‰๐ŸŽ‰๐ŸŽ‰

Testing criteria

For each rollout (in sets of 50?), select 5 wikis with different scripts:

For each wiki:

  • Go to the main page of the wiki and hover over a couple of links, ensure previews appear
  • Open a new page by selecting the preview, ensure page opens
  • On new page, ensure previews appear
  • Hover over an article containing disambiguation text, ensure disambiguation text does not appear in preview
  • If the script is rtl, ensure preview appears rtl and the settings cog is placed accordingly

For a single wiki:

  • Disable previews using settings cog, ensure previews are disabled
  • Re-enable previews using link in footer, ensure previews appear again
  • Log in, enable previews, ensure previews appear
  • Disable previews, ensure previews do not appear

Also:

  • Ensure previews are NOT enabled on dewiki and enwiki
  • Ensure previews are enabled on ruwiki

Related Objects

StatusAssignedTask
ResolvedDereckson
ResolvedJdlrobson
Resolvedovasileva
Resolvedovasileva
ResolvedTbayer
ResolvedTbayer
ResolvedTbayer
ResolvedTbayer
ResolvedTbayer
Resolvedphuedx
ResolvedJdlrobson
ResolvedTbayer
DeclinedTbayer
DeclinedTbayer
Resolvedphuedx
ResolvedTbayer
Resolvedpmiazga
ResolvedTbayer
Resolvedphuedx
Resolvedpmiazga
Resolvedphuedx
Resolvedovasileva
Resolvedphuedx
Resolvedphuedx
There are a very large number of changes, so older changes are hidden. Show Older Changes
Jdlrobson updated the task description. (Show Details)Jun 6 2017, 10:06 PM
Jdlrobson reassigned this task from ovasileva to phuedx.

Can you update this task?

Jdlrobson changed the task status from Open to Stalled.Jun 7 2017, 4:57 PM
phuedx updated the task description. (Show Details)Jun 8 2017, 9:35 AM
phuedx updated the task description. (Show Details)
ovasileva updated the task description. (Show Details)Jul 11 2017, 1:25 PM
ovasileva changed the task status from Stalled to Open.
Jdlrobson updated the task description. (Show Details)Jul 11 2017, 6:17 PM
Jdlrobson moved this task from Upcoming to 2016-17 Q4 on the Readers-Web-Backlog board.
phuedx removed phuedx as the assignee of this task.Jul 12 2017, 5:04 PM

Change 365696 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[operations/mediawiki-config@master] Enable page previews for everyone on labs

https://gerrit.wikimedia.org/r/365696

We will have a week's worth of data from tomorrow. Olga and Tilman have a meeting on Friday to confirm we are still good for deploy on Monday.

ovasileva updated the task description. (Show Details)Jul 24 2017, 5:26 PM
phuedx updated the task description. (Show Details)Jul 25 2017, 8:56 AM

@Jdlrobson: Are you happy with the plan thus far? Is the estimate still applicable?

Why wouldn't I be? Can't wait! :)

It's Wednesday! Roll out time?

Not yet. @Tbayer is still working on the analysis. We're looking at Aug 3 or August 7th currently.

phuedx updated the task description. (Show Details)Aug 2 2017, 2:01 PM
Jdlrobson updated the task description. (Show Details)Aug 2 2017, 2:21 PM

Now unblocked based on initial analysis provided by @Tbayer. Let's do this!

phuedx added a comment.Aug 3 2017, 1:28 PM

Now unblocked based on initial analysis provided by @Tbayer. Let's do this!

If possible, could you provide a link to the analysis for posterity?

phuedx updated the task description. (Show Details)Aug 3 2017, 1:35 PM

We'll be publishing it once we've run through the entire set of queries (will provide link then). The decision here in particular was based on a the small negative effects on pageviews (we were not able to reject the null hypothesis that there is no change between the experimental and control groups) as well as on the rate of previews usage of the test wikis.

phuedx updated the task description. (Show Details)Aug 3 2017, 2:54 PM

Change 369960 had a related patch set uploaded (by Phuedx; owner: Phuedx):
[operations/mediawiki-config@master] pagePreviews: Deploy to first 50 of stage 1 wikis

https://gerrit.wikimedia.org/r/369960

Change 351287 merged by jenkins-bot:
[operations/mediawiki-config@master] pagePreviews: Tidy up config

https://gerrit.wikimedia.org/r/351287

Change 351286 abandoned by Phuedx:
pagePreviews: Create pp_stage0.dblist

Reason:
Superseded by I505e9100.

https://gerrit.wikimedia.org/r/351286

Change 369960 merged by jenkins-bot:
[operations/mediawiki-config@master] pagePreviews: Deploy to first 50 of stage 1 wikis

https://gerrit.wikimedia.org/r/369960

I have some concerns regarding the time line of the roll-out. As per @Jdlrobson (from IRC):

jdlrobson: all wikis but en and de will be out within next 2 weeks
jdlrobson: next one monday

While the page summary API is cached really well (>95% of responses are served from Varnish) and the load induced upon RESTBase by Page Previews is not significant, I feel like it would be best for all parties involved if we coordinate the complete roll out a bit better. Namely, the problem is that next week both Wikimania and DebConf are happening, which means very reduced resources on both the Ops and Services teams. In addition, Aug 21st - Aug 25th is the Services team offsite. I am proposing to slightly modify the schedule:

  • no deploys the week of Aug 7th (next week)
  • all but the big five go out in the week of Aug 14th
  • gradually roll out the big five starting from in the week of Aug 28th.

Thoughts? Suggestions? Are there any constraints on the Readers team side that warrant the roll out to happen sooner?

It depends on the level of technical concerns we have. If they are low, I think it makes sense to proceed with the rollouts as planned. Perhaps the next chunk can contain 1-2 or the larger wikis so we can gauge performance. If any red flags occur, we can roll back and examine a more relaxed schedule. My main concern is that we've informed all of our communities about the rollouts (twice) so they are expecting them sooner or later. Also, as we approach the English and German communities during the upcoming A/B test and begin discussions about rollout, it's helpful for us to be able to know that the feature is ready and rolled out on other larger wikis without issues.

I am not worried about the performance side of things. As stated, we have enough capacity in Varnish + RB to support switching all of the wikis right now without suffering performance issues. I am more concerned about the actual content being served/displayed and its correctness. Not all wikis are the same, and it is highly probable that some new corner cases will be discovered, which may warrant assistance from us (purging/content regeneration/etc). And with reduced human power next week (and focus on other things) I wonder if that is a good idea.

That said, if these conditions are acceptable to you (delayed response and action from our side), I will not object or stop you :)

phuedx updated the task description. (Show Details)Aug 4 2017, 10:50 AM
phuedx added a comment.Aug 7 2017, 7:32 PM

IMO it'd be unnecessarily risky to continue with the deployments with limited support from Services even if it costs us 7 days (see T162672#3498422). We're currently rolled out to 57 wikis, including ruwiki, which we can point to when reaching out to enwiki and dewiki. That we've deployed to those wikis relatively pain-free, doesn't mean we won't need support with the 240 others ๐Ÿ™‚

phuedx added a comment.Aug 8 2017, 5:15 PM

@ovasileva to add QA steps if she's unavailable during a deploy.

For visibility, this will remain in Blocked until Monday, 14th August per T162672#3498422.

ovasileva updated the task description. (Show Details)Aug 12 2017, 6:31 PM

testing steps added in description

Jdlrobson assigned this task to phuedx.

We plan to do the rest of the deploy tomorrow.
@phuedx will SWAT to 100 wikis at 1pm UTC and I will SWAT to the final wikis at 6pm.

Change 372137 had a related patch set uploaded (by Phuedx; owner: Phuedx):
[operations/mediawiki-config@master] pagePreviews: Deploy to next 100 stage 1 wikis

https://gerrit.wikimedia.org/r/372137

Change 372137 merged by jenkins-bot:
[operations/mediawiki-config@master] pagePreviews: Deploy to next 100 stage 1 wikis

https://gerrit.wikimedia.org/r/372137

Mentioned in SAL (#wikimedia-operations) [2017-08-16T13:35:05Z] <zfilipin@tin> Synchronized dblists/pp_stage1.dblist: SWAT: [[gerrit:372137|pagePreviews: Deploy to next 100 stage 1 wikis (T162672)]] (duration: 00m 50s)

phuedx removed phuedx as the assignee of this task.

Change 372160 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[operations/mediawiki-config@master] Roll page previews out to all wikis except en and de wiki

https://gerrit.wikimedia.org/r/372160

This appears to be blocked on T173422

The remaining wikis with the exception of zh are small, so I think it's safe to roll this out to remaining wikis. Updates on T173422 do not seem alarming right now.

Change 372160 merged by jenkins-bot:
[operations/mediawiki-config@master] Roll page previews out to all wikis except en and de wiki

https://gerrit.wikimedia.org/r/372160

Mentioned in SAL (#wikimedia-operations) [2017-08-21T18:40:02Z] <niharika29@tin> Synchronized dblists/pp_stage1.dblist: Roll page previews out to all wikis except en and de wiki T162672 (duration: 00m 44s)

phuedx updated the task description. (Show Details)Aug 22 2017, 8:28 AM
phuedx assigned this task to ovasileva.

Over to you, @ovasileva!

ovasileva closed this task as Resolved.Aug 24 2017, 1:42 PM

Tested on eswiki, tgwiki, viwiki, jawiki and confirmed functionality to work as expected. ๐Ÿ˜„