Page MenuHomePhabricator

Deploy Wikidiff2 version 1.8.2 with the timeout issue fixed
Closed, ResolvedPublic2 Estimated Story Points

Description

To do:

Event Timeline

Restricted Application added a subscriber: Aklapper. ยท View Herald TranscriptMay 15 2019, 4:40 PM
thiemowmde set the point value for this task to 3.

I don't have time for this in the next 4-6 weeks, I'm removing myself for now and tagging this with ServiceOps as well.

thiemowmde added a subscriber: Tobi_WMDE_SW.

@Tobi_WMDE_SW, can you take care of this and possibly assign people who are able to deploy this, instead of @MoritzMuehlenhoff? Thanks!

Joe triaged this task as High priority.
jijiki lowered the priority of this task from High to Medium.Jun 25 2019, 2:58 PM

@thiemowmde @WMDE-Fisch I have installed php-wikidiff2_1.8.2-1~wmf1_amd64 on deployment-mediawiki-09 and deployment-mediawiki-07 but I have not restarted php72-fpm there. You can restart it when you are ready to test it. Let me know how it went so to proceed with the production upgrade:)

Mentioned in SAL (#wikimedia-releng) [2019-06-25T15:50:50Z] <awight> deployment-prep restart php7.2-fpm for wikidiff2 upgrade (T223391)

Mentioned in SAL (#wikimedia-cloud) [2019-06-25T15:51:49Z] <awight> restart php7.2-fpm for wikidiff2 upgrade (T223391)

Smoke testing the diffs shows that nothing seems to have been broken by the upgrade. We haven't been able to verify the fixes yet, still working on that.

WMDE-Fisch changed the point value for this task from 3 to 2.

Confirmed that the old version of wikidiff2 will fall back to not computing moved lines with more than 30 moves: https://test.wikipedia.org/w/index.php?title=User:Adamw/diff40&diff=398150&oldid=398149

Confirmed that the new version will attempt to compute moves for these 40 lines of diff: https://en.wikipedia.beta.wmflabs.org/w/index.php?title=User:Adamw/diff40&diff=394821&oldid=394820&diffmode=source

Old code showing 200 lines moved: https://test.wikipedia.org/w/index.php?title=User%3AAdamw%2Fdiff_large_chunks&type=revision&diff=398154&oldid=398153

New code showing 200 lines moved, when I expected it to fail: https://en.wikipedia.beta.wmflabs.org/w/index.php?title=User:Adamw/diff_large_chunks&diff=394823&oldid=394822&diffmode=source#movedpara_0_98_lhs

I can say, the new release seems harmless and we should go ahead with deployment, but I wasn't able to convince myself that it will solve the timeout issue.

@awight I will rollout the new version to production today

Mentioned in SAL (#wikimedia-operations) [2019-06-28T07:11:00Z] <_joe_> upgrading php-wikidiff2 on the mw canaries, only on php7 - T223391

Joe added subscribers: jijiki, Joe.

I did rollout the new version on the canary servers today.

If I don't see higher error rates on monday, I'll deploy to the whole fleet.

@jijiki We're a bit confused because the beta cluster reports it's running HHVM and there's no beta feature to select PHP7. Can you advise us on how to access the PHP7 backend?

@jijiki We're a bit confused because the beta cluster reports it's running HHVM and there's no beta feature to select PHP7. Can you advise us on how to access the PHP7 backend?

You can add a PHP_ENGINE=php7 cookie to your requests.

@jijiki We're a bit confused because the beta cluster reports it's running HHVM and there's no beta feature to select PHP7. Can you advise us on how to access the PHP7 backend?

You can add a PHP_ENGINE=php7 cookie to your requests.

Thanks!

I think we can probably use the X-Wikimedia-Debug browser plugin to test on mwdebug[12]00[12], and enable PHP7 for test.wikipedia.org now that you've deployed to canaries, too?

Yes :) It's by far the best option.

Certainly seems to be the case.

Good news, with PHP7 enabled I get the expected results where the new version of wikidiff2 refuses to calculate moves for https://test.wikipedia.org/w/index.php?title=User:Adamw/diff_large_chunks&diff=398154&oldid=398153

@jkroll ^

@awight @jkroll should we rollout to all hosts?

@awight @jkroll should we rollout to all hosts?

I think we should, I didn't get to it yesterday. So we should deploy everywhere and then do a rolling restart of php-fpm.

@awight @jkroll should we rollout to all hosts?

I think we should, I didn't get to it yesterday. So we should deploy everywhere and then do a rolling restart of php-fpm.

+1 please go on with the deploy from our side it should be good

Mentioned in SAL (#wikimedia-operations) [2019-07-02T10:45:07Z] <jijiki> Rollout Wikidiff 1.8.2 to codfw - T223391

Mentioned in SAL (#wikimedia-operations) [2019-07-02T10:47:43Z] <jijiki> Rollout Wikidiff 1.8.2 to eqiad - T223391

Mentioned in SAL (#wikimedia-operations) [2019-07-02T11:26:04Z] <jijiki> Run restart-php-fpm in all-mw-eqiad - T223391

Mentioned in SAL (#wikimedia-operations) [2019-07-02T14:58:41Z] <jijiki> Run restart-php-fpm in all-mw-codfw - T223391

All service restarts should be complete in a few hours, please reopen if there are any issues :)