Deploy Reading Lists Service to production
Closed, ResolvedPublic

Description

Scheduled for Thursday Dec 7, 22h UTC.
FWIW the apps won't start using the API this year, so it won't get nontrivial load for at least a month or so after deployment.

Steps

There are a very large number of changes, so older changes are hidden. Show Older Changes
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 21 2017, 11:48 PM
Tgr renamed this task from Deploy ReadingLists to production to Deploy Reading Lists Service to production.Nov 21 2017, 11:51 PM
Tgr updated the task description. (Show Details)
Tgr updated the task description. (Show Details)Nov 28 2017, 11:11 PM
Tgr updated the task description. (Show Details)Nov 30 2017, 1:37 AM
Tgr moved this task from Backlog to Coming soon on the Reading List Service board.Dec 6 2017, 12:33 AM

Change 395687 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[operations/mediawiki-config@master] Deploy ReadingLists to production

https://gerrit.wikimedia.org/r/395687

Change 395688 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[operations/mediawiki-config@master] Enable ReadingLists on all SUL wikis

https://gerrit.wikimedia.org/r/395688

Tgr updated the task description. (Show Details)Dec 6 2017, 3:07 AM

Change 395694 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[operations/puppet@production] Add cron job for purging ReadingLists data

https://gerrit.wikimedia.org/r/395694

Tgr updated the task description. (Show Details)Dec 6 2017, 7:06 AM
Tgr updated the task description. (Show Details)Dec 6 2017, 7:08 AM
Tgr added a subscriber: jcrespo.Dec 6 2017, 7:43 AM

@jcrespo does this seem OK? RL is set to use extension1 / wikishared (patch) and the plan for setting up the DB is running mwscript sql.php --cluster external1 --wikidb wikishared /srv/mediawiki-staging/php-1.31.0-wmf.11/extensions/ReadingLists/sql/readinglists.sql (readinglists.sql).

Tgr updated the task description. (Show Details)Dec 6 2017, 8:03 AM
Tgr updated the task description. (Show Details)Dec 6 2017, 8:41 PM
Tgr updated the task description. (Show Details)
Tgr updated the task description. (Show Details)Dec 7 2017, 7:25 AM
Tgr updated the task description. (Show Details)Dec 7 2017, 7:28 AM
Tgr added a comment.EditedDec 7 2017, 8:12 AM

WRT past performance discussions (such as T164990#3264314 and T168984#3659998), see T182053#3814254 for query performance and 8f11305c7b1e for limits.

Tgr updated the task description. (Show Details)Dec 7 2017, 6:26 PM
Tgr updated the task description. (Show Details)Dec 7 2017, 10:32 PM
Tgr updated the task description. (Show Details)Dec 7 2017, 10:39 PM

Change 395687 merged by jenkins-bot:
[operations/mediawiki-config@master] Deploy ReadingLists to testwiki

https://gerrit.wikimedia.org/r/395687

Mentioned in SAL (#wikimedia-operations) [2017-12-07T23:22:43Z] <tgr@tin> Started scap: T181107 deploy ReadingLists to testwiki

Mentioned in SAL (#wikimedia-operations) [2017-12-07T23:47:32Z] <tgr@tin> Finished scap: T181107 deploy ReadingLists to testwiki (duration: 24m 44s)

Tgr updated the task description. (Show Details)Dec 7 2017, 11:51 PM

Change 395688 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable ReadingLists on all SUL wikis

https://gerrit.wikimedia.org/r/395688

Mentioned in SAL (#wikimedia-operations) [2017-12-07T23:59:49Z] <tgr@tin> Synchronized wmf-config/InitialiseSettings.php: T181107 enable ReadingLists on all wikis (duration: 00m 46s)

Tgr updated the task description. (Show Details)Dec 8 2017, 12:01 AM

The MediaWiki part is now live everywhere. Follow-up item: fix T157651: sql.php runs LoadExtensionSchemaUpdates which caused a bit of headache. (I guess I deserve it for not fixing when I first ran into it.)

@mobrovac @Pchelolo the MediaWiki side is now live. Any thoughts on scheduling the RESTBase deployment?

Pchelolo added a comment.EditedDec 8 2017, 1:47 AM

@Tgr given that the next week is the last one before deployment freeze an dI assume you want to be able to test against it during the freeze, we must get it out next week.

Monday or Tuesday we will be deploying quite huge Parsoid storage changes and will need at least a day of calm weather after that, but since your changes do not rely on storage at all and there will be no additional traffic, I think we can stack the reading lists together with parsoid deploy. So, if @mobrovac agrees I'd say we do it Mon/Tue.

In the meantime, could you create a PR from your reading_lists_beta brunch into master for the final glance over the changes?

Yup, +1. From what I can see, all concerns have been addressed so we should be good to go. I propose to deploy it Monday, 2017-12-11 during EU or SF hours, whichever works best for you guys (as you should be around when this happens).

Tgr added a comment.Dec 10 2017, 9:14 AM

PR #926. Thanks! SF hours would be better.

There is one thing missing that I would like you to address- I believe these tables will be private. While we do not replicate x1 to labsdb hosts, we do not discard that happening at some point. There is 2 places were we record private tables- if you could add the ones you added there, that would help prevent data leaks:

(yes, this is meaningless duplication, this is to be addressed at T103011)

We started adding x1 tables there, too for fear that they could end up in labs.

Mentioned in SAL (#wikimedia-operations) [2017-12-11T19:26:38Z] <mobrovac@tin> Started deploy [restbase/deploy@bce2885]: Expose the Reading Lists end points - T181107

Mentioned in SAL (#wikimedia-operations) [2017-12-11T19:28:05Z] <mobrovac@tin> Finished deploy [restbase/deploy@bce2885]: Expose the Reading Lists end points - T181107 (duration: 01m 26s)

Mentioned in SAL (#wikimedia-operations) [2017-12-11T19:28:35Z] <mobrovac@tin> Started deploy [restbase/deploy@bce2885]: Expose the Reading Lists end points, take #2 - T181107

Mentioned in SAL (#wikimedia-operations) [2017-12-11T19:34:07Z] <mobrovac@tin> Finished deploy [restbase/deploy@bce2885]: Expose the Reading Lists end points, take #2 - T181107 (duration: 05m 33s)

Mentioned in SAL (#wikimedia-operations) [2017-12-11T19:40:55Z] <mobrovac@tin> Started deploy [restbase/deploy@be7d72f]: Expose the Reading Lists end points, take #3 - T181107

Mentioned in SAL (#wikimedia-operations) [2017-12-11T19:47:14Z] <mobrovac@tin> Finished deploy [restbase/deploy@be7d72f]: Expose the Reading Lists end points, take #3 - T181107 (duration: 06m 19s)

mobrovac updated the task description. (Show Details)Dec 11 2017, 7:48 PM
mobrovac edited projects, added Services (done); removed Services (doing).

The RESTBase side of things has been deployed \o/

Tgr updated the task description. (Show Details)Dec 11 2017, 8:00 PM
Tgr updated the task description. (Show Details)Dec 11 2017, 8:41 PM
Tgr updated the task description. (Show Details)Dec 11 2017, 8:46 PM
Tgr updated the task description. (Show Details)Dec 13 2017, 9:34 PM
Tgr updated the task description. (Show Details)Dec 14 2017, 8:42 PM
Tgr updated the task description. (Show Details)
Tgr updated the task description. (Show Details)Dec 14 2017, 8:49 PM
Tgr updated the task description. (Show Details)Dec 14 2017, 9:08 PM
Tgr updated the task description. (Show Details)Dec 15 2017, 12:34 AM
Tgr updated the task description. (Show Details)
Tgr updated the task description. (Show Details)Dec 15 2017, 12:59 AM
Tgr updated the task description. (Show Details)Dec 16 2017, 1:08 AM
Tgr updated the task description. (Show Details)Dec 16 2017, 9:05 AM
Tgr updated the task description. (Show Details)Jan 7 2018, 3:36 AM

Change 395694 merged by Jcrespo:
[operations/puppet@production] Add cron job for purging ReadingLists data

https://gerrit.wikimedia.org/r/395694

Tgr updated the task description. (Show Details)Jan 10 2018, 1:23 AM
Tgr closed this task as Resolved.
Tgr claimed this task.
jcrespo reopened this task as Open.Jan 12 2018, 1:18 PM

Cron job fails.

Change 404049 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[mediawiki/extensions/ReadingLists@master] Fix purge.php parameter handling

https://gerrit.wikimedia.org/r/404049

Change 404049 merged by jenkins-bot:
[mediawiki/extensions/ReadingLists@master] Fix purge.php parameter handling

https://gerrit.wikimedia.org/r/404049

Change 409645 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[operations/puppet@production] Re-enable cron job for purging ReadingLists data

https://gerrit.wikimedia.org/r/409645

Change 409645 merged by Filippo Giunchedi:
[operations/puppet@production] Re-enable cron job for purging ReadingLists data

https://gerrit.wikimedia.org/r/409645

Tgr closed this task as Resolved.