Page MenuHomePhabricator

Unify revision table on s3
Closed, ResolvedPublic

Description

The last shard to unify the revision table on is s3.
This is going to be a tricky one as we need to see how many versions of the revision table we have there.

wikis that are correct:

dtywiki
ecwikimedia
fiwikivoyage
olowiki
pawikisource
projectcomwiki
ptwikimedia
wbwikimedia

Special cases - this is needed: drop key rev_page_id, add key rev_page_id (rev_page,rev_id)

adywiki
azbwiki
cawikimedia
elwikivoyage
enwikivoyage
eswikivoyage
fawikivoyage
fdcwiki
frwikivoyage
gomwiki
hewikivoyage
iegcomwiki
jamwiki
kowikiversity
legalteamwiki
loginwiki
lrcwiki
maiwiki
minwiki
nlwikivoyage
ombudsmenwiki
orwikisource
plwikivoyage
ptwikivoyage
rowikivoyage
ruwikivoyage
sawikiquote
svwikivoyage
tcywiki
testwikidatawiki
transitionteamwiki
tyvwiki
ukwikivoyage
vecwiktionary
viwikivoyage
votewiki
wikimania2013wiki
wikimania2014wiki
wikimania2015wiki
wikimania2016wiki
wikimania2017wiki
zerowiki
zhwikivoyage

Special cases - this is needed: drop key rev_id, drop primary key, add primary key (rev_id), add key rev_page_id (rev_page,rev_id)

dewikivoyage
itwikivoyage

Special cases - this is needed: drop key rev_page_id, add key rev_page_id (rev_page,rev_id), add key page_user_timestamp (rev_page,rev_user,rev_timestamp)

acewiki
arbcom_cswiki
arbcom_fiwiki
arwikimedia
arwikiversity
aswikisource (does not need add key page_user_timestamp (rev_page,rev_user,rev_timestamp)  )
bdwikimedia
bewikimedia
bewikisource
bjnwiki
boardgovcomwiki
brwikimedia
brwikisource
checkuserwiki
ckbwiki
cnwikimedia (does not need add key page_user_timestamp (rev_page,rev_user,rev_timestamp)  )
cowikimedia
dkwikimedia
donatewiki
elwikinews
eowikinews
eowikisource
etwikimedia
fawikinews
fiwikimedia
fiwikiversity
flaggedrevs_labswikimedia
frrwiki
gagwiki
guwikisource
kbdwiki
koiwiki
kowikinews
krcwiki
lezwiki
liquidthreads_labswikimedia
liwikibooks
ltgwiki
mhrwiki
mkwikimedia
movementroleswiki
mrjwiki
mrwikisource
mwlwiki
mxwikimedia
noboard_chapterswikimedia
noboardwiki
nsowiki
nycwikimedia
outreachwiki
pcdwiki
pflwiki
pnbwiki
pnbwiktionary
pntwiki
readerfeedback_labswikimedia
ruewiki
ruwikimedia
ruwikiversity
sahwikisource
sawikisource
slwikiversity
sqwikinews
steward
stewardwiki
strategyappswiki
strategywiki
svwikiversity
tenwiki
test2wiki
trwikimedia
trwikinews
uawikimedia
usabilitywiki
vecwikisource
vepwiki
vewikimedia
wikimania2010wiki
wikimania2011wiki
wikimania2012wiki
xmfwiki

The rest of the wikis - need: drop key rev_id, drop primary key, add primary key (rev_id), add key page_user_timestamp (rev_page,rev_user,rev_timestamp), add key rev_page_id (rev_page,rev_id)

Where to apply it:

  • dbstore2001.codfw.wmnet
  • dbstore2002.codfw.wmnet
  • db2036.codfw.wmnet
  • db2043.codfw.wmnet
  • db2050.codfw.wmnet
  • db2057.codfw.wmnet
  • db2018.codfw.wmnet
  • labsdb1001.eqiad.wmnet
  • labsdb1003.eqiad.wmnet
  • db1069.eqiad.wmnet
  • labsdb1009.eqiad.wmnet
  • labsdb1010.eqiad.wmnet
  • labsdb1011.eqiad.wmnet
  • dbstore1001.eqiad.wmnet
  • dbstore1002.eqiad.wmnet
  • db1095.eqiad.wmnet
  • db1015.eqiad.wmnet
  • db1035.eqiad.wmnet
  • db1038.eqiad.wmnet
  • db1044.eqiad.wmnet
  • db1077.eqiad.wmnet
  • db1078.eqiad.wmnet
  • db1075.eqiad.wmnet

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 355581 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-codfw.php: Depool db2057

https://gerrit.wikimedia.org/r/355581

Change 355581 merged by jenkins-bot:
[operations/mediawiki-config@master] db-codfw.php: Depool db2057

https://gerrit.wikimedia.org/r/355581

Mentioned in SAL (#wikimedia-operations) [2017-05-25T07:40:00Z] <marostegui@tin> Synchronized wmf-config/db-codfw.php: Depool db2057 - T166278 (duration: 00m 41s)

Change 355744 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-codfw.php: Repool db2057, depool db2050

https://gerrit.wikimedia.org/r/355744

Change 355744 merged by jenkins-bot:
[operations/mediawiki-config@master] db-codfw.php: Repool db2057, depool db2050

https://gerrit.wikimedia.org/r/355744

Mentioned in SAL (#wikimedia-operations) [2017-05-26T06:09:15Z] <marostegui@tin> Synchronized wmf-config/db-codfw.php: Repool db2057, depool db2050 - T166278 (duration: 00m 56s)

Mentioned in SAL (#wikimedia-operations) [2017-05-26T06:10:15Z] <marostegui> Deploy alter table on s3 - db2050 - T166278

Change 355779 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-codfw.php: Repool db2050, depool db2043

https://gerrit.wikimedia.org/r/355779

Change 355779 merged by jenkins-bot:
[operations/mediawiki-config@master] db-codfw.php: Repool db2050, depool db2043

https://gerrit.wikimedia.org/r/355779

Mentioned in SAL (#wikimedia-operations) [2017-05-26T14:03:06Z] <marostegui@tin> Synchronized wmf-config/db-codfw.php: Repool db2050, depool db2043 - T166278 (duration: 00m 41s)

Mentioned in SAL (#wikimedia-operations) [2017-05-26T14:04:34Z] <marostegui> Deploy alter tabel on s3 revision table db2043 - T166278

Change 356006 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-codfw.php: Repool db2043, depool db2036

https://gerrit.wikimedia.org/r/356006

Change 356006 merged by jenkins-bot:
[operations/mediawiki-config@master] db-codfw.php: Repool db2043, depool db2036

https://gerrit.wikimedia.org/r/356006

Mentioned in SAL (#wikimedia-operations) [2017-05-29T06:32:11Z] <marostegui@tin> Synchronized wmf-config/db-codfw.php: Repool db2043, depool db2036 - T166278 (duration: 01m 44s)

Mentioned in SAL (#wikimedia-operations) [2017-05-29T06:32:29Z] <marostegui> Deploy alter table s3 - db2036 - T166278

Mentioned in SAL (#wikimedia-operations) [2017-05-29T06:42:52Z] <marostegui> Deploy alter table s3 - dbstore2002 - T166278

Change 356044 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-codfw.php: Repool db2036

https://gerrit.wikimedia.org/r/356044

Mentioned in SAL (#wikimedia-operations) [2017-05-29T14:01:25Z] <marostegui> Deploy alter table s3 on codfw master db2018 - T166278

Change 356044 merged by jenkins-bot:
[operations/mediawiki-config@master] db-codfw.php: Repool db2036

https://gerrit.wikimedia.org/r/356044

Mentioned in SAL (#wikimedia-operations) [2017-05-29T14:03:51Z] <marostegui@tin> Synchronized wmf-config/db-codfw.php: Repool db2036 - T166278 (duration: 00m 41s)

Mentioned in SAL (#wikimedia-operations) [2017-05-30T06:23:58Z] <marostegui> Deploy alter table on s3 dbstore2001 - T166278

Mentioned in SAL (#wikimedia-operations) [2017-05-30T06:41:24Z] <marostegui> Deploy alter table on s3 dbstore1002 - https://phabricator.wikimedia.org/T166278

This is not going to happen as I realised it is still executing other heavy alter tables, so not a good idea to combine both at the same time.

Mentioned in SAL (#wikimedia-operations) [2017-05-30T06:45:34Z] <marostegui> Deploy alter table on s3 db1038 - T166278

Mentioned in SAL (#wikimedia-operations) [2017-05-30T15:33:23Z] <marostegui> Deploy alter table on s3.revision on labsdb1009 - T166278

Mentioned in SAL (#wikimedia-operations) [2017-05-31T06:04:17Z] <marostegui> Deploy alter table on s3 revision table - db1095 - T166278

Mentioned in SAL (#wikimedia-operations) [2017-05-31T06:04:40Z] <marostegui> Deploy alter table on s3 revision table - db1078 - T166278

Change 356332 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1078

https://gerrit.wikimedia.org/r/356332

Change 356332 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1078

https://gerrit.wikimedia.org/r/356332

Mentioned in SAL (#wikimedia-operations) [2017-05-31T06:12:17Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1078 - T166278 (duration: 00m 43s)

Mentioned in SAL (#wikimedia-operations) [2017-05-31T06:45:20Z] <marostegui> Deploy alter table s3 revision table - dbstore1002 - T166278

Change 356345 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Repool db1078, depool db1077

https://gerrit.wikimedia.org/r/356345

Change 356345 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Repool db1078, depool db1077

https://gerrit.wikimedia.org/r/356345

Mentioned in SAL (#wikimedia-operations) [2017-05-31T09:27:58Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1078, depool db1077 - T166278 (duration: 00m 42s)

Mentioned in SAL (#wikimedia-operations) [2017-05-31T09:30:15Z] <marostegui> Deploy alter table on s3 revision table - db1078 - https://phabricator.wikimedia.org/T166278

Change 356375 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Repool db1077, depool db1035

https://gerrit.wikimedia.org/r/356375

Change 356375 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Repool db1077, depool db1035

https://gerrit.wikimedia.org/r/356375

Mentioned in SAL (#wikimedia-operations) [2017-05-31T12:35:15Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1077, depool db1035 - T166278 (duration: 00m 41s)

Mentioned in SAL (#wikimedia-operations) [2017-05-31T12:35:24Z] <marostegui> Deploy alter table on s3 revision table - db1035 - https://phabricator.wikimedia.org/T166278

Change 356532 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Repool db1035

https://gerrit.wikimedia.org/r/356532

Mentioned in SAL (#wikimedia-operations) [2017-06-01T06:02:48Z] <marostegui> Deploy alter table on s3, dbstore1001 - T166278

Mentioned in SAL (#wikimedia-operations) [2017-06-01T06:04:42Z] <marostegui> Deploy alter table on s3, db1044 - T166278

Change 356532 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Repool db1035

https://gerrit.wikimedia.org/r/356532

Mentioned in SAL (#wikimedia-operations) [2017-06-01T06:07:54Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1035 - T166278 (duration: 00m 57s)

Mentioned in SAL (#wikimedia-operations) [2017-06-01T06:08:14Z] <marostegui> Deploy alter table on s3, labsdb1010 - T166278

Mentioned in SAL (#wikimedia-operations) [2017-06-01T13:15:14Z] <marostegui> Deploy alter table s3 revision on labsdb1011 - T166278

Mentioned in SAL (#wikimedia-operations) [2017-06-01T13:18:39Z] <marostegui> Deploy alter table s3 revision on labsdb1001 - T166278

Mentioned in SAL (#wikimedia-operations) [2017-06-02T09:12:27Z] <marostegui> Deploy alter table s3 - labsdb1003 - T166278

I will do db1015 in the end, just to make sure it can be checksummed and doesn't get lagged when running pt-table-checksum on the revision tables.

Mentioned in SAL (#wikimedia-operations) [2017-06-02T09:18:55Z] <marostegui> Deploy alter table s3 - db1015 - T166278

Mentioned in SAL (#wikimedia-operations) [2017-06-05T06:15:37Z] <marostegui> Deploy alter table on s3 - db1069 - T166278

Mentioned in SAL (#wikimedia-operations) [2017-06-06T05:56:11Z] <marostegui> Deploy alter table s3 on db1075 (eqiad master) - T166278

db1075 the master is done - the whole shard is completed.

Marostegui updated the task description. (Show Details)