Page MenuHomePhabricator

Compress new Wikibase tables
Open, MediumPublic

Description

This and maybe other newly created tables are in innodb uncompressed format, and probably will benefit a lot from it:

| wbt_item_terms           | InnoDB |      10 | Compact    |   79850814 |             38 |   3068133376 |               
| wbt_property_terms       | InnoDB |      10 | Compact    |     261980 |             34 |      8929280 |               
| wbt_term_in_lang         | InnoDB |      10 | Compact    |   45801912 |             35 |   1633681408 |               
| wbt_text                 | InnoDB |      10 | Compact    |   17176409 |             60 |   1044381696 |               
| wbt_text_in_lang         | InnoDB |      10 | Compact    |   45872021 |             36 |   1680867328 |               
| wbt_type                 | InnoDB |      10 | Compact    |          2 |           8192 |        16384 |

Progress:

  • db1109
  • db1087
  • db1092
  • db1099:3318
  • db1101:3318
  • db1104
  • db1116:3318
  • db1124:3318
  • db1126
  • db2079
  • db2080
  • db2081
  • db2082
  • db2083
  • db2085:3318
  • db2086:3318
  • db2094:3318
  • db2100:3318
  • dbstore1005:3318
  • labsdb1009
  • labsdb1010
  • labsdb1011
  • labsdb1012

Also the new wikibase tables created in s4 need compression:

  • labsdb1012
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1004:3314
  • db2119
  • db2110
  • db2106
  • db2099
  • db2095
  • db2091
  • db2090
  • db2084
  • db2073
  • db1138
  • db1125
  • db1121
  • db1103
  • db1102
  • db1097
  • db1091
  • db1084
  • db1081

Details

Related Gerrit Patches:
operations/puppet : productiondb1087: Enable notifications
operations/puppet : productiondb1087: Disable notifications
operations/puppet : productiondb1124: Enable notifications
operations/puppet : productiondb1124: Disable notifications
operations/puppet : productiondb1091: Enable notifications
operations/puppet : productiondb1126: Enable notifications
operations/puppet : productiondb1091,db1126: Disable notifications
operations/puppet : productiondb1121: Enable notifications
operations/puppet : productiondb1125: Disable notifications
operations/puppet : productiondb1125: Enable notifications
operations/puppet : productiondb1121: Disable notifications
operations/puppet : productiondb1081: Disable notifications

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Marostegui updated the task description. (Show Details)Dec 5 2019, 5:45 AM
Marostegui updated the task description. (Show Details)Dec 10 2019, 6:01 AM
Marostegui updated the task description. (Show Details)Jan 13 2020, 5:52 AM

Mentioned in SAL (#wikimedia-operations) [2020-01-13T06:20:08Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1081 for compression T232446', diff saved to https://phabricator.wikimedia.org/P10130 and previous config saved to /var/cache/conftool/dbconfig/20200113-062007-marostegui.json

Marostegui updated the task description. (Show Details)Jan 17 2020, 6:31 AM
Marostegui updated the task description. (Show Details)Jan 20 2020, 6:04 AM

Mentioned in SAL (#wikimedia-operations) [2020-01-20T06:08:00Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1121, pool db1084 into vslow T232446', diff saved to https://phabricator.wikimedia.org/P10214 and previous config saved to /var/cache/conftool/dbconfig/20200120-060759-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2020-01-20T06:08:55Z] <marostegui> Compress db1121 - T232446

Change 565832 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1121: Disable notifications

https://gerrit.wikimedia.org/r/565832

Change 565832 merged by Marostegui:
[operations/puppet@production] db1121: Disable notifications

https://gerrit.wikimedia.org/r/565832

Change 565979 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1125: Enable notifications

https://gerrit.wikimedia.org/r/565979

Change 565979 merged by Marostegui:
[operations/puppet@production] db1125: Enable notifications

https://gerrit.wikimedia.org/r/565979

Mentioned in SAL (#wikimedia-operations) [2020-01-22T15:38:44Z] <marostegui> Compress wikidatawiki.wbt_text wikidatawiki.wbt_text_in_lang on db1124:3318 (this might cause lag on s8 labs) - T232446

Mentioned in SAL (#wikimedia-operations) [2020-01-23T05:55:37Z] <marostegui> Compress some tables on db1124:3318, this might generate lag on s8 labs - T232446

Marostegui updated the task description. (Show Details)Jan 24 2020, 1:31 PM

Change 567042 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1125: Disable notifications

https://gerrit.wikimedia.org/r/567042

Change 567042 merged by Marostegui:
[operations/puppet@production] db1125: Disable notifications

https://gerrit.wikimedia.org/r/567042

Change 568121 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1121: Enable notifications

https://gerrit.wikimedia.org/r/568121

Change 568121 merged by Marostegui:
[operations/puppet@production] db1121: Enable notifications

https://gerrit.wikimedia.org/r/568121

Mentioned in SAL (#wikimedia-operations) [2020-01-28T23:40:38Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1121 T232446', diff saved to https://phabricator.wikimedia.org/P10284 and previous config saved to /var/cache/conftool/dbconfig/20200128-234037-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2020-02-04T07:08:04Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1091 - T232446', diff saved to https://phabricator.wikimedia.org/P10301 and previous config saved to /var/cache/conftool/dbconfig/20200204-070804-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2020-02-04T07:08:59Z] <marostegui> Compress db1091 - T232446

Mentioned in SAL (#wikimedia-operations) [2020-02-04T07:14:20Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1126 - T232446', diff saved to https://phabricator.wikimedia.org/P10302 and previous config saved to /var/cache/conftool/dbconfig/20200204-071420-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2020-02-04T07:15:01Z] <marostegui> Compress db1126 - T232446

Change 569717 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1091,db1126: Disable notifications

https://gerrit.wikimedia.org/r/569717

Change 569717 merged by Marostegui:
[operations/puppet@production] db1091,db1126: Disable notifications

https://gerrit.wikimedia.org/r/569717

Marostegui updated the task description. (Show Details)Fri, Feb 7, 6:21 AM

Change 570790 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1126: Enable notifications

https://gerrit.wikimedia.org/r/570790

Change 570790 merged by Marostegui:
[operations/puppet@production] db1126: Enable notifications

https://gerrit.wikimedia.org/r/570790

Mentioned in SAL (#wikimedia-operations) [2020-02-07T06:25:02Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1126 T232446', diff saved to https://phabricator.wikimedia.org/P10330 and previous config saved to /var/cache/conftool/dbconfig/20200207-062502-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2020-02-07T06:38:31Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1126 T232446', diff saved to https://phabricator.wikimedia.org/P10333 and previous config saved to /var/cache/conftool/dbconfig/20200207-063831-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2020-02-07T07:30:26Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1126 T232446', diff saved to https://phabricator.wikimedia.org/P10334 and previous config saved to /var/cache/conftool/dbconfig/20200207-073026-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2020-02-07T07:45:12Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Fullyy repool db1126 T232446', diff saved to https://phabricator.wikimedia.org/P10338 and previous config saved to /var/cache/conftool/dbconfig/20200207-074511-marostegui.json

Marostegui updated the task description. (Show Details)Fri, Feb 7, 3:25 PM

Mentioned in SAL (#wikimedia-operations) [2020-02-10T06:16:57Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1091 T232446', diff saved to https://phabricator.wikimedia.org/P10355 and previous config saved to /var/cache/conftool/dbconfig/20200210-061656-marostegui.json

Change 571114 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1091: Enable notifications

https://gerrit.wikimedia.org/r/571114

Change 571114 merged by Marostegui:
[operations/puppet@production] db1091: Enable notifications

https://gerrit.wikimedia.org/r/571114

Mentioned in SAL (#wikimedia-operations) [2020-02-10T06:37:16Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1091 T232446', diff saved to https://phabricator.wikimedia.org/P10358 and previous config saved to /var/cache/conftool/dbconfig/20200210-063716-marostegui.json

Change 571116 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1124: Disable notifications

https://gerrit.wikimedia.org/r/571116

Mentioned in SAL (#wikimedia-operations) [2020-02-10T06:39:09Z] <marostegui> Compress db1124:3318 - this will generate lag on s8 wiki replicas - T232446

Change 571116 merged by Marostegui:
[operations/puppet@production] db1124: Disable notifications

https://gerrit.wikimedia.org/r/571116

Mentioned in SAL (#wikimedia-operations) [2020-02-10T06:44:59Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1091 T232446', diff saved to https://phabricator.wikimedia.org/P10359 and previous config saved to /var/cache/conftool/dbconfig/20200210-064458-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2020-02-10T06:51:36Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1091 T232446', diff saved to https://phabricator.wikimedia.org/P10361 and previous config saved to /var/cache/conftool/dbconfig/20200210-065135-marostegui.json

Marostegui updated the task description. (Show Details)Tue, Feb 11, 6:09 AM

Change 571417 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1124: Enable notifications

https://gerrit.wikimedia.org/r/571417

Change 571417 merged by Marostegui:
[operations/puppet@production] db1124: Enable notifications

https://gerrit.wikimedia.org/r/571417

Change 571864 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1087: Disable notifications

https://gerrit.wikimedia.org/r/571864

Change 571864 merged by Marostegui:
[operations/puppet@production] db1087: Disable notifications

https://gerrit.wikimedia.org/r/571864

Mentioned in SAL (#wikimedia-operations) [2020-02-13T06:35:35Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1087 for compression - T232446', diff saved to https://phabricator.wikimedia.org/P10401 and previous config saved to /var/cache/conftool/dbconfig/20200213-063535-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2020-02-13T06:36:42Z] <marostegui> Upgrade and compress db1087, this will generate lag on s8 on the wiki replicas - T232446

db1087 has all its tables compressed but wb_terms.
I am going to wait for labs to catch up a bit before moving them under codfw's sanitarium before starting the compression on that table, which I guess it will take around 4-5 days.

Change 573436 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1087: Enable notifications

https://gerrit.wikimedia.org/r/573436

Change 573436 merged by Marostegui:
[operations/puppet@production] db1087: Enable notifications

https://gerrit.wikimedia.org/r/573436

Mentioned in SAL (#wikimedia-operations) [2020-02-24T10:10:30Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1087 for compression and place db1101:3318 into vslow,dump - T232446', diff saved to https://phabricator.wikimedia.org/P10493 and previous config saved to /var/cache/conftool/dbconfig/20200224-101030-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2020-02-24T10:12:34Z] <marostegui> Stop db1087 and db2079 in sync - T232446

Mentioned in SAL (#wikimedia-operations) [2020-02-24T11:02:45Z] <marostegui> Move labsdb1009, labsdb1011 and labsdb1012 (labsdb1010 is currently delayed, will be done later) to replicate under codfw for a few days while we alter wb_terms on db1087 - T232446