Page MenuHomePhabricator
Create Task
Maniphest T232446

Compress new Wikibase tables
Closed, ResolvedPublic
Actions

Description

This and maybe other newly created tables are in innodb uncompressed format, and probably will benefit a lot from it:

| wbt_item_terms           | InnoDB |      10 | Compact    |   79850814 |             38 |   3068133376 |               
| wbt_property_terms       | InnoDB |      10 | Compact    |     261980 |             34 |      8929280 |               
| wbt_term_in_lang         | InnoDB |      10 | Compact    |   45801912 |             35 |   1633681408 |               
| wbt_text                 | InnoDB |      10 | Compact    |   17176409 |             60 |   1044381696 |               
| wbt_text_in_lang         | InnoDB |      10 | Compact    |   45872021 |             36 |   1680867328 |               
| wbt_type                 | InnoDB |      10 | Compact    |          2 |           8192 |        16384 |

Progress:

  • db1109
  • db1087
  • db1092
  • db1099:3318
  • db1101:3318
  • db1104
  • db1116:3318
  • db1124:3318
  • db1126
  • db2079
  • db2080
  • db2081
  • db2082
  • db2083
  • db2085:3318
  • db2086:3318
  • db2094:3318
  • db2100:3318
  • dbstore1005:3318
  • labsdb1009
  • labsdb1010
  • labsdb1011
  • labsdb1012

Also the new wikibase tables created in s4 need compression:

  • labsdb1012
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1004:3314
  • db2119
  • db2110
  • db2106
  • db2099
  • db2095
  • db2091
  • db2090
  • db2084
  • db2073
  • db1138
  • db1125
  • db1121
  • db1103
  • db1102
  • db1097
  • db1091
  • db1084
  • db1081

Details

Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Maintenance_bot removed a project: Patch-For-Review.Feb 4 2020, 8:10 AM
Marostegui updated the task description. (Show Details)Feb 7 2020, 6:21 AM
gerritbot added a comment.Feb 7 2020, 6:23 AM

Change 570790 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1126: Enable notifications

https://gerrit.wikimedia.org/r/570790

gerritbot added a project: Patch-For-Review.Feb 7 2020, 6:23 AM

Change 570790 merged by Marostegui:
[operations/puppet@production] db1126: Enable notifications

https://gerrit.wikimedia.org/r/570790

Stashbot added a comment.Feb 7 2020, 6:25 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-07T06:25:02Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1126 T232446', diff saved to https://phabricator.wikimedia.org/P10330 and previous config saved to /var/cache/conftool/dbconfig/20200207-062502-marostegui.json

Stashbot added a comment.Feb 7 2020, 6:38 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-07T06:38:31Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1126 T232446', diff saved to https://phabricator.wikimedia.org/P10333 and previous config saved to /var/cache/conftool/dbconfig/20200207-063831-marostegui.json

Maintenance_bot removed a project: Patch-For-Review.Feb 7 2020, 7:10 AM
Stashbot added a comment.Feb 7 2020, 7:30 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-07T07:30:26Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1126 T232446', diff saved to https://phabricator.wikimedia.org/P10334 and previous config saved to /var/cache/conftool/dbconfig/20200207-073026-marostegui.json

Stashbot added a comment.Feb 7 2020, 7:45 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-07T07:45:12Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Fullyy repool db1126 T232446', diff saved to https://phabricator.wikimedia.org/P10338 and previous config saved to /var/cache/conftool/dbconfig/20200207-074511-marostegui.json

Marostegui updated the task description. (Show Details)Feb 7 2020, 3:25 PM
Stashbot added a comment.Feb 10 2020, 6:16 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-10T06:16:57Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1091 T232446', diff saved to https://phabricator.wikimedia.org/P10355 and previous config saved to /var/cache/conftool/dbconfig/20200210-061656-marostegui.json

gerritbot added a comment.Feb 10 2020, 6:17 AM

Change 571114 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1091: Enable notifications

https://gerrit.wikimedia.org/r/571114

gerritbot added a project: Patch-For-Review.Feb 10 2020, 6:17 AM

Change 571114 merged by Marostegui:
[operations/puppet@production] db1091: Enable notifications

https://gerrit.wikimedia.org/r/571114

Stashbot added a comment.Feb 10 2020, 6:37 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-10T06:37:16Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1091 T232446', diff saved to https://phabricator.wikimedia.org/P10358 and previous config saved to /var/cache/conftool/dbconfig/20200210-063716-marostegui.json

gerritbot added a comment.Feb 10 2020, 6:38 AM

Change 571116 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1124: Disable notifications

https://gerrit.wikimedia.org/r/571116

Stashbot added a comment.Feb 10 2020, 6:39 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-10T06:39:09Z] <marostegui> Compress db1124:3318 - this will generate lag on s8 wiki replicas - T232446

gerritbot added a comment.Feb 10 2020, 6:39 AM

Change 571116 merged by Marostegui:
[operations/puppet@production] db1124: Disable notifications

https://gerrit.wikimedia.org/r/571116

Stashbot added a comment.Feb 10 2020, 6:44 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-10T06:44:59Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1091 T232446', diff saved to https://phabricator.wikimedia.org/P10359 and previous config saved to /var/cache/conftool/dbconfig/20200210-064458-marostegui.json

Stashbot added a comment.Feb 10 2020, 6:51 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-10T06:51:36Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1091 T232446', diff saved to https://phabricator.wikimedia.org/P10361 and previous config saved to /var/cache/conftool/dbconfig/20200210-065135-marostegui.json

Maintenance_bot removed a project: Patch-For-Review.Feb 10 2020, 7:11 AM
Marostegui updated the task description. (Show Details)Feb 11 2020, 6:09 AM
gerritbot added a comment.Feb 11 2020, 6:12 AM

Change 571417 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1124: Enable notifications

https://gerrit.wikimedia.org/r/571417

gerritbot added a project: Patch-For-Review.Feb 11 2020, 6:12 AM

Change 571417 merged by Marostegui:
[operations/puppet@production] db1124: Enable notifications

https://gerrit.wikimedia.org/r/571417

Maintenance_bot removed a project: Patch-For-Review.Feb 11 2020, 7:10 AM
gerritbot added a comment.Feb 13 2020, 6:26 AM

Change 571864 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1087: Disable notifications

https://gerrit.wikimedia.org/r/571864

gerritbot added a project: Patch-For-Review.Feb 13 2020, 6:26 AM

Change 571864 merged by Marostegui:
[operations/puppet@production] db1087: Disable notifications

https://gerrit.wikimedia.org/r/571864

Stashbot added a comment.Feb 13 2020, 6:35 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T06:35:35Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1087 for compression - T232446', diff saved to https://phabricator.wikimedia.org/P10401 and previous config saved to /var/cache/conftool/dbconfig/20200213-063535-marostegui.json

Stashbot added a comment.Feb 13 2020, 6:36 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T06:36:42Z] <marostegui> Upgrade and compress db1087, this will generate lag on s8 on the wiki replicas - T232446

Maintenance_bot removed a project: Patch-For-Review.Feb 13 2020, 7:10 AM
Marostegui added a comment.Feb 19 2020, 6:03 AM

db1087 has all its tables compressed but wb_terms.
I am going to wait for labs to catch up a bit before moving them under codfw's sanitarium before starting the compression on that table, which I guess it will take around 4-5 days.

gerritbot added a comment.Feb 20 2020, 6:01 AM

Change 573436 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1087: Enable notifications

https://gerrit.wikimedia.org/r/573436

gerritbot added a project: Patch-For-Review.Feb 20 2020, 6:01 AM
gerritbot added a comment.Feb 20 2020, 6:03 AM

Change 573436 merged by Marostegui:
[operations/puppet@production] db1087: Enable notifications

https://gerrit.wikimedia.org/r/573436

Maintenance_bot removed a project: Patch-For-Review.Feb 20 2020, 6:10 AM
Stashbot added a comment.Feb 24 2020, 10:10 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-24T10:10:30Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1087 for compression and place db1101:3318 into vslow,dump - T232446', diff saved to https://phabricator.wikimedia.org/P10493 and previous config saved to /var/cache/conftool/dbconfig/20200224-101030-marostegui.json

Stashbot added a comment.Feb 24 2020, 10:12 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-24T10:12:34Z] <marostegui> Stop db1087 and db2079 in sync - T232446

Stashbot added a comment.Feb 24 2020, 11:02 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-24T11:02:45Z] <marostegui> Move labsdb1009, labsdb1011 and labsdb1012 (labsdb1010 is currently delayed, will be done later) to replicate under codfw for a few days while we alter wb_terms on db1087 - T232446

Stashbot added a comment.Feb 24 2020, 2:42 PM

Mentioned in SAL (#wikimedia-operations) [2020-02-24T14:42:33Z] <marostegui> Compress innodb on wb_terms on db1087 - T232446

Stashbot added a comment.Feb 25 2020, 6:02 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-25T06:02:05Z] <marostegui> Move labsdb1010 under db2094:3318 - T232446

Marostegui added a comment.Feb 26 2020, 9:05 PM

db1087 finished and it is now catching up

Marostegui updated the task description. (Show Details)Feb 26 2020, 9:06 PM
jcrespo mentioned this in T246159: Setting on read new for items causes increased worst response time and number of open connections.Feb 27 2020, 4:59 PM
jcrespo added a comment.EditedFeb 27 2020, 5:07 PM

I've given db1087 a bit of weight (weight 100, percentage: 20) so it can get some small amount of traffic, even if that means to depool it back down again for replication stop.

Stashbot added a comment.Feb 27 2020, 5:30 PM

Mentioned in SAL (#wikimedia-operations) [2020-02-27T17:30:17Z] <jynus@cumin1001> dbctl commit (dc=all): 'Repool db1087 at 20% T232446', diff saved to https://phabricator.wikimedia.org/P10547 and previous config saved to /var/cache/conftool/dbconfig/20200227-173017-jynus.json

Marostegui closed subtask T245107: Possibly replace db1087 (s8) with db1127 (x1) due to disk space constrains as Declined.Feb 28 2020, 6:27 AM
gerritbot added a comment.Feb 28 2020, 9:54 AM

Change 575477 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1087: Enable notifications

https://gerrit.wikimedia.org/r/575477

gerritbot added a project: Patch-For-Review.Feb 28 2020, 9:54 AM

Change 575477 merged by Marostegui:
[operations/puppet@production] db1087: Enable notifications

https://gerrit.wikimedia.org/r/575477

Maintenance_bot removed a project: Patch-For-Review.Feb 28 2020, 10:10 AM
Marostegui added a comment.Mar 5 2020, 6:30 AM

Due to the load issues we are experimenting on Wikidata I am not compressing the remaining hosts till the load has decreased or till we've done the DC switchover and eqiad becomes passive for a few weeks. Whatever comes first cc @Addshore

Marostegui mentioned this in T207006: Set wb_changes_dispatch ROW_FORMAT=COMPRESSED on install and update.Mar 18 2020, 12:21 PM
Bugreporter added a subscriber: Bugreporter.Apr 7 2020, 9:31 AM

Per T249565: Wikidata's wb_items_per_site table has suddenly disappeared, creating DBQueryErrors on page views DBA should check whether wb_items_per_site table is compressed in each DB server.

Marostegui added a comment.Apr 7 2020, 9:33 AM

It was compressed everywhere but on db1109, db1104, db1092. The new table that will be reimported is already compressed.

Stashbot added a comment.Apr 13 2020, 7:39 AM

Mentioned in SAL (#wikimedia-operations) [2020-04-13T07:39:40Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1092 T232446', diff saved to https://phabricator.wikimedia.org/P10963 and previous config saved to /var/cache/conftool/dbconfig/20200413-073939-marostegui.json

Stashbot added a comment.Apr 13 2020, 7:43 AM

Mentioned in SAL (#wikimedia-operations) [2020-04-13T07:43:23Z] <marostegui> Compress db1092 T232446

Marostegui updated the task description. (Show Details)Apr 16 2020, 6:59 AM
Stashbot added a comment.Apr 27 2020, 5:53 AM

Mentioned in SAL (#wikimedia-operations) [2020-04-27T05:53:20Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1104 for defragmentation - T232446', diff saved to https://phabricator.wikimedia.org/P11039 and previous config saved to /var/cache/conftool/dbconfig/20200427-055320-marostegui.json

Stashbot added a comment.Apr 27 2020, 5:56 AM

Mentioned in SAL (#wikimedia-operations) [2020-04-27T05:56:50Z] <marostegui> Compress tables on db1104 - T232446

Marostegui changed the task status from Open to Stalled.May 1 2020, 5:12 AM
Marostegui updated the task description. (Show Details)
Marostegui moved this task from In progress to Blocked external/Not db team on the DBA board.

Only the primary database masters are pending.
To do those we'd either need a DC switchover or switch them individually.

Stashbot added a comment.May 1 2020, 1:28 PM

Mentioned in SAL (#wikimedia-operations) [2020-05-01T13:28:04Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly warm up db1104 - T232446', diff saved to https://phabricator.wikimedia.org/P11107 and previous config saved to /var/cache/conftool/dbconfig/20200501-132804-marostegui.json

Stashbot added a comment.May 1 2020, 1:47 PM

Mentioned in SAL (#wikimedia-operations) [2020-05-01T13:47:07Z] <marostegui@cumin1001> dbctl commit (dc=all): 'More traffic to db1104 - T232446', diff saved to https://phabricator.wikimedia.org/P11108 and previous config saved to /var/cache/conftool/dbconfig/20200501-134707-marostegui.json

Stashbot added a comment.May 1 2020, 2:06 PM

Mentioned in SAL (#wikimedia-operations) [2020-05-01T14:06:06Z] <marostegui@cumin1001> dbctl commit (dc=all): 'More traffic to db1104 - T232446', diff saved to https://phabricator.wikimedia.org/P11109 and previous config saved to /var/cache/conftool/dbconfig/20200501-140603-marostegui.json

Stashbot added a comment.May 1 2020, 2:23 PM

Mentioned in SAL (#wikimedia-operations) [2020-05-01T14:23:54Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1104 - T232446', diff saved to https://phabricator.wikimedia.org/P11110 and previous config saved to /var/cache/conftool/dbconfig/20200501-142354-marostegui.json

Marostegui added a comment.May 29 2020, 5:21 AM

db1138 is no longer s4 master, so this can be done as well there

Stashbot added a comment.May 29 2020, 10:02 AM

Mentioned in SAL (#wikimedia-operations) [2020-05-29T10:02:37Z] <marostegui> Compress InnoDB on db1138 T232446

Marostegui updated the task description. (Show Details)Jun 1 2020, 8:19 AM
Marostegui changed the task status from Stalled to Open.Sep 7 2020, 5:07 AM
Marostegui moved this task from Blocked external/Not db team to In progress on the DBA board.
Stashbot added a comment.Sep 7 2020, 8:03 AM

Mentioned in SAL (#wikimedia-operations) [2020-09-07T08:03:39Z] <marostegui> Compress InnoDB on s8 eqiad master (db1109) - T232446

Marostegui closed this task as Resolved.Sep 14 2020, 5:13 AM
Marostegui updated the task description. (Show Details)

This is all done

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL