Page MenuHomePhabricator

Upgrade s7 to Debian Buster and MariaDB 10.4
Closed, ResolvedPublic

Description

Steps to upgrade:

Please read the doc about procedure for more details.

Original master: db2118
Original candidate master: db2121

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 710501 merged by Kormat:

[operations/puppet@production] db1181: Disable notifications for reimage.

https://gerrit.wikimedia.org/r/710501

Change 710502 had a related patch set uploaded (by Kormat; author: Kormat):

[operations/puppet@production] install_server: switch db1181 to buster

https://gerrit.wikimedia.org/r/710502

Change 710502 merged by Kormat:

[operations/puppet@production] install_server: switch db1181 to buster

https://gerrit.wikimedia.org/r/710502

Mentioned in SAL (#wikimedia-operations) [2021-08-06T09:58:33Z] <kormat> reimaging db1181 (s7) to buster T288244

Script wmf-auto-reimage was launched by kormat on cumin1001.eqiad.wmnet for hosts:

['db1181.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202108061000_kormat_23872.log.

Completed auto-reimage of hosts:

['db1181.eqiad.wmnet']

and were ALL successful.

Change 710946 had a related patch set uploaded (by Kormat; author: Kormat):

[operations/puppet@production] db1136: Disable notifications for reimage.

https://gerrit.wikimedia.org/r/710946

Change 710947 had a related patch set uploaded (by Kormat; author: Kormat):

[operations/puppet@production] install_server: Switch db1136 to buster

https://gerrit.wikimedia.org/r/710947

Change 710946 merged by Kormat:

[operations/puppet@production] db1136: Disable notifications for reimage.

https://gerrit.wikimedia.org/r/710946

Change 710947 merged by Kormat:

[operations/puppet@production] install_server: Switch db1136 to buster

https://gerrit.wikimedia.org/r/710947

Mentioned in SAL (#wikimedia-operations) [2021-08-09T11:04:52Z] <kormat@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on 15 hosts with reason: Reimage db1136 (s7 primary) to buster T288244

Mentioned in SAL (#wikimedia-operations) [2021-08-09T11:05:04Z] <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 15 hosts with reason: Reimage db1136 (s7 primary) to buster T288244

Script wmf-auto-reimage was launched by kormat on cumin1001.eqiad.wmnet for hosts:

['db1136.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202108091113_kormat_15919.log.

Completed auto-reimage of hosts:

['db1136.eqiad.wmnet']

and were ALL successful.

Change 710977 had a related patch set uploaded (by Jcrespo; author: Jcrespo):

[operations/puppet@production] dbbackups: Switch s7 backups from stretch to buster (db1171)

https://gerrit.wikimedia.org/r/710977

Change 710977 merged by Jcrespo:

[operations/puppet@production] dbbackups: Switch s7 backups from stretch (db1116) to buster (db1171)

https://gerrit.wikimedia.org/r/710977

Change 710981 had a related patch set uploaded (by Jcrespo; author: Jcrespo):

[operations/puppet@production] dbbackups: Switch s7 backups from stretch (db2100) to buster (db2098)

https://gerrit.wikimedia.org/r/710981

Change 711383 had a related patch set uploaded (by Jcrespo; author: Jcrespo):

[operations/puppet@production] dbbackups: Fix wrongly configured target host for s7 (buster)

https://gerrit.wikimedia.org/r/711383

Change 711383 merged by Jcrespo:

[operations/puppet@production] dbbackups: Fix wrongly configured target host for s7 (buster)

https://gerrit.wikimedia.org/r/711383

Change 713442 had a related patch set uploaded (by MVernon; author: MVernon):

[operations/puppet@production] disable notifications on db2121 for buster reimage

https://gerrit.wikimedia.org/r/713442

Change 713442 merged by MVernon:

[operations/puppet@production] db2121: Disable notifications for buster reimage

https://gerrit.wikimedia.org/r/713442

Mentioned in SAL (#wikimedia-operations) [2021-08-17T09:16:02Z] <Emperor> reimaging db2121 to buster T288244

Mentioned in SAL (#wikimedia-operations) [2021-08-17T09:20:45Z] <mvernon@cumin1001> dbctl commit (dc=all): 'db2121 depooling: reimage to buster T288244', diff saved to https://phabricator.wikimedia.org/P17033 and previous config saved to /var/cache/conftool/dbconfig/20210817-092045-mvernon.json

Script wmf-auto-reimage was launched by mvernon on cumin2002.codfw.wmnet for hosts:

['db2121.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202108170928_mvernon_829395.log.

Completed auto-reimage of hosts:

['db2121.codfw.wmnet']

and were ALL successful.

Mentioned in SAL (#wikimedia-operations) [2021-08-17T10:31:19Z] <mvernon@cumin1001> dbctl commit (dc=all): 'db2121 (re)pooling @ 25%: buster reimage T288244', diff saved to https://phabricator.wikimedia.org/P17034 and previous config saved to /var/cache/conftool/dbconfig/20210817-103118-mvernon.json

Mentioned in SAL (#wikimedia-operations) [2021-08-17T10:46:22Z] <mvernon@cumin1001> dbctl commit (dc=all): 'db2121 (re)pooling @ 50%: buster reimage T288244', diff saved to https://phabricator.wikimedia.org/P17035 and previous config saved to /var/cache/conftool/dbconfig/20210817-104622-mvernon.json

Mentioned in SAL (#wikimedia-operations) [2021-08-17T11:01:26Z] <mvernon@cumin1001> dbctl commit (dc=all): 'db2121 (re)pooling @ 75%: buster reimage T288244', diff saved to https://phabricator.wikimedia.org/P17037 and previous config saved to /var/cache/conftool/dbconfig/20210817-110125-mvernon.json

Mentioned in SAL (#wikimedia-operations) [2021-08-17T11:16:30Z] <mvernon@cumin1001> dbctl commit (dc=all): 'db2121 (re)pooling @ 100%: buster reimage T288244', diff saved to https://phabricator.wikimedia.org/P17038 and previous config saved to /var/cache/conftool/dbconfig/20210817-111629-mvernon.json

LSobanski triaged this task as Medium priority.Thu, Aug 19, 8:30 AM

Mentioned in SAL (#wikimedia-operations) [2021-08-25T06:08:27Z] <kormat@cumin1001> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on db2118.codfw.wmnet with reason: Reimaging T288244

Mentioned in SAL (#wikimedia-operations) [2021-08-25T06:08:31Z] <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db2118.codfw.wmnet with reason: Reimaging T288244

@jcrespo: s7/codfw primary switchover is complete, so please go ahead with the backup generation switch at your leisure.

Change 710981 merged by Jcrespo:

[operations/puppet@production] dbbackups: Switch s7 backups from stretch (db2100) to buster (db2098)

https://gerrit.wikimedia.org/r/710981

Only clean up of unused db1116:s7 db2100:s7 is missing regarding backups.

Change 714724 had a related patch set uploaded (by Kormat; author: Kormat):

[operations/puppet@production] db2118: Disable notifications during reimage.

https://gerrit.wikimedia.org/r/714724

Change 714725 had a related patch set uploaded (by Kormat; author: Kormat):

[operations/puppet@production] install_server: switch db2118 to buster

https://gerrit.wikimedia.org/r/714725

Change 714724 merged by Kormat:

[operations/puppet@production] db2118: Disable notifications during reimage.

https://gerrit.wikimedia.org/r/714724

Change 714725 merged by Kormat:

[operations/puppet@production] install_server: switch db2118 to buster

https://gerrit.wikimedia.org/r/714725

Script wmf-auto-reimage was launched by kormat on cumin2001.codfw.wmnet for hosts:

['db2118.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202108251106_kormat_10802.log.

Completed auto-reimage of hosts:

['db2118.codfw.wmnet']

Of which those FAILED:

['db2118.codfw.wmnet']

Reimaging failure seems likely to be a combination of running wmf-auto-reimage from cumin2001 instead of cumin2002, and major puppetdb slowness in codfw (T263578: puppetdb seems to be slow on host reimage).

I've manually run puppet, and rebooted the host, and ran puppet again.

mysqlcheck --all-databases running on db2118 now.

db2118 failed due to corruption

1Aug 25 12:48:00 db2118 mysqld[4586]: 2021-08-25 12:48:00 0 [Note] InnoDB: Buffer pool(s) load completed at 210825 12:48:00
2Aug 25 19:42:36 db2118 mysqld[4586]: 2021-08-25 19:42:36 0 [ERROR] InnoDB: Database page corruption on disk or a failed file read of tablespace metawiki/content page [page id: space=4936, page number=49350]. You may have to recover from a backup.
3Aug 25 19:42:36 db2118 mysqld[4586]: 2021-08-25 19:42:36 0 [Note] InnoDB: Page dump in ascii and hex (16384 bytes):
4Aug 25 19:42:36 db2118 mysqld[4586]: len 16384; hex 58252d810000c0c60000c0c50000c0c7000014f062d2a6a045bf00000000000000000000134800323b8c80c9000000003b47000200c600c7000000000000000000000000000000004d7e0000000000000000000000000000000000000000010002001c696e66696d756d0008000b000073757072656d756d0b1f000010004c0000000000967dde0004634a1a2dff000002a63b8e000003cd646368746a30797a616c74396e3834386d70396b73637130326b6c7074336c000174743a31303136363635310b1f000018004c0000000000967ddf0004634a1a2dff000002a63ba0000003cd646368746a30797a616c74396e3834386d70396b73637130326b6c7074336c000174743a31303136363635320b1f000020004c0000000000967de00004634a1a2dff000002a63bb2000003cd646368746a30797a616c74396e3834386d70396b73637130326b6c7074336c000174743a31303136363635330b1f040028004c0000000000967de10004634a1a2dff000002a63bc4000003cd646368746a30797a616c74396e3834386d70396b73637130326b6c7074336c000174743a31303136363635340b1f000030004c0000000000967de20004634a1a2dff000002a63bd6000003cd646368746a30797a616c74396e3834386d70396b73637130326b6c7074336c000174743a31303136363635350b1f000038004c0000000000967de30004634a1a2dff000002a63be8000003cd39733962726e7a763063683170306f7a7037656d6b6b706b376a3661356561000174743a31303136363635360b1f000040004c0000000000967de40004634a1a2dff000002a63bfa000003cd39733962726e7a763063683170306f7a7037656d6b6b706b376a3661356561000174743a31303136363635370b1f040048004c0000000000967de50004634a1a2dff000002a63c0c000003cd39733962726e7a763063683170306f7a7037656d6b6b706b376a3661356561000174743a31303136363635380b1f000050004c0000000000967de60004634a1a2dff000002a63c1e000003cd39733962726e7a763063683170306f7a7037656d6b6b706b376a3661356561000174743a31303136363635390b1f000058004c0000000000967de70004634a1a2dff000002a63c30000003cd39733962726e7a763063683170306f7a7037656d6b6b706b376a3661356561000174743a31303136363636300b1f000060004c0000000000967de80004634a1a2dff000002a63c42000003cd39733962726e7a763063683170306f7a7037656d6b6b706b376a3661356561000174743a31303136363636310b1f040068004c0000000000967de90004634a1a2dff000002a63c54000003cd6a736877786265356c3173723965696c6f3335627462356371706f6d75746e000174743a31303136363636320b1f000070004c0000000000967dea0004634a1a2dff000002a63c66000003cd6a736877786265356c3173723965696c6f3335627462356371706f6d75746e000174743a31303136363636330b1f000078004c0000000000967deb0004634a1a2dff000002a63c78000003cd6a736877786265356c3173723965696c6f3335627462356371706f6d75746e000174743a31303136363636340b1f000080004c0000000000967dec0004634a1a2dff000002a63c8a000003cd6a736877786265356c3173723965696c6f3335627462356371706f6d75746e000174743a31303136363636350b1f040088004c0000000000967ded0004634a1a2dff000002a63c9c000003cd6a736877786265356c3173723965696c6f3335627462356371706f6d75746e000174743a31303136363636360b1f000090004c0000000000967dee0004634a1a2dff000002a63cae000003cd6a736877786265356c3173723965696c6f3335627462356371706f6d75746e000174743a31303136363636370b1f000098004c0000000000967def0004634a1a2dff000002a63cc0000003cd6a736877786265356c3173723965696c6f3335627462356371706f6d75746e000174743a31303136363636380b1f0000a0004c0000000000967df00004634a1a2dff000002a63cd2000003cd6a736877786265356c3173723965696c6f3335627462356371706f6d75746e000174743a31303136363636390b1f0400a8004c0000000000967df10004634a1a2dff000002a63ce4000003cd6a736877786265356c3173723965696c6f3335627462356371706f6d75746e000174743a31303136363637300b1f0000b0004c0000000000967df20004634a1a2dff000002a63cf6000003cd6a736877786265356c3173723965696c6f3335627462356371706f6d75746e000174743a31303136363637310b1f0000b8004c0000000000967df30004634a1a2dff000002a63d08000003cd6a736877786265356c3173723965696c6f3335627462356371706f6d75746e000174743a31303136363637320b1f0000c0004c0000000000967df40004634a1a2dff000002a63d1a000003cd616b3837773335376d6e65623867316174716e32756d736d62646c6e756865000174743a31303136363637330b1f0400c8004c0000000000967df50004634a1a2dff000002a63d2c000003cd616b3837773335376d6e65623867316174716e32756d736d62646c6e756865000174743a31303136363637340b1f0000d0004c0000000000967df60004634a1a2dff000002a63d3e000003cd616b3837773335376d6e65623867316174716e32756d736d62646c6e756865000174743a31303136363637350b1f0000d8004c0000000000967df70004634a1a2dff000002a63d50000003cd616b3837773335376d6e65623867316174716e32756d736d62646c6e756865000174743a31303136363637360b1f0000e0004c0000000000967df80004634a1a2dff000002a63d62000003cd616b3837773335376d6e65623867316174716e32756d736d62646c6e756865000174743a31303136363637370b1f0400e8004c0000000000967df90004634a1a2dff000002a63d74000003cd616b3837773335376d6e65623867316174716e32756d736d62646c6e756865000174743a31303136363637380b1f0000f0004c0000000000967dfa0004634a1a2dff000002a63d86000003cd61336f786c7a65653065776876766d6a377037757937336a637a6376656534000174743a31303136363637390b1f0000f8004c0000000000967dfb0004634a1a2dff000002a63d98000003cd61336f786c7a65653065776876766d6a377037757937336a637a6376656534000174743a31303136363638300b1f000100004c0000000000967dfc0004634a1a2dff000002a63daa000003cd61336f786c7a65653065776876766d6a377037757937336a637a6376656534000174743a31303136363638310b1f040108004c0000000000967dfd0004634a1a2dff000002a63dbc000000b772333133756a656b65696c706632756d706a3871766f643574717230396635000174743a31303136363638320b1f000110004c0000000000967dfe0004634a1a2dff000002a63dce0000256a37793036726c6570367a6f697630346b796639356479703432386b6e683466000174743a31303136363638330b1f000118004c0000000000967dff0004634a1a2dff000002a63de0000003cd61336f786c7a65653065776876766d6a377037757937336a637a6376656534000174743a31303136363638340b1f000120004c0000000000967e000004634a1a2dff000002a63df2000003cd61336f786c7a65653065776876766d6a377037757937336a637a6376656534000174743a31303136363638350b1f040128004c0000000000967e010004634a1a2dff000002a63e04000003cd61336f786c7a65653065776876766d6a377037757937336a637a6376656534000174743a31303136363638360b1f000130004c0000000000967e020004634a1a2dff000002a63e16000003cd61336f786c7a65653065776876766d6a377037757937336a637a6376656534000174743a31303136363638370b1f000138004c0000000000967e030004634a1a2dff000002a63e28000003cd61336f786c7a65653065776876766d6a377037757937336a637a6376656534000174743a31303136363638380b1f000140004c0000000000967e040004634a1a2dff000002a63e3a000003cd3472306e6471366838377a306b7336703168647a7537376a3674676d373077000174743a31303136363638390b1f040148004c0000000000967e050004634a1a2dff000002a63e4c000003cd3472306e6471366838377a306b7336703168647a7537376a3674676d373077000174743a31303136363639300b1f000150004c0000000000967e060004634a1a2dff000002a63e5e000003cd3472306e6471366838377a306b7336703168647a7537376a3674676d373077000174743a31303136363639310b1f000158004c0000000000967e070004634a1a2dff000002a63e70000003cd3472306e6471366838377a306b7336703168647a7537376a3674676d373077000174743a31303136363639320b1f000160004c0000000000967e080004634a1a2dff000002a63e82000003cd3472306e6471366838377a306b7336703168647a7537376a3674676d373077000174743a31303136363639330b1f040168004c0000000000967e090004634a1a2dff000002a63e94000003cd3472306e6471366838377a306b7336703168647a7537376a3674676d373077000174743a31303136363639340b1f000170004c0000000000967e0a0004634a1a2dff000002a63ea6000003cd3472306e6471366838377a306b7336703168647a7537376a3674676d373077000174743a31303136363639350b1f000178004c0000000000967e0b0004634a1a2dff000002a63eb8000003cd3472306e6471366838377a306b7336703168647a7537376a3674676d373077000174743a31303136363639360b1f000180004c0000000000967e0c0004634a1a2dff000002a63eca000003cd3472306e6471366838377a306b7336703168647a7537376a3674676d373077000174743a31303136363639370b1f040188004c0000000000967e0d0004634a1a2dff000002a63edc000003cd6d6c37346b613279347867787334797164747077777335626c376c3532686d000174743a31303136363639380b1f000190004c0000000000967e0e0004634a1a2dff000002a63eee000003cd6d6c37346b613279347867787334797164747077777335626c376c3532686d000174743a31303136363639390b1f000198004c0000000000967e0f0004634a1a2dff000002a63f00000003cd613531636839767873747635617a72333932766f6a7867376e61626e6f3176000174743a31303136363730300b1f0001a0004c0000000000967e100004634a1a2dff000002a63f12000003cd613531636839767873747635617a72333932766f6a7867376e61626e6f3176000174743a31303136363730310b1f0401a8004c0000000000967e110004634a1a2dff000002a63f24000003cd613531636839767873747635617a72333932766f6a7867376e61626e6f3176000174743a31303136363730320b1f0001b0004c0000000000967e120004634a1a2dff000002a63f36000003cd6d64677968686b6e74326966706f7a646c6f366d74723035766c3033393271000174743a31303136363730330b1f0001b8004c0000000000967e130004634a1a2dff000002a63f48000003cd6d64677968686b6e74326966706f7a646c6f366d74723035766c3033393271000174743a31303136363730340b1f0001c0004c0000000000967e140004634a1a2dff000002a63f5a000003cd6d64677968686b6e74326966706f7a646c6f366d74723035766c3033393271000174743a31303136363730350b1f0401c8004c0000000000967e150004634a1a2dff000002a63f6c000003cd6d64677968686b6e74326966706f7a646c6f366d74723035766c3033393271000174743a31303136363730360b1f0001d0004c0000000000967e160004634a1a2dff000002a63f7e000003cd6d64677968686b6e74326966706f7a646c6f366d74723035766c3033393271000174743a31303136363730370b1f0001d8004c0000000000967e170004634a1a2dff000002a63f90000003cd6d7631393231363863627076327a397a347366707a61336f616474666a6962000174743a31303136363730380b1f0001e0004c0000000000967e180004634a1a2dff000002a63fa2000000e16f6a3076717a6976687370646d323079646379636c63646d79627462347969000174743a31303136363730390b1f0401e8004c0000000000967e190004634a1a2dff000002a63fb4000003cd6d7631393231363863627076327a397a347366707a61336f616474666a6962000174743a31303136363731300b1f0001f0004c0000000000967e1a0004634a1a2dff000002a63fc600002581636c6965303638673476673276747a6d3172646b67796171386d723663716e000174743a31303136363731310b1f0001f8004c0000000000967e1b0004634a1a2dff000002a70038000003cd6d7631393231363863627076327a397a347366707a61336f616474666a6962000174743a31303136363731320b1f000200004c0000000000967e1c0004634a1a2dff000002a7004a000003cd6f32306f363566346c78376635326668773369743568397864376c796b6177000174743a31303136363731330b1f040208004c0000000000967e1d0004634a1a2dff000002a7005c000003cd6f32306f363566346c78376635326668773369743568397864376c796b6177000174743a31303136363731340b1f000210004c0000000000967e1e0004634a1a2dff000002a7006e000003cd6f32306f363566346c78376635326668773369743568397864376c796b6177000174743a31303136363731350b1f000218004c0000000000967e1f0004634a1a2dff000002a70080000003cd6f32306f363566346c78376635326668773369743568397864376c796b6177000174743a31303136363731360b1f000220004c0000000000967e200004634a1a2dff000002a70092000003cd6f32306f363566346c78376635326668773369743568397864376c796b6177000174743a31303136363731370b1f040228004c0000000000967e210004634a1a2dff000002a700a4000003cd6f32306f363566346c78376635326668773369743568397864376c796b6177000174743a31303136363731380b1f000230004c0000000000967e220004634a1a2dff000002a700b6000003cd6f32306f363566346c78376635326668773369743568397864376c796b6177000174743a31303136363731390b1f000238004c0000000000967e230004634a1a2dff000002a700c8000003cd337268796f6c6b306c706a63396a6631697533327a77727563306f697a6973000174743a31303136363732300b1f000240004c0000000000967e240004634a1a2dff000002a700da000003cd337268796f6c6b306c706a63396a6631697533327a77727563306f697a6973000174743a31303136363732310b1f040248004c0000000000967e250004634a1a2dff000002a700ec000003cd337268796f6c6b306c706a63396a6631697533327a77727563306f697a6973000174743a31303136363732320b1f000250004c0000000000967e260004634a1a2dff000002a700fe000003cd337268796f6c6b306c706a63396a6631697533327a77727563306f697a6973000174743a31303136363732330b1f000258004c0000000000967e270004634a1a2dff000002a70110000003cd337268796f6c6b306c706a63396a6631697533327a77727563306f697a6973000174743a31303136363732340b1f000260004c0000000000967e280004634a1a2dff000002a70122000003cd337268796f6c6b306c706a63396a6631697533327a77727563306f697a6973000174743a31303136363732350b1f040268004c0000000000967e290004634a1a2dff000002a70134000003cd337268796f6c6b306c706a63396a6631697533327a77727563306f697a6973000174743a31303136363732360b1f000270004c0000000000967e2a0004634a1a2dff000002a70146000003cd337268796f6c6b306c706a63396a6631697533327a77727563306f697a6973000174743a31303136363732370b1f000278004c0000000000967e2b0004634a1a2dff000002a70158000003cd337268796f6c6b306c706a63396a6631697533327a77727563306f697a6973000174743a31303136363732380b1f000280004c0000000000967e2c0004634a1a2dff000002a7016a000003cd6639637a3579346a733568626a343763747a733269793673306a7a74327075000174743a31303136363732390b1f040288004c0000000000967e2d0004634a1a2dff000002a7017c000003cd6639637a3579346a733568626a343763747a733269793673306a7a74327075000174743a31303136363733300b1f000290004c0000000000967e2e0004634a1a2dff000002a7018e000003cd6a627568673578716e6b696261676e6b6736746769766268396736327a3136000174743a31303136363733310b1f000298004c0000000000967e2f0004634a1a2dff000002a701a0000003cd6a627568673578716e6b696261676e6b6736746769766268396736327a3136000174743a31303136363733320b1f0002a0004c0000000000967e300004634a1a2dff000002a701b2000003cd6a627568673578716e6b696261676e6b6736746769766268396736327a3136000174743a31303136363733330b1f0402a8004c0000000000967e310004634a1a2dff000002a701c4000000eb6362726b6a6a746a69666c7236636b6a6362617364396c7972317a75336b31000174743a31303136363733340b1f0002b0004c0000000000967e320004634a1a2dff000002a701d6000025cb63673032376f707273773337396f3630303970396530387172626670303173000174743a31303136363733350b1f0002b8004c0000000000967e330004634a1a2dff000002a701e8000003cd6a627568673578716e6b696261676e6b6736746769766268396736327a3136000174743a31303136363733360b1f0002c0004c0000000000967e340004634a1a2dff000002a701fa000003cd6a627568673578716e6b696261676e6b6736746769766268396736327a3136000174743a31303136363733370b1f0402c8004c0000000000967e350004634a1a2dff000002a7020c000003cd6572657832733177337a75746273643464613868316e677871717933356a30000174743a31303136363733380b1f0002d0004c0000000000967e360004634a1a2dff000002a7021e000003cd6572657832733177337a75746273643464613868316e677871717933356a30000174743a31303136363733390b1f0002d8004c0000000000967e370004634a1a2dff000002a70230000003cd6572657832733177337a75746273643464613868316e677871717933356a30000174743a31303136363734300b1f0002e0004c0000000000967e380004634a1a2dff000002a70242000003cd6572657832733177337a75746273643464613868316e677871717933356a30000174743a31303136363734310b1f0402e8004c0000000000967e390004634a1a2dff000002a70254000003cd6572657832733177337a75746273643464613868316e677871717933356a30000174743a31303136363734320b1f0002f0004c0000000000967e3a0004634a1a2dff000002a70266000003cd33766f703479326832616c61376d34716c34726e6a39336a76376a6a306633000174743a31303136363734330b1f0002f8004c0000000000967e3b0004634a1a2dff000002a70278000003cd33766f703479326832616c61376d34716c34726e6a39336a76376a6a306633000174743a31303136363734340b1f000300004c0000000000967e3c0004634a1a2dff000002a7028a000003cd33766f703479326832616c61376d34716c34726e6a39336a76376a6a306633000174743a31303136363734350b1f040308004c0000000000967e3d0004634a1a2dff000002a7029c000003cd33766f703479326832616c61376d34716c34726e6a39336a76376a6a306633000174743a31303136363734360b1f000310004c0000000000967e3e0004634a1a2dff000002a702ae000003cd366a3176706e6b34786a76727139626d6c6f773067303966617667706e7663000174743a31303136363734370b1f000318004c0000000000967e3f0004634a1a2dff000002a702c0000003cd366a3176706e6b34786a76727139626d6c6f773067303966617667706e7663000174743a31303136363734380b1f000320004c0000000000967e400004634a1a2dff000002a702d2000003cd366a3176706e6b34786a76727139626d6c6f773067303966617667706e7663000174743a31303136363734390b1f040328004c0000000000967e410004634a1a2dff000002a702e4000003cd366a3176706e6b34786a76727139626d6c6f773067303966617667706e7663000174743a31303136363735300b1f000330004c0000000000967e420004634a1a2dff000002a702f6000003cd366a3176706e6b34786a76727139626d6c6f773067303966617667706e7663000174743a31303136363735310b1f000338004c0000000000967e430004634a1a2dff000002a70308000003cd366a3176706e6b34786a76727139626d6c6f773067303966617667706e7663000174743a31303136363735320b1f000340004c0000000000967e440004634a1a2dff000002a7031a000003cd366a3176706e6b34786a76727139626d6c6f773067303966617667706e7663000174743a31303136363735330b1f040348004c0000000000967e450004634a1a2dff000002a7032c000003cd366a3176706e6b34786a76727139626d6c6f773067303966617667706e7663000174743a31303136363735340b1f000350004c0000000000967e460004634a1a2dff000002a7033e000003cd6b3266356961397969693637786c646476367574616c6a386470726b763275000174743a31303136363735350b1f000358004c0000000000967e470004634a1a2dff000002a70350000003cd6b3266356961397969693637786c646476367574616c6a386470726b763275000174743a31303136363735360b1f000360004c0000000000967e480004634a1a2dff000002a70362000003cd6b3266356961397969693637786c646476367574616c6a386470726b763275000174743a31303136363735370b1f040368004c0000000000967e490004634a1a2dff000002a70374000003cd6b3266356961397969693637786c646476367574616c6a386470726b763275000174743a31303136363735380b1f000370004c0000000000967e4a0004634a1a2dff000002a70386000003cd6b3266356961397969693637786c646476367574616c6a386470726b763275000174743a31303136363735390b1f000378004c0000000000967e4b0004634a1a2dff000002a70398000003cd6b3266356961397969693637786c646476367574616c6a386470726b763275000174743a31303136363736300b1f000380004c0000000000967e4c0004634a1a2dff000002a703aa000003cd6b3266356961397969693637786c646476367574616c6a386470726b763275000174743a31303136363736310b1f040388004c0000000000967e4d0004634a1a2dff000002a703bc000003cd6b3266356961397969693637786c646476367574616c6a386470726b763275000174743a31303136363736320b1f000390004c0000000000967e4e0004634a1a2dff000002a703ce000003cd6b3266356961397969693637786c646476367574616c6a386470726b763275000174743a31303136363736330b1f000398004c0000000000967e4f0004634a1a2dff000002a703e0000003cd6b3266356961397969693637786c646476367574616c6a386470726b763275000174743a31303136363736340b1f0003a0004c0000000000967e500004634a1a2dff000002a703f2000003cd6b3266356961397969693637786c646476367574616c6a386470726b763275000174743a31303136363736350b1f0403a8004c0000000000967e510004634a1a2dff000002a70404000003cd6b3266356961397969693637786c646476367574616c6a386470726b763275000174743a31303136363736360b1f0003b0004c0000000000967e520004634a1a2dff000002a70416000003cd633335796472396d797039306c7770646a61783334623730757370786c3361000174743a31303136363736370b1f0003b8004c0000000000967e530004634a1a2dff000002a70428000003cd633335796472396d797039306c7770646a61783334623730757370786c3361000174743a31303136363736380b1f0003c0004c0000000000967e540004634a1a2dff000002a7043a000003cd633335796472396d797039306c7770646a61783334623730757370786c3361000174743a31303136363736390b1f0403c8004c0000000000967e550004634a1a2dff000002a7044c000003cd633335796472396d797039306c7770646a61783334623730757370786c3361000174743a31303136363737300b1f0003d0004c0000000000967e560004634a1a2dff000002a7045e000003cd633335796472396d797039306c7770646a61783334623730757370786c3361000174743a31303136363737310b1f0003d8004c0000000000967e570004634a1a2dff000002a70470000055f46e63373863643938706d38397a6a6973616779373775357075383678753172000174743a31303136363737320b1f0003e0004c0000000000967e580004634a1a2dff000002a70482000003cd686d3234736b6a376e68687365786932717479343573746b75643938767766000174743a31303136363737330b1f0403e8004c0000000000967e590004634a1a2dff000002a70494000003cd686d3234736b6a376e68687365786932717479343573746b75643938767766000174743a31303136363737340b1f0003f0004c0000000000967e5a0004634a1a2dff000002a704a6000003cd686d3234736b6a376e68687365786932717479343573746b75643938767766000174743a31303136363737350b1f0003f8004c0000000000967e5b0004634a1a2dff000002a704b8000003cd686d3234736b6a376e68687365786932717479343573746b75643938767766000174743a31303136363737360b1f000400004c0000000000967e5c0004634a1a2dff000002a704ca000055f46e63373863643938706d38397a6a6973616779373775357075383678753172000174743a31303136363737370b1f040408004c0000000000967e5d0004634a1a2dff000002a704dc000003cd686d3234736b6a376e68687365786932717479343573746b75643938767766000174743a31303136363737380b1f000410004c0000000000967e5e0004634a1a2dff000002a704ee000003cd686d3234736b6a376e68687365786932717479343573746b75643938767766000174743a31303136363737390b1f000418004c0000000000967e5f0004634a1a2dff000002a70500000003cd686d3234736b6a376e68687365786932717479343573746b75643938767766000174743a31303136363738300b1f000420004c0000000000967e600004634a1a2dff000002a70512000003cd686d3234736b6a376e68687365786932717479343573746b75643938767766000174743a31303136363738310b1f040428004c0000000000967e610004634a1a2dff000002a70524000003cd703475356376733732396b63706f7875777375756e7867656c713464753266000174743a31303136363738320b1f000430004c0000000000967e620004634a1a2dff000002a70536000003cd703475356376733732396b63706f7875777375756e7867656c713464753266000174743a31303136363738330b1f000438004c0000000000967e630004634a1a2dff000002a70548000003cd703475356376733732396b63706f7875777375756e7867656c713464753266000174743a31303136363738340b1f000440004c0000000000967e640004634a1a2dff000002a7055a000003cd703475356376733732396b63706f7875777375756e7867656c713464753266000174743a31303136363738350b1f040448004c0000000000967e650004634a1a2dff000002a7056c000003cd703475356376733732396b63706f7875777375756e7867656c713464753266000174743a31303136363738360b1f000450004c0000000000967e660004634a1a2dff000002a7057e000003cd703475356376733732396b63706f7875777375756e7867656c713464753266000174743a31303136363738370b1f000458004c0000000000967e670004634a1a2dff000002a70590000003cd65637461306d6e35316e7a7661777070676368726130366b763678346e6b38000174743a31303136363738380b1f000460004c0000000000967e680004634a1a2dff000002a705a2000003cd65637461306d6e35316e7a7661777070676368726130366b763678346e6b38000174743a31303136363738390b1f040468004c0000000000967e690004634a1a2dff000002a705b4000003cd6665786b6a6279786f756368396d30666130616b6f74367a36747132387a71000174743a31303136363739300b1f000470004c0000000000967e6a0004634a1a2dff000002a705c6000003cd6665786b6a6279786f756368396d30666130616b6f74367a36747132387a71000174743a31303136363739310b1f000478004c0000000000967e6b0004634a1a2dff000002a705d8000003cd6665786b6a6279786f756368396d30666130616b6f74367a36747132387a71000174743a31303136363739320b1f000480004c0000000000967e6c0004634a1a2dff000002a705ea000003cd6665786b6a6279786f756368396d30666130616b6f74367a36747132387a71000174743a31303136363739330b1f040488004c0000000000967e6d0004634a1a2dff000002a705fc000003cd6665786b6a6279786f756368396d30666130616b6f74367a36747132387a71000174743a31303136363739340b1f000490004c0000000000967e6e0004634a1a2dff000002a7060e000003cd6665786b6a6279786f756368396d30666130616b6f74367a36747132387a71000174743a31303136363739350b1f000498004c0000000000967e6f0004634a1a2dff000002a70620000003cd6665786b6a6279786f756368396d30666130616b6f74367a36747132387a71000174743a31303136363739360b1f0004a0004c0000000000967e700004634a1a2dff000002a70632000003cd6665786b6a6279786f756368396d30666130616b6f74367a36747132387a71000174743a31303136363739370b1f0404a8004c0000000000967e710004634a1a2dff000002a70644000003cd6665786b6a6279786f756368396d30666130616b6f74367a36747132387a71000174743a31303136363739380b1f0004b0004c0000000000967e720004634a1a2dff000002a706560000d36c6c783567643869396a3832656f66727367776b357a363676627a30636f6276000174743a31303136363739390b1f0004b8004c0000000000967e730004634a1a2dff000002a70668000003cd6d327073786138656538343762616c32666361796c6d61356877667733707a000174743a31303136363830300b1f0004c0004c0000000000967e740004634a1a2dff000002a7067a000003cd6d327073786138656538343762616c32666361796c6d61356877667733707a000174743a31303136363830310b1f0404c8004c0000000000967e750004634a1a2dff000002a7068c000003cd6d327073786138656538343762616c32666361796c6d61356877667733707a000174743a31303136363830320b1f0004d0004c0000000000967e760004634a1a2dff000002a7069e000003cd6d327073786138656538343762616c32666361796c6d61356877667733707a000174743a31303136363830330b1f0004d8004c0000000000967e770004634a1a2dff000002a706b0000003cd6d327073786138656538343762616c32666361796c6d61356877667733707a000174743a31303136363830340b1f0004e0004c0000000000967e780004634a1a2dff000002a706c2000003cd6d327073786138656538343762616c32666361796c6d61356877667733707a000174743a31303136363830350b1f0404e8004c0000000000967e790004634a1a2dff000002a706d4000003cd6d327073786138656538343762616c32666361796c6d61356877667733707a000174743a31303136363830360b1f0004f0004c0000000000967e7a0004634a1a2dff000002a706e6000003cd6d327073786138656538343762616c32666361796c6d61356877667733707a000174743a31303136363830370b1f0004f8004c0000000000967e7b0004634a1a2dff000002a706f8000003cd6d327073786138656538343762616c32666361796c6d61356877667733707a000174743a31303136363830380b1f000500004c0000000000967e7c0004634a1a2dff000002a7070a000003cd736737633662746a7679377574743230383134767a6b6b306c6d3565616e66000174743a31303136363830390b1f040508004c0000000000967e7d0004634a1a2dff000002a7071c000003cd736737633662746a7679377574743230383134767a6b6b306c6d3565616e66000174743a31303136363831300b1f000510004c0000000000967e7e0004634a1a2dff000002a7072e000003cd736737633662746a7679377574743230383134767a6b6b306c6d3565616e66000174743a31303136363831310b1f000518004c0000000000967e7f0004634a1a2dff000002a70740000003cd736737633662746a7679377574743230383134767a6b6b306c6d3565616e66000174743a31303136363831320b1f000520004c0000000000967e800004634a1a2dff000002a70752000003cd736737633662746a7679377574743230383134767a6b6b306c6d3565616e66000174743a31303136363831330b1f040528004c0000000000967e810004634a1a2dff000002a70764000003cd736737633662746a7679377574743230383134767a6b6b306c6d3565616e66000174743a31303136363831340b1f000530004c0000000000967e820004634a1a2dff000002a70776000003cd736737633662746a7679377574743230383134767a6b6b306c6d3565616e66000174743a31303136363831350b1f000538004c0000000000967e830004634a1a2dff000002a70788000003cd736737633662746a7679377574743230383134767a6b6b306c6d3565616e66000174743a31303136363831360b1f000540004c0000000000967e840004634a1a2dff000002a7079a000003cd736737633662746a7679377574743230383134767a6b6b306c6d3565616e66000174743a31303136363831370b1f040548004c0000000000967e850004634a1a2dff000002a707ac0000be813933747a79693838316869316f337a646b36696e79626b6265766730707568000174743a31303136363831380b1f000550004c0000000000967e860004634a1a2dff000002a707be000003cd6137707a38797462677330787030356a6265746539306563766f7a79783030000174743a31303136363831390b1f000558004c0000000000967e870004634a1a2dff000002a707d0000003cd6137707a38797462677330787030356a6265746539306563766f7a79783030000174743a31303136363832300b1f000560004c0000000000967e880004634a1a2dff000002a707e2000055f46c6c74646e6c6c316b386d6e787236373870766a6530313763307373323569000174743a31303136363832310b1f040568004c0000000000967e890004634a1a2dff000002a707f4000003cd6137707a38797462677330787030356a6265746539306563766f7a79783030000174743a31303136363832320b1f000570004c0000000000967e8a0004634a1a2dff000002a70806000003cd6137707a38797462677330787030356a6265746539306563766f7a79783030000174743a31303136363832330b1f000578004c0000000000967e8b0004634a1a2dff000002a70818000003cd6137707a38797462677330787030356a6265746539306563766f7a79783030000174743a31303136363832350b1f000580004c0000000000967e8c0004634a1a2dff000002a7082a000003cd6137707a38797462677330787030356a6265746539306563766f7a79783030000174743a31303136363832360b1f040588004c0000000000967e8d0004634a1a2dff000002a7083c000003cd6137707a38797462677330787030356a6265746539306563766f7a79783030000174743a31303136363832370b1f000590004c0000000000967e8e0004634a1a2dff000002a7084e000003cd7079713430327a6f767a666e717576636968687663647574386c3868396570000174743a31303136363832380b1f000598004c0000000000967e8f0004634a1a2dff000002a70860000003cd7079713430327a6f767a666e717576636968687663647574386c3868396570000174743a31303136363832390b1f0005a0004c0000000000967e900004634a1a2dff000002a70872000003cd7079713430327a6f767a666e717576636968687663647574386c3868396570000174743a31303136363833300b1f0405a8004c0000000000967e910004634a1a2dff000002a70884000003cd7079713430327a6f767a666e717576636968687663647574386c3868396570000174743a31303136363833310b1f0005b0004c0000000000967e920004634a1a2dff000002a70896000003cd7079713430327a6f767a666e717576636968687663647574386c3868396570000174743a31303136363833320b1f0005b8004c0000000000967e930004634a1a2dff000002a708a8000003cd7079713430327a6f767a666e717576636968687663647574386c3868396570000174743a31303136363833330b1f0005c0004c0000000000967e940004634a1a2dff000002a708ba000003cd7079713430327a6f767a666e717576636968687663647574386c3868396570000174743a31303136363833340b1f0405c8004c0000000000967e950004634a1a2dff000002a708cc000003cd7079713430327a6f767a666e717576636968687663647574386c3868396570000174743a31303136363833350b1f0005d0004c0000000000967e960004634a1a2dff000002a708de000003cd7079713430327a6f767a666e717576636968687663647574386c3868396570000174743a31303136363833360b1f0005d8004c0000000000967e970004634a1a2dff000002a708f0000003cd31356f3479356130367774706b7761637a706f71666a7939386f6267317831000174743a31303136363833370b1f0005e0004c0000000000967e980004634a1a2dff000002a70902000003cd31356f3479356130367774706b7761637a706f71666a7939386f6267317831000174743a31303136363833380b1f0405e8004c0000000000967e990004634a1a2dff000002a70914000003cd31356f3479356130367774706b7761637a706f71666a7939386f6267317831000174743a31303136363833390b1f0005f0004c0000000000967e9a0004634a1a2dff000002a70926000003cd31356f3479356130367774706b7761637a706f71666a7939386f6267317831000174743a31303136363834300b1f0005f8004c0000000000967e9b0004634a1a2dff000002a70938000003cd31356f3479356130367774706b7761637a706f71666a7939386f6267317831000174743a31303136363834310b1f000600004c0000000000967e9c0004634a1a2dff000002a7094a0000032d64356e616c6a627335636e667a67683972786a666f33737833383966726e6f000174743a31303136363834320b1f040608004c0000000000967e9d0004634a1a2dff000002a7095c000003cd31356f3479356130367774706b7761637a706f71666a7939386f6267317831000174743a31303136363834330b1f000610004c0000000000967e9e0004634a1a2dff000002a7096e000003cd31356f3479356130367774706b7761637a706f71666a7939386f6267317831000174743a31303136363834340b1f000618004c0000000000967e9f0004634a1a2dff000002a7098000004ea36d6963396f766d65787431716b77777033367763706161696a763075653863000174743a31303136363834350b1f000620004c0000000000967ea00004634a1a2dff000002a70992000003cd31356f3479356130367774706b7761637a706f71666a7939386f6267317831000174743a31303136363834360b1f000628004c0000000000967ea10004634a1a2dff000002a709a4000003cd6b63676c3371726f62676d74753775787578716a797267726d613171393938000174743a31303136363834370b1f000630004c0000000000967ea20004634a1a2dff000002a709b6000003cd6b63676c3371726f62676d74753775787578716a797267726d613171393938000174743a31303136363834380b1f000638004c0000000000967ea30004634a1a2dff000002a709c8000003cd6b63676c3371726f62676d74753775787578716a797267726d613171393938000174743a31303136363834390b1f000640c5290000000000967ea40004634a1a2dff000002a709da00000022367a706b3034676369326a796e35717563776b6a7071373674706772396b68000174743a313031363638353000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000703933380336d335a334733343321330e32fb32e832d532c232af329c3289327632633250323d322a3217320431f131de31cb31b831a53192317f316c3159314631333120310d30fa30e730d430c130ae309b308830753062304f303c302930163006366e0573362d2a6a0; asc X%- b E H 2; ;G M~ infimum supremum L } cJ - ; dchtj0yzalt9n848mp9kscq02klpt3l tt:10166651 L } cJ - ; dchtj0yzalt9n848mp9kscq02klpt3l tt:10166652 L } cJ - ; dchtj0yzalt9n848mp9kscq02klpt3l tt:10166653 ( L } cJ - ; dchtj0yzalt9n848mp9kscq02klpt3l tt:10166654 0 L } cJ - ; dchtj0yzalt9n848mp9kscq02klpt3l tt:10166655 8 L } cJ - ; 9s9brnzv0ch1p0ozp7emkkpk7j6a5ea tt:10166656 @ L } cJ - ; 9s9brnzv0ch1p0ozp7emkkpk7j6a5ea tt:10166657 H L } cJ - < 9s9brnzv0ch1p0ozp7emkkpk7j6a5ea tt:10166658 P L } cJ - < 9s9brnzv0ch1p0ozp7emkkpk7j6a5ea tt:10166659 X L } cJ - <0 9s9brnzv0ch1p0ozp7emkkpk7j6a5ea tt:10166660 ` L } cJ - <B 9s9brnzv0ch1p0ozp7emkkpk7j6a5ea tt:10166661 h L } cJ - <T jshwxbe5l1sr9eilo35btb5cqpomutn tt:10166662 p L } cJ - <f jshwxbe5l1sr9eilo35btb5cqpomutn tt:10166663 x L } cJ - <x jshwxbe5l1sr9eilo35btb5cqpomutn tt:10166664 L } cJ - < jshwxbe5l1sr9eilo35btb5cqpomutn tt:10166665 L } cJ - < jshwxbe5l1sr9eilo35btb5cqpomutn tt:10166666 L } cJ - < jshwxbe5l1sr9eilo35btb5cqpomutn tt:10166667 L } cJ - < jshwxbe5l1sr9eilo35btb5cqpomutn tt:10166668 L } cJ - < jshwxbe5l1sr9eilo35btb5cqpomutn tt:10166669 L } cJ - < jshwxbe5l1sr9eilo35btb5cqpomutn tt:10166670 L } cJ - < jshwxbe5l1sr9eilo35btb5cqpomutn tt:10166671 L } cJ - = jshwxbe5l1sr9eilo35btb5cqpomutn tt:10166672 L } cJ - = ak87w357mneb8g1atqn2umsmbdlnuhe tt:10166673 L } cJ - =, ak87w357mneb8g1atqn2umsmbdlnuhe tt:10166674 L } cJ - => ak87w357mneb8g1atqn2umsmbdlnuhe tt:10166675 L } cJ - =P ak87w357mneb8g1atqn2umsmbdlnuhe tt:10166676 L } cJ - =b ak87w357mneb8g1atqn2umsmbdlnuhe tt:10166677 L } cJ - =t ak87w357mneb8g1atqn2umsmbdlnuhe tt:10166678 L } cJ - = a3oxlzee0ewhvvmj7p7uy73jczcvee4 tt:10166679 L } cJ - = a3oxlzee0ewhvvmj7p7uy73jczcvee4 tt:10166680 L } cJ - = a3oxlzee0ewhvvmj7p7uy73jczcvee4 tt:10166681 L } cJ - = r313ujekeilpf2umpj8qvod5tqr09f5 tt:10166682 L } cJ - = %j7y06rlep6zoiv04kyf95dyp428knh4f tt:10166683 L } cJ - = a3oxlzee0ewhvvmj7p7uy73jczcvee4 tt:10166684 L ~ cJ - = a3oxlzee0ewhvvmj7p7uy73jczcvee4 tt:10166685 ( L ~ cJ - > a3oxlzee0ewhvvmj7p7uy73jczcvee4 tt:10166686 0 L ~ cJ - > a3oxlzee0ewhvvmj7p7uy73jczcvee4 tt:10166687 8 L ~ cJ - >( a3oxlzee0ewhvvmj7p7uy73jczcvee4 tt:10166688 @ L ~ cJ - >: 4r0ndq6h87z0ks6p1hdzu77j6tgm70w tt:10166689 H L ~ cJ - >L 4r0ndq6h87z0ks6p1hdzu77j6tgm70w tt:10166690 P L ~ cJ - >^ 4r0ndq6h87z0ks6p1hdzu77j6tgm70w tt:10166691 X L ~ cJ - >p 4r0ndq6h87z0ks6p1hdzu77j6tgm70w tt:10166692 ` L ~ cJ - > 4r0ndq6h87z0ks6p1hdzu77j6tgm70w tt:10166693 h L ~ cJ - > 4r0ndq6h87z0ks6p1hdzu77j6tgm70w tt:10166694 p L ~ cJ - > 4r0ndq6h87z0ks6p1hdzu77j6tgm70w tt:10166695 x L ~ cJ - > 4r0ndq6h87z0ks6p1hdzu77j6tgm70w tt:10166696 L ~ cJ - > 4r0ndq6h87z0ks6p1hdzu77j6tgm70w tt:10166697 L ~ cJ - > ml74ka2y4xgxs4yqdtpwws5bl7l52hm tt:10166698 L ~ cJ - > ml74ka2y4xgxs4yqdtpwws5bl7l52hm tt:10166699 L ~ cJ - ? a51ch9vxstv5azr392vojxg7nabno1v tt:10166700 L ~ cJ - ? a51ch9vxstv5azr392vojxg7nabno1v tt:10166701 L ~ cJ - ?$ a51ch9vxstv5azr392vojxg7nabno1v tt:10166702 L ~ cJ - ?6 mdgyhhknt2ifpozdlo6mtr05vl0392q tt:10166703 L ~ cJ - ?H mdgyhhknt2ifpozdlo6mtr05vl0392q tt:10166704 L ~ cJ - ?Z mdgyhhknt2ifpozdlo6mtr05vl0392q tt:10166705 L ~ cJ - ?l mdgyhhknt2ifpozdlo6mtr05vl0392q tt:10166706 L ~ cJ - ?~ mdgyhhknt2ifpozdlo6mtr05vl0392q tt:10166707 L ~ cJ - ? mv192168cbpv2z9z4sfpza3oadtfjib tt:10166708 L ~ cJ - ? oj0vqzivhspdm20ydcyclcdmybtb4yi tt:10166709 L ~ cJ - ? mv192168cbpv2z9z4sfpza3oadtfjib tt:10166710 L ~ cJ - ? % clie068g4vg2vtzm1rdkgyaq8mr6cqn tt:10166711 L ~ cJ - 8 mv192168cbpv2z9z4sfpza3oadtfjib tt:10166712 L ~ cJ - J o20o65f4lx7f52fhw3it5h9xd7lykaw tt:10166713 L ~ cJ - \ o20o65f4lx7f52fhw3it5h9xd7lykaw tt:10166714 L ~ cJ - n o20o65f4lx7f52fhw3it5h9xd7lykaw tt:10166715 L ~ cJ - o20o65f4lx7f52fhw3it5h9xd7lykaw tt:10166716 L ~ cJ - o20o65f4lx7f52fhw3it5h9xd7lykaw tt:10166717 ( L ~! cJ - o20o65f4lx7f52fhw3it5h9xd7lykaw tt:10166718 0 L ~" cJ - o20o65f4lx7f52fhw3it5h9xd7lykaw tt:10166719 8 L ~# cJ - 3rhyolk0lpjc9jf1iu32zwruc0oizis tt:10166720 @ L ~$ cJ - 3rhyolk0lpjc9jf1iu32zwruc0oizis tt:10166721 H L ~% cJ - 3rhyolk0lpjc9jf1iu32zwruc0oizis tt:10166722 P L ~& cJ - 3rhyolk0lpjc9jf1iu32zwruc0oizis tt:10166723 X L ~' cJ - 3rhyolk0lpjc9jf1iu32zwruc0oizis tt:10166724 ` L ~( cJ - " 3rhyolk0lpjc9jf1iu32zwruc0oizis tt:10166725 h L ~) cJ - 4 3rhyolk0lpjc9jf1iu32zwruc0oizis tt:10166726 p L ~* cJ - F 3rhyolk0lpjc9jf1iu32zwruc0oizis tt:10166727 x L ~+ cJ - X 3rhyolk0lpjc9jf1iu32zwruc0oizis tt:10166728 L ~, cJ - j f9cz5y4js5hbj47ctzs2iy6s0jzt2pu tt:10166729 L ~- cJ - | f9cz5y4js5hbj47ctzs2iy6s0jzt2pu tt:10166730 L ~. cJ - jbuhg5xqnkibagnkg6tgivbh9g62z16 tt:10166731 L ~/ cJ - jbuhg5xqnkibagnkg6tgivbh9g62z16 tt:10166732 L ~0 cJ - jbuhg5xqnkibagnkg6tgivbh9g62z16 tt:10166733 L ~1 cJ - cbrkjjtjiflr6ckjcbasd9lyr1zu3k1 tt:10166734 L ~2 cJ - % cg027oprsw379o6009p9e08qrbfp01s tt:10166735 L ~3 cJ - jbuhg5xqnkibagnkg6tgivbh9g62z16 tt:10166736 L ~4 cJ - jbuhg5xqnkibagnkg6tgivbh9g62z16 tt:10166737 L ~5 cJ - erex2s1w3zutbsd4da8h1ngxqqy35j0 tt:10166738 L ~6 cJ - erex2s1w3zutbsd4da8h1ngxqqy35j0 tt:10166739 L ~7 cJ - 0 erex2s1w3zutbsd4da8h1ngxqqy35j0 tt:10166740 L ~8 cJ - B erex2s1w3zutbsd4da8h1ngxqqy35j0 tt:10166741 L ~9 cJ - T erex2s1w3zutbsd4da8h1ngxqqy35j0 tt:10166742 L ~: cJ - f 3vop4y2h2ala7m4ql4rnj93jv7jj0f3 tt:10166743 L ~; cJ - x 3vop4y2h2ala7m4ql4rnj93jv7jj0f3 tt:10166744 L ~< cJ - 3vop4y2h2ala7m4ql4rnj93jv7jj0f3 tt:10166745 L ~= cJ - 3vop4y2h2ala7m4ql4rnj93jv7jj0f3 tt:10166746 L ~> cJ - 6j1vpnk4xjvrq9bmlow0g09favgpnvc tt:10166747 L ~? cJ - 6j1vpnk4xjvrq9bmlow0g09favgpnvc tt:10166748 L ~@ cJ - 6j1vpnk4xjvrq9bmlow0g09favgpnvc tt:10166749 ( L ~A cJ - 6j1vpnk4xjvrq9bmlow0g09favgpnvc tt:10166750 0 L ~B cJ - 6j1vpnk4xjvrq9bmlow0g09favgpnvc tt:10166751 8 L ~C cJ - 6j1vpnk4xjvrq9bmlow0g09favgpnvc tt:10166752 @ L ~D cJ - 6j1vpnk4xjvrq9bmlow0g09favgpnvc tt:10166753 H L ~E cJ - , 6j1vpnk4xjvrq9bmlow0g09favgpnvc tt:10166754 P L ~F cJ - > k2f5ia9yii67xlddv6utalj8dprkv2u tt:10166755 X L ~G cJ - P k2f5ia9yii67xlddv6utalj8dprkv2u tt:10166756 ` L ~H cJ - b k2f5ia9yii67xlddv6utalj8dprkv2u tt:10166757 h L ~I cJ - t k2f5ia9yii67xlddv6utalj8dprkv2u tt:10166758 p L ~J cJ - k2f5ia9yii67xlddv6utalj8dprkv2u tt:10166759 x L ~K cJ - k2f5ia9yii67xlddv6utalj8dprkv2u tt:10166760 L ~L cJ - k2f5ia9yii67xlddv6utalj8dprkv2u tt:10166761 L ~M cJ - k2f5ia9yii67xlddv6utalj8dprkv2u tt:10166762 L ~N cJ - k2f5ia9yii67xlddv6utalj8dprkv2u tt:10166763 L ~O cJ - k2f5ia9yii67xlddv6utalj8dprkv2u tt:10166764 L ~P cJ - k2f5ia9yii67xlddv6utalj8dprkv2u tt:10166765 L ~Q cJ - k2f5ia9yii67xlddv6utalj8dprkv2u tt:10166766 L ~R cJ - c35ydr9myp90lwpdjax34b70uspxl3a tt:10166767 L ~S cJ - ( c35ydr9myp90lwpdjax34b70uspxl3a tt:10166768 L ~T cJ - : c35ydr9myp90lwpdjax34b70uspxl3a tt:10166769 L ~U cJ - L c35ydr9myp90lwpdjax34b70uspxl3a tt:10166770 L ~V cJ - ^ c35ydr9myp90lwpdjax34b70uspxl3a tt:10166771 L ~W cJ - p U nc78cd98pm89zjisagy77u5pu86xu1r tt:10166772 L ~X cJ - hm24skj7nhhsexi2qty45stkud98vwf tt:10166773 L ~Y cJ - hm24skj7nhhsexi2qty45stkud98vwf tt:10166774 L ~Z cJ - hm24skj7nhhsexi2qty45stkud98vwf tt:10166775 L ~[ cJ - hm24skj7nhhsexi2qty45stkud98vwf tt:10166776 L ~\ cJ - U nc78cd98pm89zjisagy77u5pu86xu1r tt:10166777 L ~] cJ - hm24skj7nhhsexi2qty45stkud98vwf tt:10166778 L ~^ cJ - hm24skj7nhhsexi2qty45stkud98vwf tt:10166779 L ~_ cJ - hm24skj7nhhsexi2qty45stkud98vwf tt:10166780 L ~` cJ - hm24skj7nhhsexi2qty45stkud98vwf tt:10166781 ( L ~a cJ - $ p4u5cvs729kcpoxuwsuunxgelq4du2f tt:10166782 0 L ~b cJ - 6 p4u5cvs729kcpoxuwsuunxgelq4du2f tt:10166783 8 L ~c cJ - H p4u5cvs729kcpoxuwsuunxgelq4du2f tt:10166784 @ L ~d cJ - Z p4u5cvs729kcpoxuwsuunxgelq4du2f tt:10166785 H L ~e cJ - l p4u5cvs729kcpoxuwsuunxgelq4du2f tt:10166786 P L ~f cJ - ~ p4u5cvs729kcpoxuwsuunxgelq4du2f tt:10166787 X L ~g cJ - ecta0mn51nzvawppgchra06kv6x4nk8 tt:10166788 ` L ~h cJ - ecta0mn51nzvawppgchra06kv6x4nk8 tt:10166789 h L ~i cJ - fexkjbyxouch9m0fa0akot6z6tq28zq tt:10166790 p L ~j cJ - fexkjbyxouch9m0fa0akot6z6tq28zq tt:10166791 x L ~k cJ - fexkjbyxouch9m0fa0akot6z6tq28zq tt:10166792 L ~l cJ - fexkjbyxouch9m0fa0akot6z6tq28zq tt:10166793 L ~m cJ - fexkjbyxouch9m0fa0akot6z6tq28zq tt:10166794 L ~n cJ - fexkjbyxouch9m0fa0akot6z6tq28zq tt:10166795 L ~o cJ - fexkjbyxouch9m0fa0akot6z6tq28zq tt:10166796 L ~p cJ - 2 fexkjbyxouch9m0fa0akot6z6tq28zq tt:10166797 L ~q cJ - D fexkjbyxouch9m0fa0akot6z6tq28zq tt:10166798 L ~r cJ - V llx5gd8i9j82eofrsgwk5z66vbz0cobv tt:10166799 L ~s cJ - h m2psxa8ee847bal2fcaylma5hwfw3pz tt:10166800 L ~t cJ - z m2psxa8ee847bal2fcaylma5hwfw3pz tt:10166801 L ~u cJ - m2psxa8ee847bal2fcaylma5hwfw3pz tt:10166802 L ~v cJ - m2psxa8ee847bal2fcaylma5hwfw3pz tt:10166803 L ~w cJ - m2psxa8ee847bal2fcaylma5hwfw3pz tt:10166804 L ~x cJ - m2psxa8ee847bal2fcaylma5hwfw3pz tt:10166805 L ~y cJ - m2psxa8ee847bal2fcaylma5hwfw3pz tt:10166806 L ~z cJ - m2psxa8ee847bal2fcaylma5hwfw3pz tt:10166807 L ~{ cJ - m2psxa8ee847bal2fcaylma5hwfw3pz tt:10166808 L ~| cJ - sg7c6btjvy7utt20814vzkk0lm5eanf tt:10166809 L ~} cJ - sg7c6btjvy7utt20814vzkk0lm5eanf tt:10166810 L ~~ cJ - . sg7c6btjvy7utt20814vzkk0lm5eanf tt:10166811 L ~ cJ - @ sg7c6btjvy7utt20814vzkk0lm5eanf tt:10166812 L ~ cJ - R sg7c6btjvy7utt20814vzkk0lm5eanf tt:10166813 ( L ~ cJ - d sg7c6btjvy7utt20814vzkk0lm5eanf tt:10166814 0 L ~ cJ - v sg7c6btjvy7utt20814vzkk0lm5eanf tt:10166815 8 L ~ cJ - sg7c6btjvy7utt20814vzkk0lm5eanf tt:10166816 @ L ~ cJ - sg7c6btjvy7utt20814vzkk0lm5eanf tt:10166817 H L ~ cJ - 93tzyi881hi1o3zdk6inybkbevg0puh tt:10166818 P L ~ cJ - a7pz8ytbgs0xp05jbete90ecvozyx00 tt:10166819 X L ~ cJ - a7pz8ytbgs0xp05jbete90ecvozyx00 tt:10166820 ` L ~ cJ - U lltdnll1k8mnxr678pvje017c0ss25i tt:10166821 h L ~ cJ - a7pz8ytbgs0xp05jbete90ecvozyx00 tt:10166822 p L ~ cJ - a7pz8ytbgs0xp05jbete90ecvozyx00 tt:10166823 x L ~ cJ - a7pz8ytbgs0xp05jbete90ecvozyx00 tt:10166825 L ~ cJ - * a7pz8ytbgs0xp05jbete90ecvozyx00 tt:10166826 L ~ cJ - < a7pz8ytbgs0xp05jbete90ecvozyx00 tt:10166827 L ~ cJ - N pyq402zovzfnquvcihhvcdut8l8h9ep tt:10166828 L ~ cJ - ` pyq402zovzfnquvcihhvcdut8l8h9ep tt:10166829 L ~ cJ - r pyq402zovzfnquvcihhvcdut8l8h9ep tt:10166830 L ~ cJ - pyq402zovzfnquvcihhvcdut8l8h9ep tt:10166831 L ~ cJ - pyq402zovzfnquvcihhvcdut8l8h9ep tt:10166832 L ~ cJ - pyq402zovzfnquvcihhvcdut8l8h9ep tt:10166833 L ~ cJ - pyq402zovzfnquvcihhvcdut8l8h9ep tt:10166834 L ~ cJ - pyq402zovzfnquvcihhvcdut8l8h9ep tt:10166835 L ~ cJ - pyq402zovzfnquvcihhvcdut8l8h9ep tt:10166836 L ~ cJ - 15o4y5a06wtpkwaczpoqfjy98obg1x1 tt:10166837 L ~ cJ - 15o4y5a06wtpkwaczpoqfjy98obg1x1 tt:10166838 L ~ cJ - 15o4y5a06wtpkwaczpoqfjy98obg1x1 tt:10166839 L ~ cJ - & 15o4y5a06wtpkwaczpoqfjy98obg1x1 tt:10166840 L ~ cJ - 8 15o4y5a06wtpkwaczpoqfjy98obg1x1 tt:10166841 L ~ cJ - J -d5naljbs5cnfzgh9rxjfo3sx389frno tt:10166842 L ~ cJ - \ 15o4y5a06wtpkwaczpoqfjy98obg1x1 tt:10166843 L ~ cJ - n 15o4y5a06wtpkwaczpoqfjy98obg1x1 tt:10166844 L ~ cJ - N mic9ovmext1qkwwp36wcpaaijv0ue8c tt:10166845 L ~ cJ - 15o4y5a06wtpkwaczpoqfjy98obg1x1 tt:10166846 ( L ~ cJ - kcgl3qrobgmtu7uxuxqjyrgrma1q998 tt:10166847 0 L ~ cJ - kcgl3qrobgmtu7uxuxqjyrgrma1q998 tt:10166848 8 L ~ cJ - kcgl3qrobgmtu7uxuxqjyrgrma1q998 tt:10166849 @ ) ~ cJ - "6zpk04gci2jyn5qucwkjpq76tpgr9kh tt:10166850 p938 6 5 4s3C2 0 / . -S,#* ) ( 'c&3% # " !s C S # c 3 s C
5Aug 25 19:42:36 db2118 mysqld[4586]: S # c cf W3b ;
6Aug 25 19:42:36 db2118 mysqld[4586]: InnoDB: End of page dump
7Aug 25 19:42:36 db2118 mysqld[4586]: 2021-08-25 19:42:36 0 [Note] InnoDB: Uncompressed page, stored checksum in field1 1478831489, calculated checksums for field1: crc32 1725978419, innodb 1478831489, page type 17855 == INDEX.none 3735928559, stored checksum in field2 1725978419, calculated checksums for field2: crc32 1725978419, innodb 1725978419, none 3735928559, page LSN 5360 1657972384, low 4 bytes of LSN at page end 1657972384, page number (if stored to page already) 49350, space id (if created with >= MySQL-4.1.1 and stored already) 4936
8Aug 25 19:42:36 db2118 mysqld[4586]: 2021-08-25 19:42:36 0 [Note] InnoDB: Page may be an index page where index id is 19838
9Aug 25 19:42:36 db2118 mysqld[4586]: 2021-08-25 19:42:36 0 [Note] InnoDB: Index 19838 is `PRIMARY` in table `metawiki`.`content`
10Aug 25 19:42:36 db2118 mysqld[4586]: 2021-08-25 19:42:36 0 [Note] InnoDB: It is also possible that your operating system has corrupted its own file cache and rebooting your computer removes the error. If the corrupt page is an index page. You can also try to fix the corruption by dumping, dropping, and reimporting the corrupt table. You can use CHECK TABLE to scan your table for corruption. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
11Aug 25 19:42:36 db2118 mysqld[4586]: 2021-08-25 19:42:36 0 [ERROR] InnoDB: Failed to read file './metawiki/content.ibd' at offset 49350: Page read from tablespace is corrupted.
12Aug 25 19:42:36 db2118 mysqld[4586]: 210825 19:42:36 [ERROR] mysqld got signal 11 ;
13Aug 25 19:42:36 db2118 mysqld[4586]: This could be because you hit a bug. It is also possible that this binary
14Aug 25 19:42:36 db2118 mysqld[4586]: or one of the libraries it was linked against is corrupt, improperly built,
15Aug 25 19:42:36 db2118 mysqld[4586]: or misconfigured. This error can also be caused by malfunctioning hardware.
16Aug 25 19:42:36 db2118 mysqld[4586]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
17Aug 25 19:42:36 db2118 mysqld[4586]: We will try our best to scrape up some info that will hopefully help
18Aug 25 19:42:36 db2118 mysqld[4586]: diagnose the problem, but since we have already crashed,
19Aug 25 19:42:36 db2118 mysqld[4586]: something is definitely wrong and this may fail.
20Aug 25 19:42:36 db2118 mysqld[4586]: Server version: 10.4.19-MariaDB-log
21Aug 25 19:42:36 db2118 mysqld[4586]: key_buffer_size=134217728
22Aug 25 19:42:36 db2118 mysqld[4586]: read_buffer_size=131072
23Aug 25 19:42:36 db2118 mysqld[4586]: max_used_connections=6
24Aug 25 19:42:36 db2118 mysqld[4586]: max_threads=2010
25Aug 25 19:42:36 db2118 mysqld[4586]: thread_count=8
26Aug 25 19:42:36 db2118 mysqld[4586]: It is possible that mysqld could use up to
27Aug 25 19:42:36 db2118 mysqld[4586]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 4752970 K bytes of memory
28Aug 25 19:42:36 db2118 mysqld[4586]: Hope that's ok; if not, decrease some variables in the equation.
29Aug 25 19:42:36 db2118 mysqld[4586]: Thread pointer: 0x7f34d4000ce8
30Aug 25 19:42:36 db2118 mysqld[4586]: Attempting backtrace. You can use the following information to find out
31Aug 25 19:42:36 db2118 mysqld[4586]: where mysqld died. If you see no messages after this, something went
32Aug 25 19:42:36 db2118 mysqld[4586]: terribly wrong...
33Aug 25 19:42:36 db2118 mysqld[4586]: stack_bottom = 0x7f95341a2cb8 thread_stack 0x30000
34Aug 25 19:42:39 db2118 mysqld[4586]: /opt/wmf-mariadb104/bin/mysqld(my_print_stacktrace+0x2e)[0x5565e4485d9e]
35Aug 25 19:42:39 db2118 mysqld[4586]: /opt/wmf-mariadb104/bin/mysqld(handle_fatal_signal+0x54d)[0x5565e3f75e4d]
36Aug 25 19:42:40 db2118 mysqld[4586]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x7f95904bc730]
37Aug 25 19:42:43 db2118 mysqld[4586]: /opt/wmf-mariadb104/bin/mysqld(+0xb953eb)[0x5565e42493eb]
38Aug 25 19:42:43 db2118 mysqld[4586]: /opt/wmf-mariadb104/bin/mysqld(+0xa6e038)[0x5565e4122038]
39Aug 25 19:42:43 db2118 mysqld[4586]: /opt/wmf-mariadb104/bin/mysqld(_ZN7handler8ha_checkEP3THDP15st_ha_check_opt+0x5d)[0x5565e3f7ed0d]
40Aug 25 19:42:43 db2118 mysqld[4586]: /opt/wmf-mariadb104/bin/mysqld(+0x7ad89a)[0x5565e3e6189a]
41Aug 25 19:42:43 db2118 mysqld[4586]: /opt/wmf-mariadb104/bin/mysqld(_ZN19Sql_cmd_check_table7executeEP3THD+0x82)[0x5565e3e62a22]
42Aug 25 19:42:43 db2118 mysqld[4586]: /opt/wmf-mariadb104/bin/mysqld(_Z21mysql_execute_commandP3THD+0x1209)[0x5565e3d670e9]
43Aug 25 19:42:43 db2118 mysqld[4586]: /opt/wmf-mariadb104/bin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_statebb+0x1e9)[0x5565e3d6d9e9]
44Aug 25 19:42:43 db2118 mysqld[4586]: /opt/wmf-mariadb104/bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0x17f5)[0x5565e3d700b5]
45Aug 25 19:42:43 db2118 mysqld[4586]: /opt/wmf-mariadb104/bin/mysqld(_Z10do_commandP3THD+0x13a)[0x5565e3d7129a]
46Aug 25 19:42:43 db2118 mysqld[4586]: /opt/wmf-mariadb104/bin/mysqld(_Z11tp_callbackP13TP_connection+0x88)[0x5565e3f48278]
47Aug 25 19:42:43 db2118 mysqld[4586]: /opt/wmf-mariadb104/bin/mysqld(+0xa446f0)[0x5565e40f86f0]
48Aug 25 19:42:43 db2118 mysqld[4586]: /opt/wmf-mariadb104/bin/mysqld(+0xd8289b)[0x5565e443689b]
49Aug 25 19:42:45 db2118 mysqld[4586]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x7fa3)[0x7f95904b1fa3]
50Aug 25 19:42:47 db2118 mysqld[4586]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7f95900d64cf]
51Aug 25 19:42:47 db2118 mysqld[4586]: Trying to get some variables.
52Aug 25 19:42:47 db2118 mysqld[4586]: Some pointers may be invalid and cause the dump to abort.
53Aug 25 19:42:47 db2118 mysqld[4586]: Query (0x7f34d403cef0): CHECK TABLE `content`
54Aug 25 19:42:47 db2118 mysqld[4586]: Connection ID (thread ID): 40
55Aug 25 19:42:47 db2118 mysqld[4586]: Status: NOT_KILLED
56Aug 25 19:42:47 db2118 mysqld[4586]: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=on,mrr_cost_based=on,mrr_sort_keys=on,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=off,condition_pushdown_from_having=on
57Aug 25 19:42:47 db2118 mysqld[4586]: The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
58Aug 25 19:42:47 db2118 mysqld[4586]: information that should help you find out what is causing the crash.
59Aug 25 19:42:47 db2118 mysqld[4586]: Writing a core file...
60Aug 25 19:42:47 db2118 mysqld[4586]: Working directory at /srv/sqldata
61Aug 25 19:42:47 db2118 mysqld[4586]: Resource Limits:
62Aug 25 19:42:47 db2118 mysqld[4586]: Limit Soft Limit Hard Limit Units
63Aug 25 19:42:47 db2118 mysqld[4586]: Max cpu time unlimited unlimited seconds
64Aug 25 19:42:47 db2118 mysqld[4586]: Max file size unlimited unlimited bytes
65Aug 25 19:42:47 db2118 mysqld[4586]: Max data size unlimited unlimited bytes
66Aug 25 19:42:47 db2118 mysqld[4586]: Max stack size 8388608 unlimited bytes
67Aug 25 19:42:47 db2118 mysqld[4586]: Max core file size 0 0 bytes
68Aug 25 19:42:47 db2118 mysqld[4586]: Max resident set unlimited unlimited bytes
69Aug 25 19:42:47 db2118 mysqld[4586]: Max processes 2058360 2058360 processes
70Aug 25 19:42:47 db2118 mysqld[4586]: Max open files 200001 200001 files
71Aug 25 19:42:47 db2118 mysqld[4586]: Max locked memory 65536 65536 bytes
72Aug 25 19:42:47 db2118 mysqld[4586]: Max address space unlimited unlimited bytes
73Aug 25 19:42:47 db2118 mysqld[4586]: Max file locks unlimited unlimited locks
74Aug 25 19:42:47 db2118 mysqld[4586]: Max pending signals 2058360 2058360 signals
75Aug 25 19:42:47 db2118 mysqld[4586]: Max msgqueue size 819200 819200 bytes
76Aug 25 19:42:47 db2118 mysqld[4586]: Max nice priority 0 0
77Aug 25 19:42:47 db2118 mysqld[4586]: Max realtime priority 0 0
78Aug 25 19:42:47 db2118 mysqld[4586]: Max realtime timeout unlimited unlimited us
79Aug 25 19:42:47 db2118 mysqld[4586]: Core pattern: /var/tmp/core/core.%h.%e.%p.%t
80Aug 25 19:42:52 db2118 systemd[1]: mariadb.service: Main process exited, code=killed, status=11/SEGV
81-- Subject: Unit process exited
82-- Defined-By: systemd
83-- Support: https://www.debian.org/support
84--
85-- An ExecStart= process belonging to unit mariadb.service has exited.
86--
87-- The process' exit code is 'killed' and its exit status is 11.

Going to reclone it.

Mentioned in SAL (#wikimedia-operations) [2021-09-06T14:53:41Z] <kormat@cumin1001> dbctl commit (dc=all): 'db2118 (re)pooling @ 25%: reimage to buster T288244', diff saved to https://phabricator.wikimedia.org/P17226 and previous config saved to /var/cache/conftool/dbconfig/20210906-145341-kormat.json

@Kormat is "Upgrade the old master and make it a candidate master." step also fully complete (as db2118 has been repooled)?

Mentioned in SAL (#wikimedia-operations) [2021-09-07T07:07:03Z] <kormat@cumin1001> dbctl commit (dc=all): 'Fixing db2118's pooling config T288244', diff saved to https://phabricator.wikimedia.org/P17230 and previous config saved to /var/cache/conftool/dbconfig/20210907-070702-kormat.json

Mentioned in SAL (#wikimedia-operations) [2021-09-07T07:07:24Z] <kormat@cumin1001> dbctl commit (dc=all): 'db2118 (re)pooling @ 25%: reimage to buster (now with fixed pool config) T288244', diff saved to https://phabricator.wikimedia.org/P17231 and previous config saved to /var/cache/conftool/dbconfig/20210907-070724-kormat.json

@Kormat is "Upgrade the old master and make it a candidate master." step also fully complete (as db2118 has been repooled)?

I was about to say 'yes', and then i double-checked the dbctl config for db2118, and noticed it was all wrong. So i've fixed that now, and am repooling it again. Once that's done, i'll mark off that step as complete.

Mentioned in SAL (#wikimedia-operations) [2021-09-07T07:22:28Z] <kormat@cumin1001> dbctl commit (dc=all): 'db2118 (re)pooling @ 50%: reimage to buster (now with fixed pool config) T288244', diff saved to https://phabricator.wikimedia.org/P17233 and previous config saved to /var/cache/conftool/dbconfig/20210907-072227-kormat.json

Mentioned in SAL (#wikimedia-operations) [2021-09-07T07:37:31Z] <kormat@cumin1001> dbctl commit (dc=all): 'db2118 (re)pooling @ 75%: reimage to buster (now with fixed pool config) T288244', diff saved to https://phabricator.wikimedia.org/P17236 and previous config saved to /var/cache/conftool/dbconfig/20210907-073731-kormat.json

Mentioned in SAL (#wikimedia-operations) [2021-09-07T07:52:35Z] <kormat@cumin1001> dbctl commit (dc=all): 'db2118 (re)pooling @ 100%: reimage to buster (now with fixed pool config) T288244', diff saved to https://phabricator.wikimedia.org/P17239 and previous config saved to /var/cache/conftool/dbconfig/20210907-075235-kormat.json

Kormat updated the task description. (Show Details)
Kormat moved this task from In progress to Done on the DBA board.

All done from the DBA side. Assigning to @jcrespo for the remaining backup item, please resolve the task when that's done.

Will do at the same time as T288803, next week.

I am guessing I can remove kormat-dump.s7.2021-08-24--00-00-02, but want to double confirm @Kormat, as every time I am about to do some potentially destructive actions (I am guessing it will be a no longer needed copy of that dump).

I am guessing I can remove kormat-dump.s7.2021-08-24--00-00-02, but want to double confirm @Kormat, as every time I am about to do some potentially destructive actions (I am guessing it will be a no longer needed copy of that dump).

Oops, thanks for catching that! I've cleaned it up, as well as the temporary copy of /srv/sqldata that i'd made on db2118. Thanks for the reminder :)

Change 720766 had a related patch set uploaded (by Jcrespo; author: Jcrespo):

[operations/puppet@production] dbbackups: Remove s4 and s7 stretch backup source instances

https://gerrit.wikimedia.org/r/720766

Change 720767 had a related patch set uploaded (by Jcrespo; author: Jcrespo):

[operations/puppet@production] dbbackups: Reenable notifications on stretch hosts after maintenance

https://gerrit.wikimedia.org/r/720767

Change 720766 merged by Jcrespo:

[operations/puppet@production] dbbackups: Remove s4 and s7 stretch backup source instances

https://gerrit.wikimedia.org/r/720766

I've cleaned up puppet, icinga and config files of:

  • db1116
  • db1139
  • db2097
  • db2100

However, it is getting late, and I still have to cleanup:

  • datadir
  • tendril
  • zarcillo
  • orchestrator
  • (final checks)

And I don't trust myself so late in the day- sorry if it creates some noise there (e.g. prometheus scrapping errors/tendril red status/orchestrator alerts) until I finish all tasks.

Change 720767 merged by Jcrespo:

[operations/puppet@production] dbbackups: Reenable notifications on stretch hosts after maintenance

https://gerrit.wikimedia.org/r/720767

jcrespo reassigned this task from jcrespo to Kormat.

Based on prometheus and tendril, I think this is done, but please do not doubt to ping me if there is any unintended leftover.