Page MenuHomePhabricator
Paste P10496

snapshotting duration
ActivePublic

Authored by jcrespo on Feb 24 2020, 11:51 AM.
Tags
None
Referenced Files
F31628972: raw.txt
Feb 24 2020, 12:07 PM
F31628965: raw.txt
Feb 24 2020, 11:57 AM
F31628955: raw.txt
Feb 24 2020, 11:51 AM
Subscribers
None
eqiad:
+----------------------------------+-------------------------+------------------+----------------+--------------+
| name | source | transfer_seconds | pepare_seconds | total_size |
+----------------------------------+-------------------------+------------------+----------------+--------------+
| snapshot.s3.2020-02-24--07-17-16 | db1140.eqiad.wmnet:3313 | 4662 | 5645 | 866897202277 |
| snapshot.s3.2020-02-21--11-17-37 | db1140.eqiad.wmnet:3313 | 4512 | 5200 | 863291509861 |
| snapshot.s3.2020-02-21--07-13-50 | db1140.eqiad.wmnet:3313 | 4583 | 5545 | 862507707493 |
| snapshot.s3.2020-02-12--05-46-09 | db1095.eqiad.wmnet:3313 | 4948 | 8439 | 932180355139 |
| snapshot.s3.2020-02-10--06-55-30 | db1095.eqiad.wmnet:3313 | 4940 | 5566 | 931939765548 |
| snapshot.s3.2020-02-07--06-03-50 | db1095.eqiad.wmnet:3313 | 4989 | 7514 | 932452502828 |
| snapshot.s3.2020-02-05--06-25-50 | db1095.eqiad.wmnet:3313 | 4902 | 6081 | 930891681068 |
| snapshot.s3.2020-02-03--09-43-15 | db1095.eqiad.wmnet:3313 | 4972 | 6061 | 929519947052 |
| snapshot.s3.2020-01-31--06-53-24 | db1095.eqiad.wmnet:3313 | 4919 | 6509 | 928222129452 |
| snapshot.s3.2020-01-29--07-14-58 | db1095.eqiad.wmnet:3313 | 4934 | 6146 | 927669726508 |
| snapshot.s3.2020-01-27--06-48-23 | db1095.eqiad.wmnet:3313 | 4976 | 6386 | 926048136492 |
| snapshot.s3.2020-01-24--06-02-52 | db1095.eqiad.wmnet:3313 | 4876 | 6939 | 925539503356 |
| snapshot.s3.2020-01-22--07-14-18 | db1095.eqiad.wmnet:3313 | 4971 | 5777 | 924890262828 |
| snapshot.s3.2020-01-20--06-51-52 | db1095.eqiad.wmnet:3313 | 4938 | 5876 | 924041399596 |
| snapshot.s3.2020-01-17--06-56-07 | db1095.eqiad.wmnet:3313 | 4941 | 6231 | 922379472172 |
| snapshot.s3.2020-01-15--05-36-55 | db1095.eqiad.wmnet:3313 | 4864 | 7927 | 921034386732 |
| snapshot.s3.2020-01-13--02-44-59 | db1095.eqiad.wmnet:3313 | 4988 | 8664 | 920657964236 |
| snapshot.s3.2020-01-10--07-26-10 | db1095.eqiad.wmnet:3313 | 4886 | 5722 | 919563291852 |
| snapshot.s3.2020-01-08--07-09-54 | db1095.eqiad.wmnet:3313 | 4845 | 5778 | 919463000064 |
| snapshot.s3.2020-01-06--04-50-55 | db1095.eqiad.wmnet:3313 | 4803 | 5638 | 917186156544 |
| snapshot.s3.2020-01-03--07-09-05 | db1095.eqiad.wmnet:3313 | 4887 | 6348 | 916276008960 |
| snapshot.s3.2020-01-01--02-32-23 | db1095.eqiad.wmnet:3313 | 4870 | 8289 | 915357874080 |
codfw:
+----------------------------------+-------------------------+------------------+----------------+--------------+
| name | source | transfer_seconds | pepare_seconds | total_size |
+----------------------------------+-------------------------+------------------+----------------+--------------+
| snapshot.s3.2020-02-21--07-32-41 | db2098.codfw.wmnet:3313 | 4579 | 12104 | 880724022868 |
| snapshot.s3.2020-02-19--07-50-36 | db2098.codfw.wmnet:3313 | 32974 | 10542 | 880089921147 |
| snapshot.s3.2020-02-19--12-44-04 | db1140.codfw.wmnet:3313 | 11993 | 4097 | 864096379284 |
| snapshot.s3.2020-02-17--07-22-25 | db2098.codfw.wmnet:3313 | 4616 | 12380 | 878445171728 |
| snapshot.s3.2020-02-14--07-33-50 | db2098.codfw.wmnet:3313 | 4615 | 12468 | 877447189496 |
| snapshot.s3.2020-02-12--07-21-30 | db2098.codfw.wmnet:3313 | 4528 | 12168 | 876868269072 |
| snapshot.s3.2020-02-10--07-20-03 | db2098.codfw.wmnet:3313 | 4540 | 11692 | 875751088338 |
| snapshot.s3.2020-02-07--07-15-51 | db2098.codfw.wmnet:3313 | 4544 | 12048 | 874997227730 |
| snapshot.s3.2020-02-05--07-39-03 | db2098.codfw.wmnet:3313 | 4569 | 12023 | 874421649618 |
| snapshot.s3.2020-02-03--07-00-57 | db2098.codfw.wmnet:3313 | 4494 | 11980 | 872414585042 |
| snapshot.s3.2020-01-31--07-31-35 | db2098.codfw.wmnet:3313 | 4499 | 11903 | 871085326546 |
| snapshot.s3.2020-01-29--02-56-54 | db2098.codfw.wmnet:3313 | 4545 | 13712 | 870379798738 |
| snapshot.s3.2020-01-27--04-10-14 | db2098.codfw.wmnet:3313 | 4576 | 11981 | 869179351250 |
| snapshot.s3.2020-01-24--02-55-25 | db2098.codfw.wmnet:3313 | 4535 | 13502 | 867932176594 |
| snapshot.s3.2020-01-22--05-55-20 | db2098.codfw.wmnet:3313 | 4492 | 11613 | 868121772242 |
| snapshot.s3.2020-01-21--10-02-45 | db2098.codfw.wmnet:3313 | 30676 | 6198 | NULL |
| snapshot.s3.2020-01-17--07-38-24 | db2098.codfw.wmnet:3313 | 4478 | 11948 | 864645759186 |
| snapshot.s3.2020-01-13--07-10-30 | db2098.codfw.wmnet:3313 | 4449 | 11619 | 862619156570 |
| snapshot.s3.2020-01-10--07-00-30 | db2098.codfw.wmnet:3313 | 4439 | 11593 | 861088940146 |
| snapshot.s3.2020-01-08--06-51-06 | db2098.codfw.wmnet:3313 | 4488 | 11750 | 860907146150 |
| snapshot.s3.2020-01-06--07-40-45 | db2098.codfw.wmnet:3313 | 4434 | 11861 | 858850282406 |
| snapshot.s3.2020-01-03--07-37-50 | db2098.codfw.wmnet:3313 | 4422 | 11852 | 857335114662 |
Averages:
mysql[zarcillo]> SELECT section, source, avg(TIMESTAMPDIFF(SECOND, TIMESTAMP(INSERT(INSERT(REPLACE(SUBSTRING_INDEX(name, '.', -1), '--', ' '), 14, 1, ':'), 17, 1, ':')), start_date)) as transfer_seconds, timestampdiff(SECOND, start_date, end_date) as pepare_seconds, avg(total_size) as total_size FROM backups WHERE type='snapshot' and status='finished' and start_date > NOW() - INTERVAL 3 month GROUP BY section, source;
+---------+-------------------------+------------------+----------------+--------------------+
| section | source | transfer_seconds | pepare_seconds | total_size |
+---------+-------------------------+------------------+----------------+--------------------+
| s3 | db1095.eqiad.wmnet:3313 | 4890.1389 | 4372 | 919109422660.9722 |
| s3 | db1140.eqiad.wmnet:3313 | 4585.6667 | 5545 | 864232139877.0000 |
| s3 | db2098.codfw.wmnet:3313 | 5922.0263 | 12169 | 862377998146.6757 |