zfilipin@tin:/srv/mediawiki-staging$ scap sync-file wmf-config/Wikibase-production.php 'SWAT: [[gerrit:416748|Enable caching of constraint check results (T184812)]]' ___ ____ ⎛ ⎛ ,---- \ //==--' _//|,.·//==--' ____________________________ _OO≣=- ︶ ᴹw ⎞_§ ______ ___\ ___\ ,\__ \/ __ \ (∞)_, ) ( | ______/__ \/ /__ / /_/ / /_/ / ¨--¨|| |- ( / ______\____/ \___/ \__^_/ .__/ ««_/ «_/ jgs/bd808 /_/ 13:28:25 Copying to tin.eqiad.wmnet from tin.eqiad.wmnet 13:28:25 Started rsync common 13:28:25 Finished rsync common (duration: 00m 00s) 13:28:26 Started cache_git_info 13:28:42 Finished cache_git_info (duration: 00m 16s) 13:28:42 Started sync-masters sync-masters: 100% (ok: 1; fail: 0; left: 0) 13:28:50 Finished sync-masters (duration: 00m 07s) 13:28:50 Started sync-pull-masters sync-pull-masters: 100% (ok: 1; fail: 0; left: 0) 13:28:51 Finished sync-pull-masters (duration: 00m 01s) 13:28:51 Started sync-check-canaries check-canaries: 100% (ok: 11; fail: 0; left: 0) 13:28:54 Finished sync-check-canaries (duration: 00m 02s) 13:28:54 Waiting for canary traffic... 13:29:14 Executing check 'Logstash Error rate for mw1279.eqiad.wmnet' 13:29:14 Executing check 'Logstash Error rate for mw1276.eqiad.wmnet' 13:29:14 Executing check 'Logstash Error rate for mw1261.eqiad.wmnet' 13:29:14 Executing check 'Logstash Error rate for mw1264.eqiad.wmnet' 13:29:14 Executing check 'Logstash Error rate for mwdebug1002.eqiad.wmnet' 13:29:14 Executing check 'Logstash Error rate for mwdebug1001.eqiad.wmnet' 13:29:14 Executing check 'Logstash Error rate for mw1263.eqiad.wmnet' 13:29:14 Executing check 'Logstash Error rate for mw1262.eqiad.wmnet' 13:29:14 Executing check 'Logstash Error rate for mw1278.eqiad.wmnet' 13:29:14 Executing check 'Logstash Error rate for mw1277.eqiad.wmnet' 13:29:14 Executing check 'Logstash Error rate for mw1265.eqiad.wmnet' 13:29:15 Started sync-proxies sync-proxies: 100% (ok: 7; fail: 0; left: 0) 13:29:18 Finished sync-proxies (duration: 00m 03s) 13:29:18 Started sync-apaches 13:31:30 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2123.codfw.wmnet returned [255]: ssh: connect to host mw2123.codfw.wmnet port 22: Connection timed out 13:31:30 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2130.codfw.wmnet returned [255]: ssh: connect to host mw2130.codfw.wmnet port 22: Connection timed out 13:31:30 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2107.codfw.wmnet returned [255]: ssh: connect to host mw2107.codfw.wmnet port 22: Connection timed out 13:31:30 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2125.codfw.wmnet returned [255]: ssh: connect to host mw2125.codfw.wmnet port 22: Connection timed out 13:31:30 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2117.codfw.wmnet returned [255]: ssh: connect to host mw2117.codfw.wmnet port 22: Connection timed out 13:31:30 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2127.codfw.wmnet returned [255]: ssh: connect to host mw2127.codfw.wmnet port 22: Connection timed out 13:31:30 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2133.codfw.wmnet returned [255]: ssh: connect to host mw2133.codfw.wmnet port 22: Connection timed out 13:31:30 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2122.codfw.wmnet returned [255]: ssh: connect to host mw2122.codfw.wmnet port 22: Connection timed out 13:31:30 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2116.codfw.wmnet returned [255]: ssh: connect to host mw2116.codfw.wmnet port 22: Connection timed out 13:31:30 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2102.codfw.wmnet returned [255]: ssh: connect to host mw2102.codfw.wmnet port 22: Connection timed out 13:31:30 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2124.codfw.wmnet returned [255]: ssh: connect to host mw2124.codfw.wmnet port 22: Connection timed out 13:31:30 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2128.codfw.wmnet returned [255]: ssh: connect to host mw2128.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2111.codfw.wmnet returned [255]: ssh: connect to host mw2111.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2115.codfw.wmnet returned [255]: ssh: connect to host mw2115.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2126.codfw.wmnet returned [255]: ssh: connect to host mw2126.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2105.codfw.wmnet returned [255]: ssh: connect to host mw2105.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2110.codfw.wmnet returned [255]: ssh: connect to host mw2110.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2101.codfw.wmnet returned [255]: ssh: connect to host mw2101.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2100.codfw.wmnet returned [255]: ssh: connect to host mw2100.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2106.codfw.wmnet returned [255]: ssh: connect to host mw2106.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2134.codfw.wmnet returned [255]: ssh: connect to host mw2134.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2109.codfw.wmnet returned [255]: ssh: connect to host mw2109.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2121.codfw.wmnet returned [255]: ssh: connect to host mw2121.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2103.codfw.wmnet returned [255]: ssh: connect to host mw2103.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2118.codfw.wmnet returned [255]: ssh: connect to host mw2118.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2132.codfw.wmnet returned [255]: ssh: connect to host mw2132.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2097.codfw.wmnet returned [255]: ssh: connect to host mw2097.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2104.codfw.wmnet returned [255]: ssh: connect to host mw2104.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2120.codfw.wmnet returned [255]: ssh: connect to host mw2120.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2119.codfw.wmnet returned [255]: ssh: connect to host mw2119.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2113.codfw.wmnet returned [255]: ssh: connect to host mw2113.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2131.codfw.wmnet returned [255]: ssh: connect to host mw2131.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2129.codfw.wmnet returned [255]: ssh: connect to host mw2129.codfw.wmnet port 22: Connection timed out 13:31:32 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2112.codfw.wmnet returned [255]: ssh: connect to host mw2112.codfw.wmnet port 22: Connection timed out 13:31:34 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2108.codfw.wmnet returned [255]: ssh: connect to host mw2108.codfw.wmnet port 22: Connection timed out 13:31:34 ['/usr/bin/scap', 'pull', '--no-update-l10n', '--include', 'wmf-config', '--include', 'wmf-config/Wikibase-production.php', 'mw1284.eqiad.wmnet', 'mw1319.eqiad.wmnet', 'mw1280.eqiad.wmnet', 'mw2215.codfw.wmnet', 'mw2187.codfw.wmnet', 'mw1250.eqiad.wmnet', 'mw1313.eqiad.wmnet'] on mw2114.codfw.wmnet returned [255]: ssh: connect to host mw2114.codfw.wmnet port 22: Connection timed out sync-apaches: 100% (ok: 238; fail: 36; left: 0) 13:31:34 36 apaches had sync errors 13:31:34 Finished sync-apaches (duration: 02m 15s) 13:31:34 Synchronized wmf-config/Wikibase-production.php: SWAT: [[gerrit:416748|Enable caching of constraint check results (T184812)]] (duration: 03m 08s)