Page MenuHomePhabricator
Paste P17197

(An Untitled Masterwork)
ActivePublic

Authored by Marostegui on Fri, Sep 3, 5:34 AM.
[05:20:26] marostegui@cumin1001:~$ sudo cookbook sre.hosts.decommission pc2007.codfw.wmnet -t T289112
>>> ATTENTION: destructive action for 1 hosts: pc2007.codfw.wmnet
Are you sure to proceed?
Type "go" to proceed or "abort" to interrupt the execution
> go
START - Cookbook sre.hosts.decommission for hosts pc2007.codfw.wmnet
Looking for matches in puppetmaster1001.eqiad.wmnet:/var/lib/git/operations/puppet
----- OUTPUT of 'git -C '/var/lib...04[^0-9A-Za-z])'' -----
manifests/site.pp:node 'pc2007.codfw.wmnet' {
modules/install_server/files/dhcpd/linux-host-entries.ttyS1-115200: fixed-address pc2007.codfw.wmnet;
================
PASS |███████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:00<00:00, 1.12hosts/s]
FAIL | | 0% (0/1) [00:00<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'git -C '/var/lib...04[^0-9A-Za-z])''.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
Looking for matches in puppetmaster1001.eqiad.wmnet:/srv/private
----- OUTPUT of 'git -C '/srv/pri...04[^0-9A-Za-z])'' -----
================
PASS |███████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:00<00:00, 1.83hosts/s]
FAIL | | 0% (0/1) [00:00<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'git -C '/srv/pri...04[^0-9A-Za-z])''.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
Looking for matches in deploy1002.eqiad.wmnet:/srv/mediawiki-staging
----- OUTPUT of 'git -C '/srv/med...04[^0-9A-Za-z])'' -----
================
PASS |███████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:01<00:00, 1.10s/hosts]
FAIL | | 0% (0/1) [00:01<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'git -C '/srv/med...04[^0-9A-Za-z])''.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
Looking for matches in deploy1002.eqiad.wmnet:/srv/deployment-charts
----- OUTPUT of 'git -C '/srv/dep...04[^0-9A-Za-z])'' -----
================
PASS |███████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:00<00:00, 1.78hosts/s]
FAIL | | 0% (0/1) [00:00<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'git -C '/srv/dep...04[^0-9A-Za-z])''.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
>>> Found match(es) in the Puppet or mediawiki-config repositories (see above), proceed anyway?
Type "go" to proceed or "abort" to interrupt the execution
> go
Looking for Kerberos credentials on KDC kadmin node.
----- OUTPUT of 'find /srv/kerber...07.codfw.wmnet*"' -----
================
PASS |███████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:00<00:00, 1.73hosts/s]
FAIL | | 0% (0/1) [00:00<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'find /srv/kerber...07.codfw.wmnet*"'.
----- OUTPUT of '/usr/local/sbin/...07.codfw.wmnet*"' -----
================
PASS |███████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:00<00:00, 1.92hosts/s]
FAIL | | 0% (0/1) [00:00<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: '/usr/local/sbin/...07.codfw.wmnet*"'.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
No Kerberos credentials found.
----- OUTPUT of 'grep -P '\s*comm...cinga/icinga.cfg' -----
command_file=/var/lib/icinga/rw/icinga.cmd
================
PASS |███████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:00<00:00, 2.68hosts/s]
FAIL | | 0% (0/1) [00:00<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'grep -P '\s*comm...cinga/icinga.cfg'.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
----- OUTPUT of '/usr/local/bin/i...atus -j "pc2007"' -----
{"pc2007": {"name": "pc2007", "state": "UP", "optimal": true, "downtimed": true, "notifications_enabled": true, "failed_services": []}}
================
PASS |███████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:00<00:00, 1.24hosts/s]
FAIL | | 0% (0/1) [00:00<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: '/usr/local/bin/i...atus -j "pc2007"'.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
Scheduling downtime on Icinga server alert1001.wikimedia.org for hosts: pc2007
----- OUTPUT of 'bash -c 'echo -n.../rw/icinga.cmd '' -----
================
PASS |███████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:00<00:00, 3.02hosts/s]
FAIL | | 0% (0/1) [00:00<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'bash -c 'echo -n.../rw/icinga.cmd ''.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
----- OUTPUT of 'bash -c 'echo -n.../rw/icinga.cmd '' -----
================
PASS |███████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:00<00:00, 2.68hosts/s]
FAIL | | 0% (0/1) [00:00<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'bash -c 'echo -n.../rw/icinga.cmd ''.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
Downtimed host on Icinga
Management Password:
Found physical host
----- OUTPUT of '/usr/local/bin/i...atus -j "pc2007"' -----
{"pc2007": {"name": "pc2007", "state": "UP", "optimal": true, "downtimed": true, "notifications_enabled": true, "failed_services": []}}
================
PASS |███████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:01<00:00, 1.07s/hosts]
FAIL | | 0% (0/1) [00:01<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: '/usr/local/bin/i...atus -j "pc2007"'.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
Scheduling downtime on Icinga server alert1001.wikimedia.org for hosts: pc2007
----- OUTPUT of 'bash -c 'echo -n.../rw/icinga.cmd '' -----
================
PASS |███████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:00<00:00, 2.35hosts/s]
FAIL | | 0% (0/1) [00:00<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'bash -c 'echo -n.../rw/icinga.cmd ''.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
----- OUTPUT of 'bash -c 'echo -n.../rw/icinga.cmd '' -----
================
PASS |███████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:00<00:00, 2.79hosts/s]
FAIL | | 0% (0/1) [00:00<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'bash -c 'echo -n.../rw/icinga.cmd ''.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
Downtimed management interface on Icinga
----- OUTPUT of 'true' -----
================
PASS |███████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:00<00:00, 1.10hosts/s]
FAIL | | 0% (0/1) [00:00<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'true'.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
----- OUTPUT of 'lsblk --all --ou...-all --force %*'' -----
/dev/sda: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 41 52 54
/dev/sda: 8 bytes were erased at offset 0x45d679ffe00 (gpt): 45 46 49 20 50 41 52 54
/dev/sda: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/sda1: 2 bytes were erased at offset 0x00000438 (ext4): 53 ef
/dev/sda2: 10 bytes were erased at offset 0x00000ff6 (swap): 53 57 41 50 53 50 41 43 45 32
/dev/sda3: 8 bytes were erased at offset 0x00000218 (LVM2_member): 4c 56 4d 32 20 30 30 31
================
PASS |███████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:00<00:00, 1.20hosts/s]
FAIL | | 0% (0/1) [00:00<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'lsblk --all --ou...-all --force %*''.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
Wiped all swraid, partition-table and filesystem signatures
Running IPMI command: ipmitool -I lanplus -H pc2007.mgmt.codfw.wmnet -U root -E chassis power off
Powered off
Disable and reset vlan on asw-a1-codfw:ge-1/0/1 for local eno1
Delete IP 10.192.0.104/22 on eno1
Delete IP 2620:0:860:101:10:192:0:104/64 on eno1
Set Netbox status to Decommissioning and deleted all non-mgmt interfaces and related IPs
Sleeping for 20s to avoid race conditions...
Removed host pc2007.codfw.wmnet from Debmonitor
Removed from DebMonitor
----- OUTPUT of 'puppet node clea...2007.codfw.wmnet' -----
Notice: Revoked certificate with serial 6397
Notice: Removing file Puppet::SSL::Certificate pc2007.codfw.wmnet at '/var/lib/puppet/server/ssl/ca/signed/pc2007.codfw.wmnet.pem'
pc2007.codfw.wmnet
================
PASS |███████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:01<00:00, 1.85s/hosts]
FAIL | | 0% (0/1) [00:01<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'puppet node clea...2007.codfw.wmnet'.
----- OUTPUT of 'puppet node deac...2007.codfw.wmnet' -----
Submitted 'deactivate node' for pc2007.codfw.wmnet with UUID 49134376-d2f7-4a7f-987e-fa1039787304
================
PASS |███████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:01<00:00, 1.80s/hosts]
FAIL | | 0% (0/1) [00:01<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'puppet node deac...2007.codfw.wmnet'.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
Removed from Puppet master and PuppetDB
Sleeping for 3 minutes to get Netbox caches in sync
Generating the DNS records from Netbox data. It will take a couple of minutes.
----- OUTPUT of 'cd /tmp && runus...e asset tag one"' -----
2021-09-03 05:24:29,276 [INFO] Gathering devices, interfaces, addresses and prefixes from Netbox
2021-09-03 05:27:52,361 [WARNING] Device frqueue1002 of IP 10.64.40.204/26 not in devices, skipping.
2021-09-03 05:27:52,365 [WARNING] Device phab1003 of IP 10.65.1.16/16 not in devices, skipping.
2021-09-03 05:27:52,405 [INFO] Gathered 2328 devices from Netbox
2021-09-03 05:27:52,405 [INFO] Generating DNS records
2021-09-03 05:28:02,216 [INFO] Generated 13396 direct and reverse records (6698 each) in 27 direct zones and 171 reverse zones
2021-09-03 05:28:02,216 [INFO] Cloning /srv/netbox-exports/dns.git/ to /tmp/dns-c25pcHBldHM-a5evgy4f ...
2021-09-03 05:28:02,455 [INFO] Generating zonefile snippets to directory /tmp/dns-c25pcHBldHM-a5evgy4f
2021-09-03 05:28:03,237 [INFO] Committed changes: eb549766e7beb726034321e0e58762d43e157be5
2021-09-03 05:28:03,264 [INFO] Validating generated data
2021-09-03 05:28:03,264 [INFO] Commit details: {'insertions': 0, 'deletions': 4, 'lines': 4, 'files': 4}
commit eb549766e7beb726034321e0e58762d43e157be5
Author: generate-dns-snippets <noc@wikimedia.org>
Date: Fri Sep 3 05:28:03 2021 +0000
marostegui@cumin1001: pc2007.codfw.wmnet decommissioned, removing all IPs except the asset tag one
diff --git a/0.192.10.in-addr.arpa b/0.192.10.in-addr.arpa
index 2f62d7d..5543f93 100644
--- a/0.192.10.in-addr.arpa
+++ b/0.192.10.in-addr.arpa
@@ -99,7 +99,6 @@
101 1H IN PTR db2094.codfw.wmnet.
102 1H IN PTR graphite2003.codfw.wmnet.
103 1H IN PTR rpki2001.codfw.wmnet.
-104 1H IN PTR pc2007.codfw.wmnet.
105 1H IN PTR ms-be2044.codfw.wmnet.
106 1H IN PTR ms-be2045.codfw.wmnet.
107 1H IN PTR elastic2037.codfw.wmnet.
diff --git a/1.193.10.in-addr.arpa b/1.193.10.in-addr.arpa
index f2d26e7..fbe0bbd 100644
--- a/1.193.10.in-addr.arpa
+++ b/1.193.10.in-addr.arpa
@@ -273,7 +273,6 @@
206 1H IN PTR wmf6646.mgmt.codfw.wmnet.
207 1H IN PTR bast2002.mgmt.codfw.wmnet.
207 1H IN PTR wmf6649.mgmt.codfw.wmnet.
-208 1H IN PTR pc2007.mgmt.codfw.wmnet.
208 1H IN PTR wmf6659.mgmt.codfw.wmnet.
209 1H IN PTR pc2008.mgmt.codfw.wmnet.
209 1H IN PTR wmf6660.mgmt.codfw.wmnet.
diff --git a/codfw.wmnet b/codfw.wmnet
index 9bdde11..b5315ae 100644
--- a/codfw.wmnet
+++ b/codfw.wmnet
@@ -738,7 +738,6 @@ parse2017 1H IN A 10.192.48.150
parse2018 1H IN A 10.192.48.151
parse2019 1H IN A 10.192.48.152
parse2020 1H IN A 10.192.48.153
-pc2007 1H IN A 10.192.0.104
pc2008 1H IN A 10.192.16.35
pc2009 1H IN A 10.192.32.10
pc2010 1H IN A 10.192.48.14
diff --git a/mgmt.codfw.wmnet b/mgmt.codfw.wmnet
index d2d167b..3cb199d 100644
--- a/mgmt.codfw.wmnet
+++ b/mgmt.codfw.wmnet
@@ -537,7 +537,6 @@ parse2017 1H IN A 10.193.3.124
parse2018 1H IN A 10.193.3.125
parse2019 1H IN A 10.193.3.126
parse2020 1H IN A 10.193.3.127
-pc2007 1H IN A 10.193.1.208
pc2008 1H IN A 10.193.1.209
pc2009 1H IN A 10.193.1.210
pc2010 1H IN A 10.193.1.211
METADATA: {"path": "/tmp/dns-c25pcHBldHM-a5evgy4f", "sha1": "eb549766e7beb726034321e0e58762d43e157be5", "insertions": 0, "deletions": 4, "lines": 4, "files": 4}
================
PASS |██████████████████████████████████████████████████████████████████████████████| 100% (1/1) [03:35<00:00, 215.06s/hosts]
FAIL | | 0% (0/1) [03:35<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'cd /tmp && runus...e asset tag one"'.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
>>> Have you checked that the diff is OK?
Type "go" to proceed or "abort" to interrupt the execution
> go
----- OUTPUT of 'cd /tmp && runus...58762d43e157be5"' -----
2021-09-03 05:28:07,787 [INFO] Pushed with bitflags 256: 5ba1dcb..eb54976
2021-09-03 05:28:07,878 [INFO] Temporary directory /tmp/dns-c25pcHBldHM-a5evgy4f removed.
================
PASS |███████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:00<00:00, 1.12hosts/s]
FAIL | | 0% (0/1) [00:00<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'cd /tmp && runus...58762d43e157be5"'.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
Updating the Netbox passive copies of the repository on netbox2001.wikimedia.org
----- OUTPUT of 'runuser -u netbo...rg master:master' -----
From https://netbox1001.wikimedia.org/dns
5ba1dcb..eb54976 master -> master
5ba1dcb..eb54976 master -> netbox1001.wikimedia.org/master
================
PASS |███████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:01<00:00, 1.54s/hosts]
FAIL | | 0% (0/1) [00:01<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'runuser -u netbo...rg master:master'.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
Updating the authdns copies of the repository on authdns[1001,2001].wikimedia.org,dns[1001-1002,2001-2002,3001-3002,4001-4002,5001-5002].wikimedia.org
===== NODE GROUP =====
(12) authdns[1001,2001].wikimedia.org,dns[1001-1002,2001-2002,3001-3002,4001-4002,5001-5002].wikimedia.org
----- OUTPUT of 'runuser -u netbo...e58762d43e157be5' -----
From https://netbox-exports.wikimedia.org/dns
5ba1dcb..eb54976 master -> origin/master
Updating 5ba1dcb..eb54976
Fast-forward
0.192.10.in-addr.arpa | 1 -
1.193.10.in-addr.arpa | 1 -
codfw.wmnet | 1 -
mgmt.codfw.wmnet | 1 -
4 files changed, 4 deletions(-)
================
PASS |█████████████████████████████████████████████████████████████████████████████| 100% (12/12) [00:06<00:00, 1.29hosts/s]
FAIL | | 0% (0/12) [00:06<?, ?hosts/s]
100.0% (12/12) success ratio (>= 100.0% threshold) for command: 'runuser -u netbo...e58762d43e157be5'.
100.0% (12/12) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
Deploying the updated zonefiles on authdns[1001,2001].wikimedia.org,dns[1001-1002,2001-2002,3001-3002,4001-4002,5001-5002].wikimedia.org
===== NODE GROUP =====
(1) dns5002.wikimedia.org
----- OUTPUT of 'cd /srv/authdns/...nippets --deploy' -----
Assembling and testing data in /tmp/dns-check.mc0iy3_c
-- Generating zonefiles from zone templates
-- Processed 209 zones into directory /tmp/dns-check.mc0iy3_c/zones
OK: No tabs
Summary of violations:
W001|MISSING_IP_FOR_NAME_AND_PTR: 37
W002|MISSING_PTR_FOR_NAME_AND_IP: 26
W103|MISSING_MGMT_FOR_NAME: 64
W105|TOO_MANY_PUBLIC_NAMES: 3
RESULT: 0 Errors, 130 Warnings, 0 Ignored violations, 0 Ignored lines
-- Copying automatically generated zone files under target tree
-- Copying repo-driven real config files and admin_state
-- Copying puppetized config and GeoIP from /etc/gdnsd
-- Checking for illegal tabs in zonefiles
-- Running zone_validator to check WMF rules
-- Running /usr/sbin/gdnsd checkconf on /tmp/dns-check.mc0iy3_c
Traceback (most recent call last):
File "utils/deploy-check.py", line 276, in <module>
main()
File "utils/deploy-check.py", line 268, in main
deploy_check(args.deploy, args.skip_reload, args.no_gdnsd, Path(tdir), gdir)
File "utils/deploy-check.py", line 214, in deploy_check
safe_cmd([GDNSD_BIN, '-c', str(tdir), 'checkconf'])
File "utils/deploy-check.py", line 88, in safe_cmd
p_err.decode('utf-8')))
Exception: Command /usr/sbin/gdnsd -c /tmp/dns-check.mc0iy3_c checkconf failed with exit code 42, stderr:
info: gdnsd version 3.8.0 @ pid 31160
info: DNS listener threads (16 UDP + 16 TCP) configured for 208.80.154.238:53
info: DNS listener threads (16 UDP + 16 TCP) configured for 208.80.153.231:53
info: DNS listener threads (16 UDP + 16 TCP) configured for 91.198.174.239:53
info: DNS listener threads (16 UDP + 16 TCP) configured for 198.35.27.27:53
info: DNS listener threads (16 TCP PROXY) configured for 127.0.0.1:535
info: DNS listener threads (1 UDP + 1 TCP) configured for 0.0.0.0:5353
info: DNS listener threads (1 UDP + 1 TCP) configured for [::]:5353
info: plugin_geoip: map 'generic-map': Loading GeoIP2 database '/tmp/dns-check.mc0iy3_c/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'generic-map' runtime db updated. nets: 1174886 dclists: 8
info: plugin_geoip: map 'checkdoh-map' runtime db updated. nets: 410 dclists: 2
info: plugin_geoip: map 'discovery-map': Loading GeoIP2 database '/tmp/dns-check.mc0iy3_c/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'discovery-map' runtime db updated. nets: 464 dclists: 2
info: admin_state: checking state file '/tmp/dns-check.mc0iy3_c/state/admin_state'...
error: CNAME 'pc1-master.eqiad.wmnet.' points to known same-zone NXDOMAIN 'pc2007.codfw.wmnet.'
info: Zone wikibook.com. with serial 2019120223 loaded
info: Zone wiktionary.eu. with serial 2019120223 loaded
info: Zone wikidisclosure.com. with serial 2019120223 loaded
info: Zone 0.8.c.e.2.0.a.2.ip6.arpa. with serial 2020100116 loaded
info: Zone en-wp.org. with serial 2019120223 loaded
info: Zone wikipedia.uk. with serial 2019120223 loaded
info: Zone xn--b1aajamacm1dkmb.xn--p1ai. with serial 2019120223 loaded
info: Zone wikiepdia.com. with serial 2019120223 loaded
info: Zone wikivoyage.com. with serial 2019120223 loaded
info: Zone wikibooks.org. with serial 2019120223 loaded
info: Zone wikidpedia.org. with serial 2019120223 loaded
info: Zone wikipedia.org.il. with serial 2019120223 loaded
info: Zone wikipedia.net. with serial 2019120223 loaded
info: Zone wikimediacommons.org. with serial 2019120223 loaded
info: Zone wikimedia-dns.net. with serial 2021051815 loaded
info: Zone wikiversity.com. with serial 2019120223 loaded
info: Zone wikipedia.lol. with serial 2019120223 loaded
info: Zone wikivoyage-old.org. with serial 2019120223 loaded
info: Zone wikivoyage.eu. with serial 2019120223 loaded
info: Zone wikispecies.com. with serial 2019120223 loaded
info: Zone wikimediafoundation.net. with serial 2019120223 loaded
info: Zone vikipedia.com.tr. with serial 2019120223 loaded
info: Zone wikimedia-dns.com. with serial 2021051815 loaded
info: Zone wikiversity.pt. with serial 2019120223 loaded
info: Zone wikidata.es. with serial 2019120223 loaded
info: Zone wmfusercontent.org. with serial 2020072316 loaded
info: Zone wikidata.us. with serial 2019120223 loaded
info: Zone wikisource.org. with serial 2019120223 loaded
info: Zone wikipedia.lt. with serial 2019120223 loaded
info: Zone wikipedia.bg. with serial 2019120223 loaded
info: Zone wwwwikimedia.com. with serial 2019120223 loaded
info: Zone wikipedia.es. with serial 2019120223 loaded
info: Zone 57.15.185.in-addr.arpa. with serial 2020102310 loaded
info: Zone wikimedia-dns.org. with serial 2021083014 loaded
info: Zone en-wp.com. with serial 2019120223 loaded
info: Zone wikipedoa.org. with serial 2019120223 loaded
info: Zone 152.80.208.in-addr.arpa. with serial 2021072114 loaded
info: Zone wiktionary.org. with serial 2019120223 loaded
info: Zone wikidata.org. with serial 2021031301 loaded
info: Zone wikjpedia.org. with serial 2019120223 loaded
info: Zone 58.15.185.in-addr.arpa. with serial 2019120223 loaded
info: Zone wikpedia.org. with serial 2019120223 loaded
info: Zone 174.198.91.in-addr.arpa. with serial 2020100816 loaded
info: Zone wikiipedia.org. with serial 2019120223 loaded
info: Zone wiktionary.pl. with serial 2019120223 loaded
info: Zone wikimediastories.net. with serial 2019120223 loaded
info: Zone 3.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa. with serial 2021020109 loaded
info: Zone wikipedia.info. with serial 2019120223 loaded
info: Zone wikiquote.com. with serial 2019120223 loaded
info: Zone wikimediafoundation.org. with serial 2020071716 loaded
info: Zone wikitrends.org. with serial 2019120223 loaded
info: Zone wilipedia.com. with serial 2019120223 loaded
info: Zone wikimediacommons.net. with serial 2019120223 loaded
info: Zone wikimediacommons.info. with serial 2019120223 loaded
info: Zone wikipedia.in. with serial 2019120223 loaded
info: Zone wikipediastats.org. with serial 2019120223 loaded
info: Zone wikiartpedia.co. with serial 2019120223 loaded
info: Zone wiktionary.pt. with serial 2019120223 loaded
info: Zone wikiartpedia.mobi. with serial 2019120223 loaded
info: Zone wikiquote.pl. with serial 2019120223 loaded
info: Zone wikimaps.org. with serial 2019120223 loaded
info: Zone wikipedial.org. with serial 2019120223 loaded
info: Zone wikipediya.com. with serial 2019120223 loaded
info: Zone wikispecies.net. with serial 2019120223 loaded
info: Zone wikisource.pl. with serial 2019120223 loaded
info: Zone indianwikimedia.com. with serial 2019120223 loaded
info: Zone wikipedia.ee. with serial 2019120223 loaded
info: Zone wikispecies.org. with serial 2019120223 loaded
info: Zone wikiworkshop.org. with serial 2020022717 loaded
info: Zone wikimediacloud.org. with serial 2021050615 loaded
info: Zone wikkipedia.com. with serial 2019120223 loaded
info: Zone wikinews.pt. with serial 2019120223 loaded
info: Zone wikimediafoundation.info. with serial 2019120223 loaded
info: Zone wikidisclosure.org. with serial 2019120223 loaded
info: Zone vikipedia.com. with serial 2019120223 loaded
info: Zone wikimaps.net. with serial 2019120223 loaded
info: Zone indiawikipedia.com. with serial 2019120223 loaded
info: Zone softwarewikipedia.net. with serial 2019120223 loaded
info: Zone wikipdia.org. with serial 2019120223 loaded
info: Zone donatetowikipedia.org. with serial 2019120223 loaded
info: Zone webhostingwikipedia.com. with serial 2019120223 loaded
info: Zone xn--wikirda-p0a62emr.pl. with serial 2020021417 loaded
info: Zone wikepedia.org. with serial 2019120223 loaded
info: Zone wikimedia.community. with serial 2020010209 loaded
info: Zone wikimediacommons.co.uk. with serial 2019120223 loaded
info: Zone wikimedia.com.pt. with serial 2019120223 loaded
info: Zone wiikipedia.com. with serial 2019120223 loaded
info: Zone wikipedia.com. with serial 2019120223 loaded
info: Zone wikimedia.jp.net. with serial 2019120223 loaded
info: Zone 59.15.185.in-addr.arpa. with serial 2020100116 loaded
info: Zone wiki-pedia.org. with serial 2019120223 loaded
info: Zone wekipedia.com. with serial 2019120223 loaded
info: Zone wikinews.org. with serial 2019120223 loaded
info: Zone wikivoyager.org. with serial 2019120223 loaded
info: Zone wikipedia.co.za. with serial 2019120223 loaded
info: Zone 0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa. with serial 2021020109 loaded
info: Zone mediawiki.org. with serial 2020043010 loaded
info: Zone wikimediastories.org. with serial 2019120223 loaded
info: Zone wmftest.org. with serial 2020011000 loaded
info: Zone w.wiki. with serial 2019120223 loaded
info: Zone 26.35.198.in-addr.arpa. with serial 2020100816 loaded
info: Zone wikibase.org. with serial 2019120223 loaded
info: Zone wikipedia.is. with serial 2019120223 loaded
info: Zone wikisource.com. with serial 2019120223 loaded
fatal: Initial load of zone data failed
===== NODE GROUP =====
(1) dns5001.wikimedia.org
----- OUTPUT of 'cd /srv/authdns/...nippets --deploy' -----
Assembling and testing data in /tmp/dns-check.gt1966ca
-- Generating zonefiles from zone templates
-- Processed 209 zones into directory /tmp/dns-check.gt1966ca/zones
OK: No tabs
Summary of violations:
W001|MISSING_IP_FOR_NAME_AND_PTR: 37
W002|MISSING_PTR_FOR_NAME_AND_IP: 26
W103|MISSING_MGMT_FOR_NAME: 64
W105|TOO_MANY_PUBLIC_NAMES: 3
RESULT: 0 Errors, 130 Warnings, 0 Ignored violations, 0 Ignored lines
-- Copying automatically generated zone files under target tree
-- Copying repo-driven real config files and admin_state
-- Copying puppetized config and GeoIP from /etc/gdnsd
-- Checking for illegal tabs in zonefiles
-- Running zone_validator to check WMF rules
-- Running /usr/sbin/gdnsd checkconf on /tmp/dns-check.gt1966ca
Traceback (most recent call last):
File "utils/deploy-check.py", line 276, in <module>
main()
File "utils/deploy-check.py", line 268, in main
deploy_check(args.deploy, args.skip_reload, args.no_gdnsd, Path(tdir), gdir)
File "utils/deploy-check.py", line 214, in deploy_check
safe_cmd([GDNSD_BIN, '-c', str(tdir), 'checkconf'])
File "utils/deploy-check.py", line 88, in safe_cmd
p_err.decode('utf-8')))
Exception: Command /usr/sbin/gdnsd -c /tmp/dns-check.gt1966ca checkconf failed with exit code 42, stderr:
info: gdnsd version 3.8.0 @ pid 25464
info: DNS listener threads (16 UDP + 16 TCP) configured for 208.80.154.238:53
info: DNS listener threads (16 UDP + 16 TCP) configured for 208.80.153.231:53
info: DNS listener threads (16 UDP + 16 TCP) configured for 91.198.174.239:53
info: DNS listener threads (16 UDP + 16 TCP) configured for 198.35.27.27:53
info: DNS listener threads (16 TCP PROXY) configured for 127.0.0.1:535
info: DNS listener threads (1 UDP + 1 TCP) configured for 0.0.0.0:5353
info: DNS listener threads (1 UDP + 1 TCP) configured for [::]:5353
info: plugin_geoip: map 'generic-map': Loading GeoIP2 database '/tmp/dns-check.gt1966ca/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'generic-map' runtime db updated. nets: 1174886 dclists: 8
info: plugin_geoip: map 'checkdoh-map' runtime db updated. nets: 410 dclists: 2
info: plugin_geoip: map 'discovery-map': Loading GeoIP2 database '/tmp/dns-check.gt1966ca/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'discovery-map' runtime db updated. nets: 464 dclists: 2
info: admin_state: checking state file '/tmp/dns-check.gt1966ca/state/admin_state'...
error: CNAME 'pc1-master.eqiad.wmnet.' points to known same-zone NXDOMAIN 'pc2007.codfw.wmnet.'
fatal: Initial load of zone data failed
===== NODE GROUP =====
(1) dns3001.wikimedia.org
----- OUTPUT of 'cd /srv/authdns/...nippets --deploy' -----
Assembling and testing data in /tmp/dns-check.trnfdvit
-- Generating zonefiles from zone templates
-- Processed 209 zones into directory /tmp/dns-check.trnfdvit/zones
OK: No tabs
Summary of violations:
W001|MISSING_IP_FOR_NAME_AND_PTR: 37
W002|MISSING_PTR_FOR_NAME_AND_IP: 26
W103|MISSING_MGMT_FOR_NAME: 64
W105|TOO_MANY_PUBLIC_NAMES: 3
RESULT: 0 Errors, 130 Warnings, 0 Ignored violations, 0 Ignored lines
-- Copying automatically generated zone files under target tree
-- Copying repo-driven real config files and admin_state
-- Copying puppetized config and GeoIP from /etc/gdnsd
-- Checking for illegal tabs in zonefiles
-- Running zone_validator to check WMF rules
-- Running /usr/sbin/gdnsd checkconf on /tmp/dns-check.trnfdvit
Traceback (most recent call last):
File "utils/deploy-check.py", line 276, in <module>
main()
File "utils/deploy-check.py", line 268, in main
deploy_check(args.deploy, args.skip_reload, args.no_gdnsd, Path(tdir), gdir)
File "utils/deploy-check.py", line 214, in deploy_check
safe_cmd([GDNSD_BIN, '-c', str(tdir), 'checkconf'])
File "utils/deploy-check.py", line 88, in safe_cmd
p_err.decode('utf-8')))
Exception: Command /usr/sbin/gdnsd -c /tmp/dns-check.trnfdvit checkconf failed with exit code 42, stderr:
info: gdnsd version 3.8.0 @ pid 29502
info: DNS listener threads (16 UDP + 16 TCP) configured for 208.80.154.238:53
info: DNS listener threads (16 UDP + 16 TCP) configured for 208.80.153.231:53
info: DNS listener threads (16 UDP + 16 TCP) configured for 91.198.174.239:53
info: DNS listener threads (16 UDP + 16 TCP) configured for 198.35.27.27:53
info: DNS listener threads (16 TCP PROXY) configured for 127.0.0.1:535
info: DNS listener threads (1 UDP + 1 TCP) configured for 0.0.0.0:5353
info: DNS listener threads (1 UDP + 1 TCP) configured for [::]:5353
info: plugin_geoip: map 'generic-map': Loading GeoIP2 database '/tmp/dns-check.trnfdvit/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'generic-map' runtime db updated. nets: 1174886 dclists: 8
info: plugin_geoip: map 'checkdoh-map' runtime db updated. nets: 410 dclists: 2
info: plugin_geoip: map 'discovery-map': Loading GeoIP2 database '/tmp/dns-check.trnfdvit/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'discovery-map' runtime db updated. nets: 464 dclists: 2
info: admin_state: checking state file '/tmp/dns-check.trnfdvit/state/admin_state'...
error: CNAME 'pc1-master.eqiad.wmnet.' points to known same-zone NXDOMAIN 'pc2007.codfw.wmnet.'
fatal: Initial load of zone data failed
===== NODE GROUP =====
(1) dns3002.wikimedia.org
----- OUTPUT of 'cd /srv/authdns/...nippets --deploy' -----
Assembling and testing data in /tmp/dns-check.stkawh1x
-- Generating zonefiles from zone templates
-- Processed 209 zones into directory /tmp/dns-check.stkawh1x/zones
OK: No tabs
Summary of violations:
W001|MISSING_IP_FOR_NAME_AND_PTR: 37
W002|MISSING_PTR_FOR_NAME_AND_IP: 26
W103|MISSING_MGMT_FOR_NAME: 64
W105|TOO_MANY_PUBLIC_NAMES: 3
RESULT: 0 Errors, 130 Warnings, 0 Ignored violations, 0 Ignored lines
-- Copying automatically generated zone files under target tree
-- Copying repo-driven real config files and admin_state
-- Copying puppetized config and GeoIP from /etc/gdnsd
-- Checking for illegal tabs in zonefiles
-- Running zone_validator to check WMF rules
-- Running /usr/sbin/gdnsd checkconf on /tmp/dns-check.stkawh1x
Traceback (most recent call last):
File "utils/deploy-check.py", line 276, in <module>
main()
File "utils/deploy-check.py", line 268, in main
deploy_check(args.deploy, args.skip_reload, args.no_gdnsd, Path(tdir), gdir)
File "utils/deploy-check.py", line 214, in deploy_check
safe_cmd([GDNSD_BIN, '-c', str(tdir), 'checkconf'])
File "utils/deploy-check.py", line 88, in safe_cmd
p_err.decode('utf-8')))
Exception: Command /usr/sbin/gdnsd -c /tmp/dns-check.stkawh1x checkconf failed with exit code 42, stderr:
info: gdnsd version 3.8.0 @ pid 14851
info: DNS listener threads (16 UDP + 16 TCP) configured for 208.80.154.238:53
info: DNS listener threads (16 UDP + 16 TCP) configured for 208.80.153.231:53
info: DNS listener threads (16 UDP + 16 TCP) configured for 91.198.174.239:53
info: DNS listener threads (16 UDP + 16 TCP) configured for 198.35.27.27:53
info: DNS listener threads (16 TCP PROXY) configured for 127.0.0.1:535
info: DNS listener threads (1 UDP + 1 TCP) configured for 0.0.0.0:5353
info: DNS listener threads (1 UDP + 1 TCP) configured for [::]:5353
info: plugin_geoip: map 'generic-map': Loading GeoIP2 database '/tmp/dns-check.stkawh1x/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'generic-map' runtime db updated. nets: 1174886 dclists: 8
info: plugin_geoip: map 'checkdoh-map' runtime db updated. nets: 410 dclists: 2
info: plugin_geoip: map 'discovery-map': Loading GeoIP2 database '/tmp/dns-check.stkawh1x/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'discovery-map' runtime db updated. nets: 464 dclists: 2
info: admin_state: checking state file '/tmp/dns-check.stkawh1x/state/admin_state'...
error: CNAME 'pc1-master.eqiad.wmnet.' points to known same-zone NXDOMAIN 'pc2007.codfw.wmnet.'
fatal: Initial load of zone data failed
===== NODE GROUP =====
(1) dns4002.wikimedia.org
----- OUTPUT of 'cd /srv/authdns/...nippets --deploy' -----
Assembling and testing data in /tmp/dns-check.e980frfq
-- Generating zonefiles from zone templates
-- Processed 209 zones into directory /tmp/dns-check.e980frfq/zones
OK: No tabs
Summary of violations:
W001|MISSING_IP_FOR_NAME_AND_PTR: 37
W002|MISSING_PTR_FOR_NAME_AND_IP: 26
W103|MISSING_MGMT_FOR_NAME: 64
W105|TOO_MANY_PUBLIC_NAMES: 3
RESULT: 0 Errors, 130 Warnings, 0 Ignored violations, 0 Ignored lines
-- Copying automatically generated zone files under target tree
-- Copying repo-driven real config files and admin_state
-- Copying puppetized config and GeoIP from /etc/gdnsd
-- Checking for illegal tabs in zonefiles
-- Running zone_validator to check WMF rules
-- Running /usr/sbin/gdnsd checkconf on /tmp/dns-check.e980frfq
Traceback (most recent call last):
File "utils/deploy-check.py", line 276, in <module>
main()
File "utils/deploy-check.py", line 268, in main
deploy_check(args.deploy, args.skip_reload, args.no_gdnsd, Path(tdir), gdir)
File "utils/deploy-check.py", line 214, in deploy_check
safe_cmd([GDNSD_BIN, '-c', str(tdir), 'checkconf'])
File "utils/deploy-check.py", line 88, in safe_cmd
p_err.decode('utf-8')))
Exception: Command /usr/sbin/gdnsd -c /tmp/dns-check.e980frfq checkconf failed with exit code 42, stderr:
info: gdnsd version 3.8.0 @ pid 26744
info: DNS listener threads (16 UDP + 16 TCP) configured for 208.80.154.238:53
info: DNS listener threads (16 UDP + 16 TCP) configured for 208.80.153.231:53
info: DNS listener threads (16 UDP + 16 TCP) configured for 91.198.174.239:53
info: DNS listener threads (16 UDP + 16 TCP) configured for 198.35.27.27:53
info: DNS listener threads (16 TCP PROXY) configured for 127.0.0.1:535
info: DNS listener threads (1 UDP + 1 TCP) configured for 0.0.0.0:5353
info: DNS listener threads (1 UDP + 1 TCP) configured for [::]:5353
info: plugin_geoip: map 'generic-map': Loading GeoIP2 database '/tmp/dns-check.e980frfq/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'generic-map' runtime db updated. nets: 1174886 dclists: 8
info: plugin_geoip: map 'checkdoh-map' runtime db updated. nets: 410 dclists: 2
info: plugin_geoip: map 'discovery-map': Loading GeoIP2 database '/tmp/dns-check.e980frfq/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'discovery-map' runtime db updated. nets: 464 dclists: 2
info: admin_state: checking state file '/tmp/dns-check.e980frfq/state/admin_state'...
error: CNAME 'pc1-master.eqiad.wmnet.' points to known same-zone NXDOMAIN 'pc2007.codfw.wmnet.'
info: Zone wikipedia.fi. with serial 2019120223 loaded
info: Zone wikiipedia.org. with serial 2019120223 loaded
info: Zone wikipedia.uk. with serial 2019120223 loaded
info: Zone wicipedia.com. with serial 2019120223 loaded
info: Zone wikimediastories.com. with serial 2019120223 loaded
info: Zone 10.in-addr.arpa. with serial 2021082316 loaded
info: Zone 152.80.208.in-addr.arpa. with serial 2021072114 loaded
info: Zone wikipedia.id. with serial 2019120223 loaded
info: Zone wikimedipedia.com. with serial 2019120223 loaded
info: Zone visualwikipedia.com. with serial 2019120223 loaded
info: Zone wikipdia.org. with serial 2019120223 loaded
info: Zone mediawiki.com. with serial 2019120223 loaded
info: Zone wikipedia.com. with serial 2019120223 loaded
info: Zone 153.80.208.in-addr.arpa. with serial 2021051720 loaded
info: Zone wikiquote.org. with serial 2019120223 loaded
info: Zone vikipedia.com.tr. with serial 2019120223 loaded
info: Zone wilkipedia.org. with serial 2019120223 loaded
info: Zone wmfusercontent.org. with serial 2020072316 loaded
info: Zone wikimedia.xyz. with serial 2019120223 loaded
info: Zone wikipedia.is. with serial 2019120223 loaded
info: Zone wikipedia.es. with serial 2019120223 loaded
info: Zone 138.71.185.in-addr.arpa. with serial 2021083014 loaded
info: Zone wikipediazero.org. with serial 2019120223 loaded
info: Zone wikiquote.com. with serial 2019120223 loaded
info: Zone wikimedia-dns.org. with serial 2021083014 loaded
info: Zone wikipedoa.org. with serial 2019120223 loaded
info: Zone webhostingwikipedia.com. with serial 2019120223 loaded
info: Zone visualwikipedia.net. with serial 2019120223 loaded
info: Zone wikimaps.com. with serial 2019120223 loaded
info: Zone wikidata.org. with serial 2021031301 loaded
info: Zone 0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa. with serial 2021020109 loaded
info: Zone wikipediafoundation.org. with serial 2019120223 loaded
info: Zone parking. with serial 2019120223 loaded
info: Zone wiki-pedia.org. with serial 2019120223 loaded
info: Zone wikiworkshop.org. with serial 2020022717 loaded
info: Zone 1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa. with serial 2021020109 loaded
info: Zone wikimemory.org. with serial 2019120223 loaded
info: Zone wikivoyage.de. with serial 2019120223 loaded
info: Zone wikijunior.com. with serial 2019120223 loaded
info: Zone wikidata.us. with serial 2019120223 loaded
info: Zone donatetowikipedia.com. with serial 2019120223 loaded
info: Zone wikimedia.jp.net. with serial 2019120223 loaded
info: Zone learn.wiki. with serial 2021083109 loaded
info: Zone xn--b1aajamacm1dkmb.xn--p1ai. with serial 2019120223 loaded
info: Zone wikiartpedia.mobi. with serial 2019120223 loaded
info: Zone wikibase.org. with serial 2019120223 loaded
info: Zone wikispecies.com. with serial 2019120223 loaded
info: Zone wikipediya.com. with serial 2019120223 loaded
info: Zone pywikibot.org. with serial 2020111908 loaded
info: Zone wikiartpedia.info. with serial 2019120223 loaded
info: Zone wikiartpedia.org. with serial 2019120223 loaded
info: Zone wekipedia.com. with serial 2019120223 loaded
info: Zone wikipedia.com.ar. with serial 2019120223 loaded
info: Zone 174.198.91.in-addr.arpa. with serial 2020100816 loaded
info: Zone en-wp.com. with serial 2019120223 loaded
info: Zone vikipedia.com. with serial 2019120223 loaded
info: Zone wikisource.org. with serial 2019120223 loaded
info: Zone 57.15.185.in-addr.arpa. with serial 2020102310 loaded
info: Zone wikidata.es. with serial 2019120223 loaded
info: Zone wiikipedia.com. with serial 2019120223 loaded
info: Zone wikkipedia.com. with serial 2019120223 loaded
info: Zone wikinews.com. with serial 2019120223 loaded
info: Zone wilipedia.com. with serial 2019120223 loaded
info: Zone wikepedia.org. with serial 2019120223 loaded
info: Zone wikimobipedia.net. with serial 2019120223 loaded
info: Zone wikipedia.org. with serial 2020120918 loaded
info: Zone wikimediacommons.org. with serial 2019120223 loaded
info: Zone 2.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa. with serial 2020100116 loaded
info: Zone wikimediacommons.net. with serial 2019120223 loaded
info: Zone 0.8.c.e.2.0.a.2.ip6.arpa. with serial 2020100116 loaded
info: Zone wikiversity.com. with serial 2019120223 loaded
info: Zone mediawiki.org. with serial 2020043010 loaded
info: Zone wikipediavideo.com. with serial 2019120223 loaded
info: Zone wikimania.asia. with serial 2019120223 loaded
info: Zone wikipediastories.com. with serial 2019120223 loaded
info: Zone wikimediafoundation.net. with serial 2019120223 loaded
info: Zone wikibooks.org. with serial 2019120223 loaded
info: Zone border-wikipedia.de. with serial 2019120223 loaded
info: Zone wikimedia.com.pt. with serial 2019120223 loaded
info: Zone wikibook.com. with serial 2019120223 loaded
info: Zone wikipedia.bg. with serial 2019120223 loaded
info: Zone wwwwikimedia.com. with serial 2019120223 loaded
info: Zone wikipedia.org.il. with serial 2019120223 loaded
info: Zone wikimediafoundation.info. with serial 2019120223 loaded
info: Zone wikispecies.net. with serial 2019120223 loaded
info: Zone wikimediacommons.co.uk. with serial 2019120223 loaded
info: Zone wikisource.pt. with serial 2019120223 loaded
info: Zone wikipedia.info. with serial 2019120223 loaded
info: Zone wikimaps.org. with serial 2019120223 loaded
info: Zone wikipediastats.com. with serial 2019120223 loaded
info: Zone wikimania.org. with serial 2019120223 loaded
info: Zone wikiepdia.org. with serial 2019120223 loaded
info: Zone wikiversity.org. with serial 2019120223 loaded
info: Zone wikipedia.lt. with serial 2019120223 loaded
info: Zone wikimediafoundation.com. with serial 2019120223 loaded
info: Zone voyagewiki.org. with serial 2019120223 loaded
info: Zone wikiartpedia.biz. with serial 2019120223 loaded
info: Zone wikidata.pt. with serial 2019120223 loaded
info: Zone wikisource.com. with serial 2019120223 loaded
info: Zone wikivoyager.org. with serial 2019120223 loaded
info: Zone xn--wikisownik-e0b.pl. with serial 2020021417 loaded
info: Zone wikjpedia.org. with serial 2019120223 loaded
info: Zone 3.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa. with serial 2021020109 loaded
info: Zone wikipedia.gr. with serial 2019120223 loaded
fatal: Initial load of zone data failed
===== NODE GROUP =====
(1) dns4001.wikimedia.org
----- OUTPUT of 'cd /srv/authdns/...nippets --deploy' -----
Assembling and testing data in /tmp/dns-check.7lxfz9ay
-- Generating zonefiles from zone templates
-- Processed 209 zones into directory /tmp/dns-check.7lxfz9ay/zones
OK: No tabs
Summary of violations:
W001|MISSING_IP_FOR_NAME_AND_PTR: 37
W002|MISSING_PTR_FOR_NAME_AND_IP: 26
W103|MISSING_MGMT_FOR_NAME: 64
W105|TOO_MANY_PUBLIC_NAMES: 3
RESULT: 0 Errors, 130 Warnings, 0 Ignored violations, 0 Ignored lines
-- Copying automatically generated zone files under target tree
-- Copying repo-driven real config files and admin_state
-- Copying puppetized config and GeoIP from /etc/gdnsd
-- Checking for illegal tabs in zonefiles
-- Running zone_validator to check WMF rules
-- Running /usr/sbin/gdnsd checkconf on /tmp/dns-check.7lxfz9ay
Traceback (most recent call last):
File "utils/deploy-check.py", line 276, in <module>
main()
File "utils/deploy-check.py", line 268, in main
deploy_check(args.deploy, args.skip_reload, args.no_gdnsd, Path(tdir), gdir)
File "utils/deploy-check.py", line 214, in deploy_check
safe_cmd([GDNSD_BIN, '-c', str(tdir), 'checkconf'])
File "utils/deploy-check.py", line 88, in safe_cmd
p_err.decode('utf-8')))
Exception: Command /usr/sbin/gdnsd -c /tmp/dns-check.7lxfz9ay checkconf failed with exit code 42, stderr:
info: gdnsd version 3.8.0 @ pid 4507
info: DNS listener threads (16 UDP + 16 TCP) configured for 208.80.154.238:53
info: DNS listener threads (16 UDP + 16 TCP) configured for 208.80.153.231:53
info: DNS listener threads (16 UDP + 16 TCP) configured for 91.198.174.239:53
info: DNS listener threads (16 UDP + 16 TCP) configured for 198.35.27.27:53
info: DNS listener threads (16 TCP PROXY) configured for 127.0.0.1:535
info: DNS listener threads (1 UDP + 1 TCP) configured for 0.0.0.0:5353
info: DNS listener threads (1 UDP + 1 TCP) configured for [::]:5353
info: plugin_geoip: map 'generic-map': Loading GeoIP2 database '/tmp/dns-check.7lxfz9ay/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'generic-map' runtime db updated. nets: 1174886 dclists: 8
info: plugin_geoip: map 'checkdoh-map' runtime db updated. nets: 410 dclists: 2
info: plugin_geoip: map 'discovery-map': Loading GeoIP2 database '/tmp/dns-check.7lxfz9ay/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'discovery-map' runtime db updated. nets: 464 dclists: 2
info: admin_state: checking state file '/tmp/dns-check.7lxfz9ay/state/admin_state'...
error: CNAME 'pc1-master.eqiad.wmnet.' points to known same-zone NXDOMAIN 'pc2007.codfw.wmnet.'
fatal: Initial load of zone data failed
===== NODE GROUP =====
(1) authdns2001.wikimedia.org
----- OUTPUT of 'cd /srv/authdns/...nippets --deploy' -----
Assembling and testing data in /tmp/dns-check.kt4zn6ev
-- Generating zonefiles from zone templates
-- Processed 209 zones into directory /tmp/dns-check.kt4zn6ev/zones
OK: No tabs
Summary of violations:
W001|MISSING_IP_FOR_NAME_AND_PTR: 37
W002|MISSING_PTR_FOR_NAME_AND_IP: 26
W103|MISSING_MGMT_FOR_NAME: 64
W105|TOO_MANY_PUBLIC_NAMES: 3
RESULT: 0 Errors, 130 Warnings, 0 Ignored violations, 0 Ignored lines
-- Copying automatically generated zone files under target tree
-- Copying repo-driven real config files and admin_state
-- Copying puppetized config and GeoIP from /etc/gdnsd
-- Checking for illegal tabs in zonefiles
-- Running zone_validator to check WMF rules
-- Running /usr/sbin/gdnsd checkconf on /tmp/dns-check.kt4zn6ev
Traceback (most recent call last):
File "utils/deploy-check.py", line 276, in <module>
main()
File "utils/deploy-check.py", line 268, in main
deploy_check(args.deploy, args.skip_reload, args.no_gdnsd, Path(tdir), gdir)
File "utils/deploy-check.py", line 214, in deploy_check
safe_cmd([GDNSD_BIN, '-c', str(tdir), 'checkconf'])
File "utils/deploy-check.py", line 88, in safe_cmd
p_err.decode('utf-8')))
Exception: Command /usr/sbin/gdnsd -c /tmp/dns-check.kt4zn6ev checkconf failed with exit code 42, stderr:
info: gdnsd version 3.8.0 @ pid 32720
info: DNS listener threads (8 UDP + 8 TCP) configured for 208.80.154.238:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 208.80.153.231:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 91.198.174.239:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 198.35.27.27:53
info: DNS listener threads (8 TCP PROXY) configured for 127.0.0.1:535
info: DNS listener threads (1 UDP + 1 TCP) configured for 0.0.0.0:5353
info: DNS listener threads (1 UDP + 1 TCP) configured for [::]:5353
info: plugin_geoip: map 'generic-map': Loading GeoIP2 database '/tmp/dns-check.kt4zn6ev/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'generic-map' runtime db updated. nets: 1174886 dclists: 8
info: plugin_geoip: map 'checkdoh-map' runtime db updated. nets: 410 dclists: 2
info: plugin_geoip: map 'discovery-map': Loading GeoIP2 database '/tmp/dns-check.kt4zn6ev/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'discovery-map' runtime db updated. nets: 464 dclists: 2
info: admin_state: checking state file '/tmp/dns-check.kt4zn6ev/state/admin_state'...
error: CNAME 'pc1-master.eqiad.wmnet.' points to known same-zone NXDOMAIN 'pc2007.codfw.wmnet.'
fatal: Initial load of zone data failed
===== NODE GROUP =====
(1) dns2002.wikimedia.org
----- OUTPUT of 'cd /srv/authdns/...nippets --deploy' -----
Assembling and testing data in /tmp/dns-check.wuq5kk2l
-- Generating zonefiles from zone templates
-- Processed 209 zones into directory /tmp/dns-check.wuq5kk2l/zones
OK: No tabs
Summary of violations:
W001|MISSING_IP_FOR_NAME_AND_PTR: 37
W002|MISSING_PTR_FOR_NAME_AND_IP: 26
W103|MISSING_MGMT_FOR_NAME: 64
W105|TOO_MANY_PUBLIC_NAMES: 3
RESULT: 0 Errors, 130 Warnings, 0 Ignored violations, 0 Ignored lines
-- Copying automatically generated zone files under target tree
-- Copying repo-driven real config files and admin_state
-- Copying puppetized config and GeoIP from /etc/gdnsd
-- Checking for illegal tabs in zonefiles
-- Running zone_validator to check WMF rules
-- Running /usr/sbin/gdnsd checkconf on /tmp/dns-check.wuq5kk2l
Traceback (most recent call last):
File "utils/deploy-check.py", line 276, in <module>
main()
File "utils/deploy-check.py", line 268, in main
deploy_check(args.deploy, args.skip_reload, args.no_gdnsd, Path(tdir), gdir)
File "utils/deploy-check.py", line 214, in deploy_check
safe_cmd([GDNSD_BIN, '-c', str(tdir), 'checkconf'])
File "utils/deploy-check.py", line 88, in safe_cmd
p_err.decode('utf-8')))
Exception: Command /usr/sbin/gdnsd -c /tmp/dns-check.wuq5kk2l checkconf failed with exit code 42, stderr:
info: gdnsd version 3.8.0 @ pid 11112
info: DNS listener threads (8 UDP + 8 TCP) configured for 208.80.154.238:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 208.80.153.231:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 91.198.174.239:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 198.35.27.27:53
info: DNS listener threads (8 TCP PROXY) configured for 127.0.0.1:535
info: DNS listener threads (1 UDP + 1 TCP) configured for 0.0.0.0:5353
info: DNS listener threads (1 UDP + 1 TCP) configured for [::]:5353
info: plugin_geoip: map 'generic-map': Loading GeoIP2 database '/tmp/dns-check.wuq5kk2l/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'generic-map' runtime db updated. nets: 1174886 dclists: 8
info: plugin_geoip: map 'checkdoh-map' runtime db updated. nets: 410 dclists: 2
info: plugin_geoip: map 'discovery-map': Loading GeoIP2 database '/tmp/dns-check.wuq5kk2l/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'discovery-map' runtime db updated. nets: 464 dclists: 2
info: admin_state: checking state file '/tmp/dns-check.wuq5kk2l/state/admin_state'...
error: CNAME 'pc1-master.eqiad.wmnet.' points to known same-zone NXDOMAIN 'pc2007.codfw.wmnet.'
fatal: Initial load of zone data failed
===== NODE GROUP =====
(1) dns2001.wikimedia.org
----- OUTPUT of 'cd /srv/authdns/...nippets --deploy' -----
Assembling and testing data in /tmp/dns-check.kwc3e7kn
-- Generating zonefiles from zone templates
-- Processed 209 zones into directory /tmp/dns-check.kwc3e7kn/zones
OK: No tabs
Summary of violations:
W001|MISSING_IP_FOR_NAME_AND_PTR: 37
W002|MISSING_PTR_FOR_NAME_AND_IP: 26
W103|MISSING_MGMT_FOR_NAME: 64
W105|TOO_MANY_PUBLIC_NAMES: 3
RESULT: 0 Errors, 130 Warnings, 0 Ignored violations, 0 Ignored lines
-- Copying automatically generated zone files under target tree
-- Copying repo-driven real config files and admin_state
-- Copying puppetized config and GeoIP from /etc/gdnsd
-- Checking for illegal tabs in zonefiles
-- Running zone_validator to check WMF rules
-- Running /usr/sbin/gdnsd checkconf on /tmp/dns-check.kwc3e7kn
Traceback (most recent call last):
File "utils/deploy-check.py", line 276, in <module>
main()
File "utils/deploy-check.py", line 268, in main
deploy_check(args.deploy, args.skip_reload, args.no_gdnsd, Path(tdir), gdir)
File "utils/deploy-check.py", line 214, in deploy_check
safe_cmd([GDNSD_BIN, '-c', str(tdir), 'checkconf'])
File "utils/deploy-check.py", line 88, in safe_cmd
p_err.decode('utf-8')))
Exception: Command /usr/sbin/gdnsd -c /tmp/dns-check.kwc3e7kn checkconf failed with exit code 42, stderr:
info: gdnsd version 3.8.0 @ pid 7640
info: DNS listener threads (8 UDP + 8 TCP) configured for 208.80.154.238:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 208.80.153.231:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 91.198.174.239:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 198.35.27.27:53
info: DNS listener threads (8 TCP PROXY) configured for 127.0.0.1:535
info: DNS listener threads (1 UDP + 1 TCP) configured for 0.0.0.0:5353
info: DNS listener threads (1 UDP + 1 TCP) configured for [::]:5353
info: plugin_geoip: map 'generic-map': Loading GeoIP2 database '/tmp/dns-check.kwc3e7kn/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'generic-map' runtime db updated. nets: 1174886 dclists: 8
info: plugin_geoip: map 'checkdoh-map' runtime db updated. nets: 410 dclists: 2
info: plugin_geoip: map 'discovery-map': Loading GeoIP2 database '/tmp/dns-check.kwc3e7kn/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'discovery-map' runtime db updated. nets: 464 dclists: 2
info: admin_state: checking state file '/tmp/dns-check.kwc3e7kn/state/admin_state'...
error: CNAME 'pc1-master.eqiad.wmnet.' points to known same-zone NXDOMAIN 'pc2007.codfw.wmnet.'
fatal: Initial load of zone data failed
===== NODE GROUP =====
(1) authdns1001.wikimedia.org
----- OUTPUT of 'cd /srv/authdns/...nippets --deploy' -----
Assembling and testing data in /tmp/dns-check.5zr6cfw8
-- Generating zonefiles from zone templates
-- Processed 209 zones into directory /tmp/dns-check.5zr6cfw8/zones
OK: No tabs
Summary of violations:
W001|MISSING_IP_FOR_NAME_AND_PTR: 37
W002|MISSING_PTR_FOR_NAME_AND_IP: 26
W103|MISSING_MGMT_FOR_NAME: 64
W105|TOO_MANY_PUBLIC_NAMES: 3
RESULT: 0 Errors, 130 Warnings, 0 Ignored violations, 0 Ignored lines
-- Copying automatically generated zone files under target tree
-- Copying repo-driven real config files and admin_state
-- Copying puppetized config and GeoIP from /etc/gdnsd
-- Checking for illegal tabs in zonefiles
-- Running zone_validator to check WMF rules
-- Running /usr/sbin/gdnsd checkconf on /tmp/dns-check.5zr6cfw8
Traceback (most recent call last):
File "utils/deploy-check.py", line 276, in <module>
main()
File "utils/deploy-check.py", line 268, in main
deploy_check(args.deploy, args.skip_reload, args.no_gdnsd, Path(tdir), gdir)
File "utils/deploy-check.py", line 214, in deploy_check
safe_cmd([GDNSD_BIN, '-c', str(tdir), 'checkconf'])
File "utils/deploy-check.py", line 88, in safe_cmd
p_err.decode('utf-8')))
Exception: Command /usr/sbin/gdnsd -c /tmp/dns-check.5zr6cfw8 checkconf failed with exit code 42, stderr:
info: gdnsd version 3.8.0 @ pid 31278
info: DNS listener threads (8 UDP + 8 TCP) configured for 208.80.154.238:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 208.80.153.231:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 91.198.174.239:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 198.35.27.27:53
info: DNS listener threads (8 TCP PROXY) configured for 127.0.0.1:535
info: DNS listener threads (1 UDP + 1 TCP) configured for 0.0.0.0:5353
info: DNS listener threads (1 UDP + 1 TCP) configured for [::]:5353
info: plugin_geoip: map 'generic-map': Loading GeoIP2 database '/tmp/dns-check.5zr6cfw8/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'generic-map' runtime db updated. nets: 1174886 dclists: 8
info: plugin_geoip: map 'checkdoh-map' runtime db updated. nets: 410 dclists: 2
info: plugin_geoip: map 'discovery-map': Loading GeoIP2 database '/tmp/dns-check.5zr6cfw8/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'discovery-map' runtime db updated. nets: 464 dclists: 2
info: admin_state: checking state file '/tmp/dns-check.5zr6cfw8/state/admin_state'...
error: CNAME 'pc1-master.eqiad.wmnet.' points to known same-zone NXDOMAIN 'pc2007.codfw.wmnet.'
fatal: Initial load of zone data failed
===== NODE GROUP =====
(1) dns1001.wikimedia.org
----- OUTPUT of 'cd /srv/authdns/...nippets --deploy' -----
Assembling and testing data in /tmp/dns-check.sbj5ptsn
-- Generating zonefiles from zone templates
-- Processed 209 zones into directory /tmp/dns-check.sbj5ptsn/zones
OK: No tabs
Summary of violations:
W001|MISSING_IP_FOR_NAME_AND_PTR: 37
W002|MISSING_PTR_FOR_NAME_AND_IP: 26
W103|MISSING_MGMT_FOR_NAME: 64
W105|TOO_MANY_PUBLIC_NAMES: 3
RESULT: 0 Errors, 130 Warnings, 0 Ignored violations, 0 Ignored lines
-- Copying automatically generated zone files under target tree
-- Copying repo-driven real config files and admin_state
-- Copying puppetized config and GeoIP from /etc/gdnsd
-- Checking for illegal tabs in zonefiles
-- Running zone_validator to check WMF rules
-- Running /usr/sbin/gdnsd checkconf on /tmp/dns-check.sbj5ptsn
Traceback (most recent call last):
File "utils/deploy-check.py", line 276, in <module>
main()
File "utils/deploy-check.py", line 268, in main
deploy_check(args.deploy, args.skip_reload, args.no_gdnsd, Path(tdir), gdir)
File "utils/deploy-check.py", line 214, in deploy_check
safe_cmd([GDNSD_BIN, '-c', str(tdir), 'checkconf'])
File "utils/deploy-check.py", line 88, in safe_cmd
p_err.decode('utf-8')))
Exception: Command /usr/sbin/gdnsd -c /tmp/dns-check.sbj5ptsn checkconf failed with exit code 42, stderr:
info: gdnsd version 3.8.0 @ pid 22791
info: DNS listener threads (8 UDP + 8 TCP) configured for 208.80.154.238:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 208.80.153.231:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 91.198.174.239:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 198.35.27.27:53
info: DNS listener threads (8 TCP PROXY) configured for 127.0.0.1:535
info: DNS listener threads (1 UDP + 1 TCP) configured for 0.0.0.0:5353
info: DNS listener threads (1 UDP + 1 TCP) configured for [::]:5353
info: plugin_geoip: map 'generic-map': Loading GeoIP2 database '/tmp/dns-check.sbj5ptsn/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'generic-map' runtime db updated. nets: 1174886 dclists: 8
info: plugin_geoip: map 'checkdoh-map' runtime db updated. nets: 410 dclists: 2
info: plugin_geoip: map 'discovery-map': Loading GeoIP2 database '/tmp/dns-check.sbj5ptsn/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'discovery-map' runtime db updated. nets: 464 dclists: 2
info: admin_state: checking state file '/tmp/dns-check.sbj5ptsn/state/admin_state'...
error: CNAME 'pc1-master.eqiad.wmnet.' points to known same-zone NXDOMAIN 'pc2007.codfw.wmnet.'
info: Zone wikipediastats.net. with serial 2019120223 loaded
info: Zone wikipedia.lol. with serial 2019120223 loaded
info: Zone wikimemory.org. with serial 2019120223 loaded
info: Zone wikipediavideo.com. with serial 2019120223 loaded
info: Zone wilipedia.com. with serial 2019120223 loaded
info: Zone 27.35.198.in-addr.arpa. with serial 2020051916 loaded
info: Zone wikidata.pt. with serial 2019120223 loaded
info: Zone wikipaedia.net. with serial 2019120223 loaded
info: Zone wikipedia.es. with serial 2019120223 loaded
info: Zone wikiquote.org. with serial 2019120223 loaded
info: Zone wikpedia.org. with serial 2019120223 loaded
info: Zone wikipdia.org. with serial 2019120223 loaded
info: Zone wikivoyage.com. with serial 2019120223 loaded
info: Zone xn--80adgfman1aa4l.xn--p1ai. with serial 2019120223 loaded
info: Zone wikiartpedia.org. with serial 2019120223 loaded
info: Zone wikinews.de. with serial 2019120223 loaded
info: Zone wikifunctions.org. with serial 2021060120 loaded
info: Zone wikimediafoundation.net. with serial 2019120223 loaded
info: Zone donatetowikipedia.org. with serial 2019120223 loaded
info: Zone wikimobipedia.net. with serial 2019120223 loaded
info: Zone wikipedia.com. with serial 2019120223 loaded
info: Zone wikimedia.is. with serial 2019120223 loaded
info: Zone wikivoyage.de. with serial 2019120223 loaded
info: Zone wikimaps.net. with serial 2019120223 loaded
info: Zone wiktionary.pl. with serial 2019120223 loaded
info: Zone wikijunior.net. with serial 2019120223 loaded
info: Zone wiktionary.com. with serial 2019120223 loaded
info: Zone wikimaps.org. with serial 2019120223 loaded
info: Zone xn--80adgdym4pbd.xn--j1amh. with serial 2019120223 loaded
info: Zone wikimediastories.net. with serial 2019120223 loaded
info: Zone voyagewiki.org. with serial 2019120223 loaded
info: Zone wikimania.org. with serial 2019120223 loaded
info: Zone xn--b1aajamacm1dkmb.xn--p1ai. with serial 2019120223 loaded
info: Zone wikinews.org. with serial 2019120223 loaded
info: Zone wikimediacommons.info. with serial 2019120223 loaded
info: Zone wikiquote.com. with serial 2019120223 loaded
info: Zone 58.15.185.in-addr.arpa. with serial 2019120223 loaded
info: Zone visualwikipedia.net. with serial 2019120223 loaded
info: Zone wikiversity.pt. with serial 2019120223 loaded
info: Zone wikimediadns.org. with serial 2021051815 loaded
info: Zone wmftest.com. with serial 2020011000 loaded
info: Zone wmfusercontent.org. with serial 2020072316 loaded
info: Zone wikidisclosure.org. with serial 2019120223 loaded
info: Zone wikimedipedia.com. with serial 2019120223 loaded
info: Zone 174.198.91.in-addr.arpa. with serial 2020100816 loaded
info: Zone wikimediacloud.org. with serial 2021050615 loaded
info: Zone wikipedia.bg. with serial 2019120223 loaded
info: Zone wikiquote.pl. with serial 2019120223 loaded
info: Zone voyagewiki.com. with serial 2019120223 loaded
info: Zone wikipedia.ee. with serial 2019120223 loaded
info: Zone wikipedia.is. with serial 2019120223 loaded
info: Zone wikinews.pt. with serial 2019120223 loaded
info: Zone wikijunior.com. with serial 2019120223 loaded
info: Zone wikipediastories.com. with serial 2019120223 loaded
info: Zone uiquipedia.org. with serial 2019120223 loaded
info: Zone wikimediacommons.org. with serial 2019120223 loaded
info: Zone wikjpedia.org. with serial 2019120223 loaded
info: Zone wikipediazero.org. with serial 2019120223 loaded
info: Zone wikisource.org. with serial 2019120223 loaded
info: Zone wikipedia.gr. with serial 2019120223 loaded
info: Zone wikidisclosure.com. with serial 2019120223 loaded
info: Zone wikipediafoundation.org. with serial 2019120223 loaded
info: Zone wikimedia.cloud. with serial 2020061109 loaded
info: Zone wikipedia.fi. with serial 2019120223 loaded
info: Zone wikimedia.us. with serial 2019120223 loaded
info: Zone wikimediafoundation.org. with serial 2020071716 loaded
info: Zone wikipediastories.net. with serial 2019120223 loaded
info: Zone wikidata.es. with serial 2019120223 loaded
info: Zone wikivoyage-old.org. with serial 2019120223 loaded
info: Zone wikisource.pt. with serial 2019120223 loaded
info: Zone wikimedia.jp.net. with serial 2019120223 loaded
info: Zone wikimobipedia.com. with serial 2019120223 loaded
info: Zone wikimedia.xyz. with serial 2019120223 loaded
info: Zone 153.80.208.in-addr.arpa. with serial 2021051720 loaded
info: Zone wikitrends.org. with serial 2019120223 loaded
info: Zone wikipedia.org.il. with serial 2019120223 loaded
info: Zone wikifamily.com. with serial 2019120223 loaded
info: Zone wikipedia.info. with serial 2019120223 loaded
info: Zone wiki.voyage. with serial 2019120223 loaded
info: Zone wikiipedia.org. with serial 2019120223 loaded
info: Zone wikimediafoundation.info. with serial 2019120223 loaded
info: Zone wikiquotes.info. with serial 2019120223 loaded
info: Zone wikipediya.com. with serial 2019120223 loaded
info: Zone wkipedia.org. with serial 2019120223 loaded
info: Zone en-wp.org. with serial 2019120223 loaded
info: Zone wikifamily.org. with serial 2019120223 loaded
info: Zone wicipedia.com. with serial 2019120223 loaded
info: Zone wikiartpedia.co. with serial 2019120223 loaded
info: Zone wikipeedia.org. with serial 2019120223 loaded
info: Zone wikiartpedia.info. with serial 2019120223 loaded
info: Zone wekipedia.com. with serial 2019120223 loaded
info: Zone xn--wikirda-p0a62emr.pl. with serial 2020021417 loaded
info: Zone 10.in-addr.arpa. with serial 2021082316 loaded
info: Zone wwwwikimedia.com. with serial 2019120223 loaded
info: Zone wikimemory.net. with serial 2019120223 loaded
info: Zone 2.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa. with serial 2020100116 loaded
info: Zone 166.102.103.in-addr.arpa. with serial 2020100816 loaded
info: Zone 57.15.185.in-addr.arpa. with serial 2020102310 loaded
info: Zone donatetowikipedia.com. with serial 2019120223 loaded
info: Zone 59.15.185.in-addr.arpa. with serial 2020100116 loaded
info: Zone wikimediacommons.co.uk. with serial 2019120223 loaded
info: Zone vikipedia.com. with serial 2019120223 loaded
info: Zone w.wiki. with serial 2019120223 loaded
info: Zone wikimedia-dns.net. with serial 2021051815 loaded
fatal: Initial load of zone data failed
===== NODE GROUP =====
(1) dns1002.wikimedia.org
----- OUTPUT of 'cd /srv/authdns/...nippets --deploy' -----
Assembling and testing data in /tmp/dns-check.u091n0za
-- Generating zonefiles from zone templates
-- Processed 209 zones into directory /tmp/dns-check.u091n0za/zones
OK: No tabs
Summary of violations:
W001|MISSING_IP_FOR_NAME_AND_PTR: 37
W002|MISSING_PTR_FOR_NAME_AND_IP: 26
W103|MISSING_MGMT_FOR_NAME: 64
W105|TOO_MANY_PUBLIC_NAMES: 3
RESULT: 0 Errors, 130 Warnings, 0 Ignored violations, 0 Ignored lines
-- Copying automatically generated zone files under target tree
-- Copying repo-driven real config files and admin_state
-- Copying puppetized config and GeoIP from /etc/gdnsd
-- Checking for illegal tabs in zonefiles
-- Running zone_validator to check WMF rules
-- Running /usr/sbin/gdnsd checkconf on /tmp/dns-check.u091n0za
Traceback (most recent call last):
File "utils/deploy-check.py", line 276, in <module>
main()
File "utils/deploy-check.py", line 268, in main
deploy_check(args.deploy, args.skip_reload, args.no_gdnsd, Path(tdir), gdir)
File "utils/deploy-check.py", line 214, in deploy_check
safe_cmd([GDNSD_BIN, '-c', str(tdir), 'checkconf'])
File "utils/deploy-check.py", line 88, in safe_cmd
p_err.decode('utf-8')))
Exception: Command /usr/sbin/gdnsd -c /tmp/dns-check.u091n0za checkconf failed with exit code 42, stderr:
info: gdnsd version 3.8.0 @ pid 15511
info: DNS listener threads (8 UDP + 8 TCP) configured for 208.80.154.238:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 208.80.153.231:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 91.198.174.239:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 198.35.27.27:53
info: DNS listener threads (8 TCP PROXY) configured for 127.0.0.1:535
info: DNS listener threads (1 UDP + 1 TCP) configured for 0.0.0.0:5353
info: DNS listener threads (1 UDP + 1 TCP) configured for [::]:5353
info: plugin_geoip: map 'generic-map': Loading GeoIP2 database '/tmp/dns-check.u091n0za/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'generic-map' runtime db updated. nets: 1174886 dclists: 8
info: plugin_geoip: map 'checkdoh-map' runtime db updated. nets: 410 dclists: 2
info: plugin_geoip: map 'discovery-map': Loading GeoIP2 database '/tmp/dns-check.u091n0za/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-02 13:16:49 UTC
info: plugin_geoip: map 'discovery-map' runtime db updated. nets: 464 dclists: 2
info: admin_state: checking state file '/tmp/dns-check.u091n0za/state/admin_state'...
error: CNAME 'pc1-master.eqiad.wmnet.' points to known same-zone NXDOMAIN 'pc2007.codfw.wmnet.'
fatal: Initial load of zone data failed
================
PASS | | 0% (0/12) [00:36<?, ?hosts/s]
FAIL |█████████████████████████████████████████████████████████████████████████████| 100% (12/12) [00:36<00:00, 1.66s/hosts]
100.0% (12/12) of nodes failed to execute command 'cd /srv/authdns/...nippets --deploy': authdns[1001,2001].wikimedia.org,dns[1001-1002,2001-2002,3001-3002,4001-4002,5001-5002].wikimedia.org
0.0% (0/12) success ratio (< 100.0% threshold) for command: 'cd /srv/authdns/...nippets --deploy'. Aborting.
0.0% (0/12) success ratio (< 100.0% threshold) of nodes successfully executed all commands. Aborting.
Failed to run the sre.dns.netbox cookbook
Traceback (most recent call last):
File "/srv/deployment/spicerack/cookbooks/sre/hosts/decommission.py", line 409, in run
dns_netbox_run(dns_netbox_args, self.spicerack)
File "/srv/deployment/spicerack/cookbooks/sre/dns/netbox.py", line 129, in run
git=AUTHDNS_DNS_CHECKOUT_PATH, netbox=AUTHDNS_NETBOX_CHECKOUT_PATH))
File "/usr/lib/python3/dist-packages/spicerack/remote.py", line 477, in run_sync
is_safe=is_safe,
File "/usr/lib/python3/dist-packages/spicerack/remote.py", line 668, in _execute
raise RemoteExecutionError(ret, "Cumin execution failed")
spicerack.remote.RemoteExecutionError: Cumin execution failed (exit_code=2)
**Failed to run the sre.dns.netbox cookbook**: Cumin execution failed (exit_code=2)
Running Homer on asw-a-codfw.mgmt.codfw.wmnet, it takes time ⏳ don't worry
INFO:homer.devices:Initialized 39 devices
INFO:homer:Committing config for query asw-a-codfw.mgmt.codfw.wmnet with message: Host decommission - marostegui@cumin1001 - T289112
INFO:homer:Gathering global Netbox data
INFO:homer.devices:Matched 1 device(s) for query 'asw-a-codfw.mgmt.codfw.wmnet'
INFO:homer:Generating configuration for asw-a-codfw.mgmt.codfw.wmnet
Configuration diff for asw-a-codfw.mgmt.codfw.wmnet:
[edit interfaces interface-range disabled]
+ member ge-1/0/1;
member ge-1/0/5 { ... }
[edit interfaces interface-range vlan-private1-a-codfw]
- member ge-1/0/1;
[edit interfaces]
- ge-1/0/1 {
- description pc2007;
- }
Type "yes" to commit, "no" to abort.
> yes
INFO:homer.transports.junos:Committing the configuration on asw-a-codfw.mgmt.codfw.wmnet
INFO:homer:Homer run completed successfully on 1 devices: ['asw-a-codfw.mgmt.codfw.wmnet']
ERROR: some step failed, check the task updates.
Updated Phabricator task T289112
END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) for hosts pc2007.codfw.wmnet
[05:30:29] marostegui@cumin1001:~$
PASS | | 0% (0/12) [00:36<?, ?hosts/s]
FAIL |█████████████████████████████████████████████████████████████████████████████| 100% (12/12) [00:36<00:00, 1.66s/hosts]
100.0% (12/12) of nodes failed to execute command 'cd /srv/authdns/...nippets --deploy': authdns[1001,2001].wikimedia.org,dns[1001-1002,2001-2002,3001-3002,4001-4002,5001-5002].wikimedia.org
0.0% (0/12) success ratio (< 100.0% threshold) for command: 'cd /srv/authdns/...nippets --deploy'. Aborting.
0.0% (0/12) success ratio (< 100.0% threshold) of nodes successfully executed all commands. Aborting.
Failed to run the sre.dns.netbox cookbook
Traceback (most recent call last):
File "/srv/deployment/spicerack/cookbooks/sre/hosts/decommission.py", line 409, in run
dns_netbox_run(dns_netbox_args, self.spicerack)
File "/srv/deployment/spicerack/cookbooks/sre/dns/netbox.py", line 129, in run
git=AUTHDNS_DNS_CHECKOUT_PATH, netbox=AUTHDNS_NETBOX_CHECKOUT_PATH))
File "/usr/lib/python3/dist-packages/spicerack/remote.py", line 477, in run_sync
is_safe=is_safe,
File "/usr/lib/python3/dist-packages/spicerack/remote.py", line 668, in _execute
raise RemoteExecutionError(ret, "Cumin execution failed")
spicerack.remote.RemoteExecutionError: Cumin execution failed (exit_code=2)
**Failed to run the sre.dns.netbox cookbook**: Cumin execution failed (exit_code=2)
Running Homer on asw-a-codfw.mgmt.codfw.wmnet, it takes time ⏳ don't worry
INFO:homer.devices:Initialized 39 devices
INFO:homer:Committing config for query asw-a-codfw.mgmt.codfw.wmnet with message: Host decommission - marostegui@cumin1001 - T289112
INFO:homer:Gathering global Netbox data
INFO:homer.devices:Matched 1 device(s) for query 'asw-a-codfw.mgmt.codfw.wmnet'
INFO:homer:Generating configuration for asw-a-codfw.mgmt.codfw.wmnet
Configuration diff for asw-a-codfw.mgmt.codfw.wmnet:
[edit interfaces interface-range disabled]
+ member ge-1/0/1;
member ge-1/0/5 { ... }
[edit interfaces interface-range vlan-private1-a-codfw]
- member ge-1/0/1;
[edit interfaces]
- ge-1/0/1 {
- description pc2007;
- }
Type "yes" to commit, "no" to abort.
> yes
INFO:homer.transports.junos:Committing the configuration on asw-a-codfw.mgmt.codfw.wmnet
INFO:homer:Homer run completed successfully on 1 devices: ['asw-a-codfw.mgmt.codfw.wmnet']
ERROR: some step failed, check the task updates.
Updated Phabricator task T289112
END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) for hosts pc2007.codfw.wmnet