Page MenuHomePhabricator
Paste P30535

pynetbox threading
ActivePublic

Authored by ayounsi on Jun 28 2022, 8:47 AM.
Tags
None
Referenced Files
F35283084: pynetbox threading
Jun 28 2022, 8:47 AM
Subscribers
None
ayounsi@netbox-dev2002:~$ time /srv/deployment/netbox-extras/dns/generate_dns_snippets.py -v commit --icinga-check "icinga-check"
2022-06-28 08:35:42,798 [ERROR] /srv/deployment/netbox-extras/dns/generate_dns_snippets.py:655 Failed to read Icinga state from /var/run/netbox_generate_dns_snippets.state: [Errno 2] No such file or directory: '/var/run/netbox_generate_dns_snippets.state'
2022-06-28 08:35:42,799 [INFO] /srv/deployment/netbox-extras/dns/generate_dns_snippets.py:148 Gathering devices, interfaces, addresses and prefixes from Netbox
2022-06-28 08:35:42,801 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:973 Starting new HTTPS connection (1): netbox-next.wikimedia.org:443
2022-06-28 08:35:43,067 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/ipam/ip-addresses/?status=active HTTP/1.1" 200 29777
2022-06-28 08:35:47,574 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/ipam/ip-addresses/?status=active&limit=7439&offset=50 HTTP/1.1" 200 814970
2022-06-28 08:35:52,473 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/ipam/ip-addresses/?limit=1000&offset=1050&status=active HTTP/1.1" 200 845440
2022-06-28 08:35:56,915 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/ipam/ip-addresses/?limit=1000&offset=2050&status=active HTTP/1.1" 200 831506
2022-06-28 08:36:01,441 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/ipam/ip-addresses/?limit=1000&offset=3050&status=active HTTP/1.1" 200 836596
2022-06-28 08:36:06,021 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/ipam/ip-addresses/?limit=1000&offset=4050&status=active HTTP/1.1" 200 830870
2022-06-28 08:36:11,334 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/ipam/ip-addresses/?limit=1000&offset=5050&status=active HTTP/1.1" 200 854477
2022-06-28 08:36:16,047 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/ipam/ip-addresses/?limit=1000&offset=6050&status=active HTTP/1.1" 200 857400
2022-06-28 08:36:18,009 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/ipam/ip-addresses/?limit=1000&offset=7050&status=active HTTP/1.1" 200 333294
2022-06-28 08:36:19,789 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/ HTTP/1.1" 200 59475
2022-06-28 08:36:24,235 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?limit=15946&offset=50 HTTP/1.1" 200 1333793
2022-06-28 08:36:30,933 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?limit=1000&offset=1050 HTTP/1.1" 200 1630348
2022-06-28 08:36:37,277 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?limit=1000&offset=2050 HTTP/1.1" 200 1620694
2022-06-28 08:36:42,722 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?limit=1000&offset=3050 HTTP/1.1" 200 1467671
2022-06-28 08:36:48,038 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?limit=1000&offset=4050 HTTP/1.1" 200 1389995
2022-06-28 08:36:51,761 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?limit=1000&offset=5050 HTTP/1.1" 200 1223879
2022-06-28 08:36:55,678 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?limit=1000&offset=6050 HTTP/1.1" 200 1196143
2022-06-28 08:36:59,486 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?limit=1000&offset=7050 HTTP/1.1" 200 1176453
2022-06-28 08:37:03,788 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?limit=1000&offset=8050 HTTP/1.1" 200 1251809
2022-06-28 08:37:08,076 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?limit=1000&offset=9050 HTTP/1.1" 200 1231642
2022-06-28 08:37:10,585 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?limit=1000&offset=10050 HTTP/1.1" 200 1044202
2022-06-28 08:37:12,936 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?limit=1000&offset=11050 HTTP/1.1" 200 1044483
2022-06-28 08:37:15,447 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?limit=1000&offset=12050 HTTP/1.1" 200 1046171
2022-06-28 08:37:18,950 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?limit=1000&offset=13050 HTTP/1.1" 200 1171196
2022-06-28 08:37:22,712 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?limit=1000&offset=14050 HTTP/1.1" 200 1198189
2022-06-28 08:37:26,358 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?limit=1000&offset=15050 HTTP/1.1" 200 1060690
2022-06-28 08:37:30,023 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/virtualization/interfaces/ HTTP/1.1" 200 29166
2022-06-28 08:37:30,445 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/virtualization/interfaces/?limit=261&offset=50 HTTP/1.1" 200 122764
2022-06-28 08:37:30,624 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/ipam/prefixes/ HTTP/1.1" 200 34342
2022-06-28 08:37:31,077 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/ipam/prefixes/?limit=474&offset=50 HTTP/1.1" 200 284492
2022-06-28 08:37:31,713 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/devices/?status=active&status=planned&status=staged&status=failed&status=inventory&status=decommissioning HTTP/1.1" 200 105398
2022-06-28 08:37:34,562 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/devices/?status=active&status=planned&status=staged&status=failed&status=inventory&status=decommissioning&limit=2421&offset=50 HTTP/1.1" 200 2022656
2022-06-28 08:37:37,049 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/devices/?limit=1000&offset=1050&status=active&status=planned&status=staged&status=failed&status=inventory&status=decommissioning HTTP/1.1" 200 2024920
2022-06-28 08:37:39,698 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/devices/?limit=1000&offset=2050&status=active&status=planned&status=staged&status=failed&status=inventory&status=decommissioning HTTP/1.1" 200 690082
[...]
2022-06-28 08:37:41,278 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/virtualization/virtual-machines/ HTTP/1.1" 200 74472
2022-06-28 08:37:42,035 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/virtualization/virtual-machines/?limit=259&offset=50 HTTP/1.1" 200 310980
2022-06-28 08:37:42,130 [ERROR] /srv/deployment/netbox-extras/dns/generate_dns_snippets.py:173 Unsupported assigned object type: ipam.fhrpgroup
[...]
2022-06-28 08:37:51,817 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:973 Starting new HTTPS connection (1): netbox-next.wikimedia.org:443
2022-06-28 08:37:51,874 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/extras/object-changes/?time_after=2022-06-28+08%3A07%3A51.815910 HTTP/1.1" 200 52
[...]
real 2m9.411s
user 0m14.902s
sys 0m0.847s
=====================
ayounsi@netbox-dev2002:~$ time /srv/deployment/netbox-extras/dns/generate_dns_snippets.py -v commit --icinga-check "icinga-check"
2022-06-28 08:39:44,240 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:973 Starting new HTTPS connection (1): netbox-next.wikimedia.org:443
2022-06-28 08:39:50,130 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/ipam/ip-addresses/?status=active&limit=0 HTTP/1.1" 200 802360
2022-06-28 08:39:50,160 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:973 Starting new HTTPS connection (2): netbox-next.wikimedia.org:443
2022-06-28 08:39:50,164 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:973 Starting new HTTPS connection (3): netbox-next.wikimedia.org:443
2022-06-28 08:39:50,174 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:973 Starting new HTTPS connection (4): netbox-next.wikimedia.org:443
2022-06-28 08:39:54,579 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/ipam/ip-addresses/?status=active&offset=1000&limit=1000 HTTP/1.1" 200 845621
2022-06-28 08:39:55,098 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/ipam/ip-addresses/?status=active&offset=3000&limit=1000 HTTP/1.1" 200 836719
2022-06-28 08:39:58,466 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/ipam/ip-addresses/?status=active&offset=4000&limit=1000 HTTP/1.1" 200 829758
2022-06-28 08:40:00,502 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/ipam/ip-addresses/?status=active&offset=2000&limit=1000 HTTP/1.1" 200 831853
2022-06-28 08:40:02,899 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/ipam/ip-addresses/?status=active&offset=5000&limit=1000 HTTP/1.1" 200 854450
2022-06-28 08:40:04,969 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/ipam/ip-addresses/?status=active&offset=7000&limit=1000 HTTP/1.1" 200 377338
2022-06-28 08:40:05,302 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/ipam/ip-addresses/?status=active&offset=6000&limit=1000 HTTP/1.1" 200 856001
2022-06-28 08:40:12,384 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?limit=0 HTTP/1.1" 200 1301453
2022-06-28 08:40:12,541 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:272 Resetting dropped connection: netbox-next.wikimedia.org
2022-06-28 08:40:12,545 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:272 Resetting dropped connection: netbox-next.wikimedia.org
2022-06-28 08:40:12,561 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:272 Resetting dropped connection: netbox-next.wikimedia.org
2022-06-28 08:40:18,880 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?offset=2000&limit=1000 HTTP/1.1" 200 1626900
2022-06-28 08:40:19,470 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?offset=1000&limit=1000 HTTP/1.1" 200 1636174
2022-06-28 08:40:23,605 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?offset=4000&limit=1000 HTTP/1.1" 200 1387417
2022-06-28 08:40:26,209 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?offset=3000&limit=1000 HTTP/1.1" 200 1479487
2022-06-28 08:40:27,469 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?offset=5000&limit=1000 HTTP/1.1" 200 1235212
2022-06-28 08:40:30,657 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?offset=6000&limit=1000 HTTP/1.1" 200 1194842
2022-06-28 08:40:30,841 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?offset=7000&limit=1000 HTTP/1.1" 200 1172883
2022-06-28 08:40:34,895 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?offset=9000&limit=1000 HTTP/1.1" 200 1241527
2022-06-28 08:40:35,644 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?offset=8000&limit=1000 HTTP/1.1" 200 1253120
2022-06-28 08:40:37,159 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?offset=10000&limit=1000 HTTP/1.1" 200 1044726
2022-06-28 08:40:38,655 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?offset=11000&limit=1000 HTTP/1.1" 200 1044441
2022-06-28 08:40:39,503 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?offset=12000&limit=1000 HTTP/1.1" 200 1045453
2022-06-28 08:40:42,720 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?offset=13000&limit=1000 HTTP/1.1" 200 1163827
2022-06-28 08:40:43,069 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?offset=14000&limit=1000 HTTP/1.1" 200 1199744
2022-06-28 08:40:46,562 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/interfaces/?offset=15000&limit=1000 HTTP/1.1" 200 1119428
2022-06-28 08:40:51,734 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/virtualization/interfaces/?limit=0 HTTP/1.1" 200 151724
2022-06-28 08:40:52,269 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/ipam/prefixes/?limit=0 HTTP/1.1" 200 318652
2022-06-28 08:40:54,928 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/devices/?status=active&status=planned&status=staged&status=failed&status=inventory&status=decommissioning&limit=0 HTTP/1.1" 200 2032172
2022-06-28 08:40:54,971 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:272 Resetting dropped connection: netbox-next.wikimedia.org
2022-06-28 08:40:56,193 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/devices/?status=active&status=planned&status=staged&status=failed&status=inventory&status=decommissioning&offset=2000&limit=1000 HTTP/1.1" 200 791553
2022-06-28 08:40:57,897 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/devices/?status=active&status=planned&status=staged&status=failed&status=inventory&status=decommissioning&offset=1000&limit=1000 HTTP/1.1" 200 2018945
[...]
2022-06-28 08:40:59,889 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/virtualization/virtual-machines/?limit=0 HTTP/1.1" 200 385234[...]
2022-06-28 08:41:05,906 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:272 Resetting dropped connection: netbox-next.wikimedia.org
2022-06-28 08:41:06,129 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/devices/2717/ HTTP/1.1" 200 2148
2022-06-28 08:41:06,360 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/devices/3572/ HTTP/1.1" 200 2175
2022-06-28 08:41:06,705 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/devices/4133/ HTTP/1.1" 200 2162
2022-06-28 08:41:06,934 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/devices/4125/ HTTP/1.1" 200 2203
2022-06-28 08:41:07,273 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/devices/2305/ HTTP/1.1" 200 2029
2022-06-28 08:41:07,546 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/dcim/devices/4124/ HTTP/1.1" 200 2198
[...]
2022-06-28 08:41:10,100 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:973 Starting new HTTPS connection (1): netbox-next.wikimedia.org:443
2022-06-28 08:41:10,194 [DEBUG] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:452 https://netbox-next.wikimedia.org:443 "GET /api/extras/object-changes/?time_after=2022-06-28+08%3A11%3A10.099049&limit=0 HTTP/1.1" 200 52
real 1m26.328s
user 0m16.770s
sys 0m0.761s