I added a filter term (disabled by default) on the Squid proxy dashboard to surface traffic to any *.wikimedia.org or *.wikipedia.org domain:
https://logstash.wikimedia.org/app/dashboards#/view/58c908a0-a394-11ec-bf8e-43f1807d5bc2
Any queries to those two domains (and more) doesn't need to (and shouldn't) go through the Squid proxies are they're internal hosts. See doc on https://wikitech.wikimedia.org/wiki/HTTP_proxy#How-to?
Longer term plans might be to block such traffic flows to prevent configuration mistake at their creation.
Here are the largest offending hosts relevant to Infrastructure Foundations in the last 24h:
build2001.codfw.wmnet.
Various UA:
Debian APT-HTTP/1.3 (1.8.2.3) 1,397
Debian APT-HTTP/1.3 (2.2.4) 526
Debian APT-HTTP/1.3 (1.4.11) 291
Debian APT-HTTP/1.3 (1.8.2.1) 113
Debian APT-HTTP/1.3 (1.8.2.2) 28
Two destinations:
mirrors.wikimedia.org 2,070
apt.wikimedia.org 285
After a chat with @MoritzMuehlenhoff they are from the pbuilder environments
It's not clear how they learn the proxy config, but we should be able to configure the DIRECT keyword for those two domains. See more information about DIRECT in https://www.claudiokuenzler.com/blog/619/apt-behind-proxy-no-proxy-for-some-repositories
Similarly, deploy1002.eqiad.wmnet have calls to the same two destinations.
Lat pattern, the host not relevant as they're provisioning hosts, but UA set to "debian-installer"
Fetching data from:
mirrors.wikimedia.org 162
apt.wikimedia.org 2