Despite https://gerrit.wikimedia.org/r/344504 having been deployed over 6 hours ago, I still see a handful of requests per hour to XHGui from app servers. At least from mw1283, mw1220, mw1194, and mw1280.
I verified on each of these servers that the patch is applied there, meaning the code submitting is now in an IF condition only true for X-Wikimedia-Debug requests. From the live php stacktraces I see in logstash I also verified it is indeed coming from /srv/mediawiki/wmf-config/StartProfiler.php line 120 (e.g. not from somewhere else).
This is a problem because tungsten is overloaded and some of the handful of profiles being sent actually end up failing to go through, which then results in an exception being thrown (either fails to connect, or connects but query times out).
It shouldn't affect end-users as far as I know since this all happens in a shutdown function, but still - tungsten is no longer meant to be involved in non-debug requests.
mw1283 WARNING /w/api.php?format=xml&rawcontinue=1&maxlag=5&action=query&prop=langlinks&... PHP Warning: failed to connect to 10.64.0.22 : Connection refused mw1220 ERROR /w/index.php?title=MediaWiki%3AGadget...&action=raw&.. MongoCursorTimeoutException from line 237 of /srv/mediawiki/multiversion/vendor/mongofill...Timed out waiting for data `