There are a couple of problems with the IRC notifications created by WebPageTest jobs in Jenkins.
The idea from the beginning was to know if the jobs failed: catch a bad push or when WebPageTest stopped working a a couple of weeks ago when our agent freezed and couldn't be contacted by the server.
However we also get notifications when the WebPageTest.org test with real 2G/3G connections fails (and that happens quite often) and when the test environment has content that returns 400.
I think we could have IRC notifications turned on only for our own instance (so we know that it works) and that we only test URL in productions that have the right content.
Lets sync tonight at our meeting.