What about test this out by purging 3-4 URLs once an hour in a crontab? Then all those URLs will be affected at the same time, so its easier to see. Not perfect as run it direct before the test, but simple to implement.

Yes, that could work. A one hour window would already be a huge improvement in terms of time correlation.

I added a job in the crontab on gpsi.webperf.eqiad.wmflabs where we purge Barack_Obama, Facebook and Sweden once an hour. I added so we purges both on en.wiki and m.en.wiki, do you know @Gilles if that's needed?

Not necessary, purging either desktop or mobile purges both variants.

Did you set it up on Beta as well? On Beta natural traffic isn't enough to ensure re-parsing before the next test run, so in addition to the purge POST request you need to make a GET request to the article right after (still from the cron job).

Yep I added https://en.m.wikipedia.beta.wmflabs.org/wiki/Barack_Obama I think we only test one article there.

Yep first we d a curl -X POST and then a get to each article.