Hi @adrelanos thank you for creating the issue! I think this is because of a the new Lighthouse version and new "headless" mode in Chrome, let me walk you through why I think that.

We do run some continuous monitoring of mediawiki.org, you can see that data in Grafana here:

https://grafana.wikimedia.org/d/IvAfnmLMk/page-drilldown?forceLogin&orgId=1&var-base=sitespeed_io&var-browser=chrome&var-connectivity=4g&var-function=median&var-group=m_mediawiki_org&var-page=_wiki_MediaWiki&var-path=emulatedMobile&var-s3path=https:%2F%2Fsynthetic-tests-result-wikimedia.s3.amazonaws.com&var-testtype=firstView

That test is using Chrome in emulated mobile mode (we do not test MediaWiki.org on our mobile phone device lab) but I think that is good enough.

Looking at TBT over time it looks like this:

We can see an increase in May here but not of the magnitude you are describing. Digging deeper into the May increase I could see we spend more time in evaluating JavaScript (we ship more JS):

And then look at the actual JS change over time:

So we started to ship more JS in May,

Back to the issue. I think the numbers we used to get out of Lighthouse on Pagespeed was wrong, you can see that the TBT more matches our own metrics (we don't test with headless mode). Lighthouse 10 was released in Feb but it usually takes some time until it's shipped on Pagespeed. At the same time the new headless mode was introduced. The new headless mode means that now Chrome actually use the real Chrome version. Old versions shipped another Chrome binary for headless and it seems like the old version didn't simulate Chrome good enough.

I'll try to summarise: The good thing is that Google now seems to run a more realistic test case, highlighting that we push too much JavaScript, especially to slow devices. I've got an upcoming blog post I hope to share soon about that.

I suspect this may be due to T336020: DiscussionTools loads 1.7MB of VisualEditor JS (350kB transfer) on mediawiki.org Main Page, which was also specific to mediawiki.org.

Let's take another look after the above is fixed.

I am late to notice this but this screenshot shows the Desktop tab of the pagespeed result. This is because Desktop is underlined with a blue line. It doesn't show the Mobile tab.