Page MenuHomePhabricator

RecentChanges: Improve performance of Special:RecentChanges rendering
Open, Needs TriagePublic

Description

At Fandom, we allow users to use limit above default 500 on RecentChanges pages. Our default $wgRCLinkLimits is set to

$wgRCLinkLimits = [ 50, 100, 250, 500, 1000, 2000, 3000, 4000 ];

Over the past few MW versions, we saw slight degradation of load times for this page and we needed to lower the limit to fit into the 15s timeout that we have set for all requests. With our investigation with the profiler, it turned out that most of the time for the process is spent on rendering the same "UserLinks" and "Tags", as this is often the case that few active users will make most of the changes or the same tag is visible on RC record. This can be easily and safely optimized with the process cache (eg. MapCacheLRU), used inside the RC rendering.

Below are (not very scientific) response times before and after, we applied the optimisation on the 1.39 version of MW.
For the simple test, we have used popular wiki, where we were varying the limit value:

and here are the results:

Before

### 1000 Records
### 6171ms
### 7335ms
### 6789ms

### 2000 Records
### 12410ms
### 12951ms
### 12976ms

### 3000 Records
### No results means above 15seconds, which looking at jump 
### on 1000 to 2000 would finish in ~18seconds

After

### 1000 Records
### 4072ms
### 4021ms
### 3826ms
### 4150ms

### 2000 Records
### 6906ms
### 6644ms
### 6511ms

### 3000 Records
### 9108ms
### 9564ms

### 4000 Records
### 11349ms
### 11763ms

### 5000 Records
### 14555ms
### 14870ms

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Change 936221 had a related patch set uploaded (by Ttomalak; author: Ttomalak):

[mediawiki/core@master] RecentChanges: Add process caching to RecentChanges UserLinks and Tags parsing

https://gerrit.wikimedia.org/r/936221

Change 938220 had a related patch set uploaded (by TK-999; author: TK-999):

[mediawiki/core@master] PermissionManager: Avoid restrictions lookup for unsupported actions

https://gerrit.wikimedia.org/r/938220

Change 936221 merged by jenkins-bot:

[mediawiki/core@master] RecentChanges: Add process caching to RecentChanges UserLinks and Tags parsing

https://gerrit.wikimedia.org/r/936221

Change 940177 had a related patch set uploaded (by Ttomalak; author: Ttomalak):

[mediawiki/extensions/CheckUser@master] Fix revId fetching on RecentChange page for IPReveal

https://gerrit.wikimedia.org/r/940177

Change 940177 merged by jenkins-bot:

[mediawiki/extensions/CheckUser@master] Fix data-mw-revid attribute read on RecentChanges page for IPReveal

https://gerrit.wikimedia.org/r/940177

Change 950805 had a related patch set uploaded (by Matěj Suchánek; author: Matěj Suchánek):

[mediawiki/core@master] ChangesList: Cache user links

https://gerrit.wikimedia.org/r/950805

Change 950805 merged by jenkins-bot:

[mediawiki/core@master] ChangesList: Cache user links

https://gerrit.wikimedia.org/r/950805

@matej_suchanek Profiling action=history shows a similar cost from ChangeTags::tagDescription that's still there via ChangeTags::formatSummaryRow and HistoryPager.

https://performance.wikimedia.org/excimer/profile/81ec3b90d675bced profiled https://www.mediawiki.org/w/index.php?action=history&title=Developers/Maintainers

Perhaps there's potential to generize that one as well, since that's the same entry point we ended up caching in ChangesList.php for calls from Special:RecentChanges.

... and Special:NewPages (profile) and Special:Log probably too.

Change 962172 had a related patch set uploaded (by Krinkle; author: Matěj Suchánek):

[mediawiki/core@master] Put tag parsing behind an in-process cache in more views

https://gerrit.wikimedia.org/r/962172

Change 969505 had a related patch set uploaded (by Matěj Suchánek; author: Matěj Suchánek):

[mediawiki/core@master] Move tag summary cache to a trait and use it in more classes

https://gerrit.wikimedia.org/r/969505

Change 962172 merged by jenkins-bot:

[mediawiki/core@master] Put tag parsing behind an in-process cache in more views

https://gerrit.wikimedia.org/r/962172

Change 938220 had a related patch set uploaded (by TK-999; author: TK-999):

[mediawiki/core@master] PermissionManager: Avoid restrictions lookup for unsupported actions

https://gerrit.wikimedia.org/r/938220

@TK-999 shared a Wikia profile at:
https://www.speedscope.app/#profileURL=https%3A%2F%2Fgist.githubusercontent.com%2Fmszabo-wikia%2Fba0af0af63cde7dd9cc289672a58d923%2Fraw%2Facd6281072d5dc1cd58b7319a6e69b0e6a75d7c5%2Frecentchanges.speedscope.json

Where the total request time (unknown RC limit, I'm guessing 50 or 100) was 6 seconds of which 217ms spent in RestrictionStore::getRestrictions via User::probablyCan. Definitely adds up quite a bit!

https://en.wikipedia.org/wiki/Special:Recentchanges with default settings, limit=500.

  • Total (wall time): 8 seconds.
  • RestrictionStore::getRestrictions: 518 ms (0.5s).

Note that I captured this profile, after an unprofiled warmup of the same request, for the same wiki, to the same mwdebug server.

https://performance.wikimedia.org/excimer/profile/b7f350519cfec7f4

Screenshot 2024-04-11 at 21.48.23.png (815×1 px, 146 KB)

Change #938220 merged by jenkins-bot:

[mediawiki/core@master] Permissions: Avoid RestrictionStore lookup for unsupported actions

https://gerrit.wikimedia.org/r/938220