With lint=true in the VE API, this page takes ~2.7s to load
https://en.wikipedia.org/w/api.php?action=visualeditor&format=json&paction=metadata&page=Talk%3ADementia_with_Lewy_bodies&uselang=en-gb&lint=true&formatversion=2&oldid=956794424
without it it takes only ~300ms
https://en.wikipedia.org/w/api.php?action=visualeditor&format=json&paction=metadata&page=Talk%3ADementia_with_Lewy_bodies&uselang=en-gb&formatversion=2&oldid=956794424
Fetching the linting data was our solution to T246481.
This longer page (village pump technical) takes ~3.7s: https://en.wikipedia.org/w/api.php?action=visualeditor&format=json&paction=metadata&page=Wikipedia%3AVillage_pump_(technical)&uselang=en-gb&lint=true&formatversion=2&oldid=959179040
A nearly empty page took 400ms.
That would suggest that the time is spent doing the linting, and the result is not being cached.
CC @cscott
I get similar timings talking to RESTBase directly: https://en.wikipedia.org/api/rest_v1/page/lint/Wikipedia%3AVillage_pump_(technical)/959179040
So this definitely looks like a caching issue on the RESTBase side.
@ssastry please correct me if I'm wrong, but Linter extension stores Parsoid lints in MySQL. If I understand it well, the flow is the following: page edit -> parsoid rerender -> parsoid calling MW API to record lint errors -> API is scheduling a job -> errors are stored in the database. So, VE doesn't need to call RESTBase to re-render the lint errors, it can just execute an action API query if working from the client side, or depend on Linter extension and get the errors from linter table directly.
Quick test on beta reveals the results are equivalent:
curl 'https://en.wikipedia.beta.wmflabs.org/w/api.php?action=query&list=linterrors&lntpageid=1&format=json'
{"batchcomplete":"","query":{"linterrors":[{"pageid":1,"ns":0,"title":"Main Page","lintId":16114,"category":"bogus-image-options","location":[3862,3877],"templateInfo":{"name":"Template:In_the_news"},"params":{"items":[""]}}]}}
curl -XPOST -H 'Content-Type: application/json' -H 'Host: en.wikipedia.beta.wmflabs.org' --data '{}' http://parsoid-external-ci-access.beta.wmflabs.org/w/rest.php/en.wikipedia.beta.wmflabs.org/v3/transform/wikitext/to/lint/Main_Page/416077
[{"type":"bogus-image-options","dsr":[3862,3877,null,null],"templateInfo":{"name":"Template:In_the_news"},"params":{"items":[""]}}]In general, we're trying to replace RESTBase caching with other means of caching for Parsoid since it's moving to core, so I would like to avoid adding more caching into RESTBase
Change 599418 had a related patch set uploaded (by Esanders; owner: Esanders):
[mediawiki/extensions/VisualEditor@master] Fetch lint errors from MW API, not RESTBase
Change 599420 had a related patch set uploaded (by Esanders; owner: Esanders):
[mediawiki/extensions/DiscussionTools@master] Use new lint API
Change 602182 had a related patch set uploaded (by Esanders; owner: Esanders):
[mediawiki/extensions/DiscussionTools@master] Call the linter API directly if installed
Change 599420 abandoned by Bartosz Dziewoński:
Use new lint API
Reason:
Doing https://gerrit.wikimedia.org/r/602182 instead
Change 602182 merged by jenkins-bot:
[mediawiki/extensions/DiscussionTools@master] Call the linter API directly if installed
Change 599418 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@master] Drop linting from the VE API