Page MenuHomePhabricator

Varnish 503 errors on page with large number of flag icons.
Open, LowPublic

Description

In this enwiki Village Pump thread, a user reports getting

Request from 2a02:c7d:c50:c400:882e:902e:b6a:2ba7 via cp3056 cp3056, Varnish XID 987116681 Error: 503, Backend fetch failed at Thu, 12 Nov 2020 07:04:32 GMT.

errors when editing List of shipwrecks in November 1869 and several other similarly large pages with lots of flag icons. Editing individual sections instead of the whole page at once apparently works (but is not a viable workaround for their needs).

Possibly related: T248796

Event Timeline

Joe triaged this task as High priority.Nov 12 2020, 4:54 PM

I strongly doubt the problem happens at the traffic layer. This seems to be a different kind of problem - maybe those pages once edited overflow some specific limit.

A null edit didn't result in any error in my test. Do we have better steps for reproducing the issue?

Joe lowered the priority of this task from High to Low.Nov 13 2020, 7:54 AM
Joe edited projects, added MediaWiki-Parser; removed Traffic.

From the diuscussion on VP:T I would assume the editor just hit the maximum amount of templates to include in a single article. Lowering priority and removing traffic from the task, adding what I think is the actual problem.

Hmm, going to the wikipage; selecting Show Preview, and then expanding Parser profiling data shows nothing close to limits.
(Same for looking at the NewPP limit report comment in the HTML source of that rendered page).

User:Davidwr reports on the VP:T thread that this diff hits the limit, and I can reproduce that myself:

Post-expand include size	2,097,152/2,097,152 bytes

But I still can't repro the 503 errors.

It is not normal for a PEIS max to cause a 503, in my experience. While it is an actual problem due to the artificial limit, we more-or-less always see the page as expected along with as much content as will render before hitting the limit, even when trying to edit.

Speculating, but one or another of the caches could be having trouble with serving so many images on the page, possibly even a specific machine. These are half-kb images and I'd guess there are a couple hundred. Coupled with the HTML load associated with a table...

Alternatively, mjroots may not have that great of internet service.

It might be interesting for mjroots to try editing without having the 'show a preview above my edit box'. Maybe also try changing to the no-Javascript wikitext editor and seeing if any of those improves editing the page in question.

Izno, there are no issues with my internet service, the rest of the net is working fine without any issues accessing websites and web pages. I don't understand much of the technical stuff behind the editing software, so am unsure about the not having the show a preview and no-javascripte wikitext editor stuff. I don't like to interfere with my settings where this sort of stuff stuff is concerned. Can we rule out a web browser issue as it doesn't seem to be a PEIS issue from what has been said at VPT. I use Firefox.

I could not reproduce with Firefox, Win10, latest version (82?).

Izno, there are no issues with my internet service, the rest of the net is working fine without any issues accessing websites and web pages.

When you're editing, we don't have the benefit of the page cache so everything has to be rendered from scratch and downloaded to your computer (in two forms, one the preview and one the wikitext). I'm exploring that as a potential point for why you are having problems. (More below)

show a preview and no-javascripte wikitext editor stuff. I don't like to interfere with my settings where this sort of stuff stuff is concerned.

These are preferences you should be able to find on the Editing tab. One is named "Enable the editing toolbar" and the other is named "Show preview on first edit". Consider trying to edit one of the painful pages/revisions without one or both of those on.

You may also have a gadget or some JavaScript that is taking a while to download or load on those pages.

Izno - under preferences > editing I have the following ticked:-

General options
Show the difference between the latest accepted version and the latest pending revision when editing pages
Enable showing appreciation for other users with the WikiLove tab

Editor
Prompt me when entering a blank edit summary (or the default undo summary)
Enable the editing toolbar (This is sometimes called the '2010 wikitext editor)
Temporarily disable the visual editor while it is in beta

Preview
Show preview before edit box'.

The November 1869 list has been loading correctly so far today, so it is unclear whether altering anything will result in knowing whether or not the alteration has had an effect.

Afaik, on (file object) cache miss, parser doesn't bulk load file objects, but loads them one at a time from db as it encounters them in the wikitext (compared to say how doing link existence checks works with batching). Even though its a local db, this probably adds up, especially if there's other complex stuff on the page. That would also explain why the null edit is fast - its probably succesfully hitting memcached much more.

This is all just a guess though based on not much evidence. You would have to use profiling to confirm.

Basically i suspect this is a dupe of T56033