Page MenuHomePhabricator

504 Gateway Time-out errors after creating new formulas on pt.wikipedia
Closed, ResolvedPublic

Description

I'm getting Time-out errors every time I review a math article and rewrite many of its formulas. The most recent example is
https://pt.wikipedia.org/w/index.php?diff=prev&oldid=37303227

where I got

504 Gateway Time-out

nginx/1.1.19

But, if I remeber correctly, this edit also caused the same error:
https://pt.wikipedia.org/w/index.php?diff=prev&oldid=37279458
and I'm certain there was more cases before this.

In all cases the edit seems to be saved anyway.

Maybe it is related to bug 53400?


Version: unspecified
Severity: normal
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=71772

Details

Reference
bz56769

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 2:19 AM
bzimport set Reference to bz56769.
bzimport added a subscriber: Unknown Object (MLST).
He7d3r created this task.Nov 8 2013, 11:59 AM
He7d3r added a comment.Nov 8 2013, 1:28 PM

When I tried to replicate the first edit above on English Wikipedia, the error was this:

Request: POST http://en.wikipedia.org/w/index.php?title=Wikipedia:Sandbox&action=submit, from 208.80.154.8 via cp1002.eqiad.wmnet (squid/2.7.STABLE9) to 10.64.0.130 (10.64.0.130)
Error: ERR_READ_TIMEOUT, errno [No Error] at Fri, 08 Nov 2013 13:15:35 GMT
and the edit was not saved.

Then I tried again and it was saved without problem:
https://en.wikipedia.org/w/index.php?diff=580753031&oldid=580752780

Then I used the content of the link
https://pt.wikipedia.org/w/index.php?title=Espa%C3%A7o_vetorial&action=edit&oldid=37279458
and got:
Request: POST http://en.wikipedia.org/w/index.php?title=Wikipedia:Sandbox&action=submit, from 10.64.0.138 via cp1008.eqiad.wmnet (squid/2.7.STABLE9) to 10.2.2.1 (10.2.2.1)
Error: ERR_READ_TIMEOUT, errno [No Error] at Fri, 08 Nov 2013 13:26:34 GMT

but this time the edit was saved:
https://en.wikipedia.org/w/index.php?title=Wikipedia:Sandbox&diff=prev&oldid=580753431

Helder: Is this still a problem?

Yes: I just copied the code from first link above to
https://pt.wikipedia.org/wiki/WP:P%C3%A1gina_de_testes/1?diff=37413683
and then made the replacement
/<math>/g --> <math>bug
to generate new formulas. The result was this (and the edit was not saved):

Request: POST http://pt.wikipedia.org/w/index.php?title=Wikip%C3%A9dia:P%C3%A1gina_de_testes/1&action=submit, from 208.80.154.51 via cp1052 frontend ([10.2.2.25]:80), Varnish XID 1553287318
Forwarded for: 143.107.45.11, 208.80.154.51
Error: 503, Service Unavailable at Mon, 18 Nov 2013 10:00:15 GMT

Then (after a few minutes) I hit the back button from my browser and clicked save again. The edit was saved but the error persisted:

Request: POST http://pt.wikipedia.org/w/index.php?title=Wikip%C3%A9dia:P%C3%A1gina_de_testes/1&action=submit, from 208.80.154.51 via cp1068 frontend ([10.2.2.25]:80), Varnish XID 3606772663
Forwarded for: 143.107.45.11, 208.80.154.51
Error: 503, Service Unavailable at Mon, 18 Nov 2013 10:04:59 GMT

On Portuguese Wikibooks, I also get an error message when I save edits in which lots of formulas[1] are changed:
Request: POST http://pt.wikibooks.org/w/index.php?title=Matem%C3%A1tica_elementar/Trigonometria/Lei_dos_senos_e_dos_cossenos&action=submit, from 208.80.154.9 via cp1065 frontend ([10.2.2.25]:80), Varnish XID 2785820477
Forwarded for: 200.144.188.135, 208.80.154.9
Error: 503, Service Unavailable at Wed, 20 Nov 2013 09:59:28 GMT

[1] Example: https://pt.wikibooks.org/w/index.php?diff=264420&oldid=264419

It happened again:
https://pt.wikibooks.org/w/index.php?diff=264423
Request: POST http://pt.wikibooks.org/w/index.php?title=Matem%C3%A1tica_elementar/Polin%C3%B4mios&action=submit, from 208.80.154.9 via cp1066 frontend ([10.2.2.25]:80), Varnish XID 3663099635
Forwarded for: 200.144.188.135, 208.80.154.9
Error: 503, Service Unavailable at Wed, 20 Nov 2013 10:44:44 GMT

aaron added a comment.Nov 26 2013, 8:49 PM

Some eval.php profiling.

$title = Title::newFromText( 'Math page' );

$text = ''; for ( $i=0; $i<100; ++$i ) { $text .= "<math>i = " . mt_rand( 0, 2000000000 ) . "</mat h>\n"; }

$content = ContentHandler::makeContent( $text, $title );

$content->getParserOutput( $title );

wfLogProfilingData();

<!--
100.00% 93.631767 1 - -total
99.96% 93.597670 1 - Parser::parse
99.96% 93.597645 1 - Parser::parse-call_user_func_array
99.84% 93.479773 317 - PPFrame_DOM::expand
99.83% 93.474312 1 - Parser::internalParse
99.83% 93.472083 5 - Parser::replaceVariables
99.74% 93.385419 100 - MathHooks::mathTagHook
98.58% 92.301875 100 - MathTexvc::callTexvc
31.91% 29.878679 100 - FileBackendStore::doQuickOperationsInternal-global-swift
31.90% 29.865421 100 - FileBackendStore::storeInternal-global-swift
24.23% 22.683115 100 - FileBackendStore::doQuickOperationsInternal-global-swift-eqiad
24.21% 22.667449 100 - FileBackendStore::storeInternal-global-swift-eqiad

4.10% 3.843399    100 - FileBackendStore::doPrepare-global-swift
0.96% 0.894944    100 - FileBackendStore::doPrepare-global-swift-eqiad
0.79% 0.738057    100 - MathTexvc::writeCache
0.59% 0.550794    100 - DatabaseBase::query-master
0.58% 0.545479    100 - query-m: REPLACE INTO `math` (math_inputhash,math_outputhash,math_html_con                                                                        servativeness,math_html,math_mathml) VALUES ('X')

Change 103644 had a related patch set uploaded by Aaron Schulz:
Buffer png writes and flush them all at once

https://gerrit.wikimedia.org/r/103644

Change 103644 merged by Physikerwelt:
Buffer png writes and flush them all at once

https://gerrit.wikimedia.org/r/103644

physik wrote:

I'm really unhappy that nobody is reviewing math 2.0 which would resolve all the unnecessary interactions with the file system.
It takes a lot of time to manually rebase the change over and again.
Compar this change:
Uploaded Dec 25, 2013 9:39 AM
Updated Dec 27, 2013 11:42 AM (merged)
with that one https://gerrit.wikimedia.org/r/#/c/85801/
Uploaded Sep 24, 2013 2:28 AM
...

aaron added a comment.Dec 28 2013, 2:28 AM

I was going to just wait, but it figured I may as well make a simple fix to help now. I did checkout Math 2.0 out of curiousity a few days ago, but all the pngs just were replaced with "texvc not found" errors and I ended up doing other things.

physik wrote:

Yes this happens if texvccheck is not compiled, since the binaries are ignored in the .gitignore file.
If you switch to texvccheck and run make there the problem should disappear.

Patch was merged a while ago - is there more work left to do here (if yes: please reset the bug report status to NEW or ASSIGNED), or can you close this ticket as RESOLVED FIXED?