@Arlolra HTTP error code is not the same but otherwise looks for me to be a similar problem like https://phabricator.wikimedia.org/T280381

An other example https://fr.wikipedia.org/api/rest_v1/page/mobile-sections/Liste_des_cantons_fran%C3%A7ais_depuis_2015

It was noted in T280381#7185326 that this ruwiki page times out. That particular page might benefit from T251624 or T214662

The frwiki page can probably be filed as a dupe of T206040

Parsoid/JS used to have a timeout of 110s but Parsoid/PHP is down to 1min,
https://github.com/wikimedia/mediawiki-services-parsoid-deploy/blob/master/scap/templates/config.yaml.j2#L63-L67

In T288889#7283144, @Kelson wrote:

@Arlolra HTTP error code is not the same but otherwise looks for me to be a similar problem like https://phabricator.wikimedia.org/T280381

T280381 was about removing arbitrary complexity limits that Parsoid had in place that differed from the legacy parser.

These pages are failing for performance reasons. In order for Parsoid to be the main parser in use, we're going to have to address them. It's helpful to collect examples here for when we get to that work, which I believe may get started in the next few quarters.

@Arlolra Sounds good. Overall situation with Wiki timeouts/errors has been improved over the past months, but we still have problems to fully scrape many of them, mostly the big ones. Will keep trying to attach our MWoffliner tickets to Phabricator tickets so info keep percolating up to you :)

Other case:

$ time curl -sI "https://zh.wikisource.org/api/rest_v1/page/html/%E6%98%8E%E6%9C%AC%E6%8E%92%E5%AD%97%E4%B9%9D%E7%B6%93%E7%9B%B4%E9%9F%B3_(%E5%9B%9B%E5%BA%AB%E5%85%A8%E6%9B%B8%E6%9C%AC)%2F%E5%8D%B7%E4%B8%8B" | grep 504
HTTP/2 504 

real	2m33.215s
user	0m0.035s
sys	0m0.006s

Other case:

$ time curl -sI "https://de.wikisource.org/api/rest_v1/page/html/Schwere%2C_Elektricit%C3%A4t_und_Magnetismus%2FErster_Theil" | grep 504
HTTP/2 504 

real	1m5.362s
user	0m0.036s
sys	0m0.012s

$ time curl -sI "https://zh.wikisource.org/api/rest_v1/page/html/%E6%98%8E%E6%9C%AC%E6%8E%92%E5%AD%97%E4%B9%9D%E7%B6%93%E7%9B%B4%E9%9F%B3_(%E5%9B%9B%E5%BA%AB%E5%85%A8%E6%9B%B8%E6%9C%AC)%2F%E5%8D%B7%E4%B8%8B" | grep 504

This one looks like T275505

Other case:

$ time curl -sI "https://id.wikipedia.org/api/rest_v1/page/mobile-sections/Daftar_tokoh_Wales" | grep 504
HTTP/2 504 

real	2m29.723s
user	0m0.019s
sys	0m0.018s

Another case with WPFR:

 time curl -sI "https://fr.wikipedia.org/api/rest_v1/page/mobile-sections/Liste_des_cantons_fran%C3%A7ais_depuis_2015" | grep 504
HTTP/2 504 

real	1m9.307s
user	0m0.020s
sys	0m0.005s

@Arlolra @tstarling The situation on this front has improved *a lot* these last months. I'm not sure to know who to thank you for that but please transmit my "thank you". That said the situation is not fully sanitised. I went through the past buggy examples and have added a few new ones which make recipes failing at https://farm.openzim.org:

• https://de.wikisource.org/api/rest_v1/page/html/Schwere%2C_Elektricit%C3%A4t_und_Magnetismus%2FErster_Theil
• https://fr.wikiversity.org/api/rest_v1/page/html/M%C3%A9canique_des_syst%C3%A8mes_de_points%2FCin%C3%A9tique_et_dynamique_d'un_syst%C3%A8me_discret_de_points_mat%C3%A9riels
• https://en.wikiversity.org/api/rest_v1/page/html/Quizbank%2FcalcPhyEMqAll%2Fc05
• https://pt.wikibooks.org/api/rest_v1/page/html/An%C3%A1lise_real%2FImprimir

I have rechecked these 4 URLs posted in my lastest comment 6 months again. I can confirmed that all the four of them still failed with a HTTP 504 error.