Page MenuHomePhabricator

Citation tool fails to add citations for calls longer than 20 seconds
Open, Needs TriagePublic

Description

Trying to add citations to the Folha de S.Paulo newspaper today I noticed all attempts are failing.

Citoid does manage to respond, but usually just over 20 seconds, apparently the citation tool in visual editor is set to consider it a failure at 20 seconds?

Could we increase the timeout?

Event Timeline

@Chicocvenancio for, let's say, the last 10 citations you've tried to add using Citoid, how many have timed-out? Also, have you noticed any similarities among the attempts that timeout?

It seems to me that 20 seconds is way longer than it should take. Perhaps that's the real problem.

Visual editor indeed only waits for 20 seconds, this was implemented in 2015 by @Mooeypoo: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Citoid/+/197774/3..4 (the default timeout is 30 seconds). I don't know why it was done this way; maybe it can be changed.

@Chicocvenancio for, let's say, the last 10 citations you've tried to add using Citoid, how many have timed-out? Also, have you noticed any similarities among the attempts that timeout?

As per the bug, it's all on the same website.

As a web scraper we are somewhat limited by upstream (read: how fast the website we're requesting is) but even so citoid performance is pretty bad and there is definitely room for improvement. I have a change in to switch back to gzip which should help a bit (https://gerrit.wikimedia.org/r/c/mediawiki/services/zotero/+/682708) but there's probably a lot more we should do that we aren't doing. @ppelberg are there any sort of performance review or metrics people that we could loop in? I've actually just finished performance training but it was front end performance, not back end! (Still, some generally applicable things were in there that I can use to work on this!)

It seems to me that 20 seconds is way longer than it should take. Perhaps that's the real problem.

Visual editor indeed only waits for 20 seconds, this was implemented in 2015 by @Mooeypoo: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Citoid/+/197774/3..4 (the default timeout is 30 seconds). I don't know why it was done this way; maybe it can be changed.

The problem is that sometimes requests that take a really long time like this ultimately fail and that is frustrating to the user that then has nothing to show for all the waiting. Obviously in this case it is bad though. @ppelberg maybe this is something maybe UX should have input on? Is there a more scientific way to come up with this number?