Page MenuHomePhabricator

Timeout in Special Page Constraints check for France (Q142)
Open, LowPublic

Description

The WikiData item for France (Q142) is very large. As a side effect, the constraints check via the special page for that item, https://www.wikidata.org/wiki/Special:ConstraintReport/Q142, results in a server error, most likely due to a timeout.

Acceptance criteria:

  • Constraint checks for very large pages no longer results in a fatal server error and timeout

Event Timeline

Michael created this task.Dec 19 2018, 10:08 AM
Restricted Application added a project: Wikidata. · View Herald TranscriptDec 19 2018, 10:08 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

So, I managed to make this page work https://www.wikidata.org/wiki/Special:ConstraintReport/Q142

But I also see:

2018-12-19T12:32:49	mw1252	WARNING	Full constraint check on Q142 took longer than 55 second(s) (duration: 56.151183128357 seconds).

Which is pretty close to the timeout for a web request (I believe 60 seconds)...

Addshore triaged this task as Normal priority.Dec 19 2018, 12:39 PM
Addshore added a project: Wikibase-Quality.
Addshore moved this task from incoming to monitoring on the Wikidata board.

The idea here is that constraint checks would be regularly run / run post edit T201150 and store the results more persistently T204024 meaning web requests like this don't actually have to process the checks.

Well, not really, I believe… the special page already doesn’t use cached results, and I think it should continue to be a way to request a full, fresh constraint check in the future as well.

Well, not really, I believe… the special page already doesn’t use cached results, and I think it should continue to be a way to request a full, fresh constraint check in the future as well.

It should probably start to used the cached checks, but provide a way to purge the check.

[...]
But I also see:

2018-12-19T12:32:49	mw1252	WARNING	Full constraint check on Q142 took longer than 55 second(s) (duration: 56.151183128357 seconds).

[...]

Are we logging this anywhere? Maybe the time these constraints checks take in general and the also items for which the checks take longer than $threshold.

Michael moved this task from 🗃️ Incoming to 👁️ watching on the User-Michael board.

Are we logging this anywhere? Maybe the time these constraints checks take in general and the also items for which the checks take longer than $threshold.

Yes, these all appear in logstash.
EG: https://logstash.wikimedia.org/goto/f7ec3a5c71949b20e1b08a9a1b2f2325

Krinkle added a subscriber: Krinkle.EditedMar 19 2019, 6:17 PM

(Example of task description is still reproducible.)

If it is feasible that no constraint checks are never run on-demand, then I suppose the solution is to (after that) remove the access path to it. Replacing it an error message to the user and an application log message with warning or info level (severity depends on whether it is expected to sometimes still be possible).

However, if that is not likely to become reality soon, then perhaps it makes sense to acknowledge the current reality that for the common case of them not yet being precomputed, to set a very low self-imposed timeout (e.g. 2 seconds) and then bail out with a user-focussed error message (not an exception or application error log).

When a request is timing out, that is a sign of a problem in the software. The internal threshold should be far below 60 (not 55).

tldr; Yes we should and are going toward not having these run in the requests.

Constraint check runs are going to run post edit and be persistently stored.
T201150: Regularly run constraint checks for all items
T204024: Store WikibaseQualityConstraint check data in persistent storage instead of in the cache

This will mean most of these requests for constraint check data will just load from the store.

There will be some cases that the data stored is out of date and will need to be recalculated.
This is discussed in the current rfc @ T214362#4980932 and the few comments after that one.
The initial plan for those cases would be to keep the current behaviour (run the checks in the request, as well as trigger something to store some updated values for subsequent requests) however when we discuss this in the future there are other options such as don't show anything / say they are generating, do the checking in the job queue, save them, and then get the UI to poll & wait for them etc.

Addshore lowered the priority of this task from Normal to Low.Sat, Jun 22, 10:03 PM
Addshore removed a project: Wikidata-Campsite.

Removing from the campsite.
What we ideally want to do here is store constraint check data, and have the checks run post edit.
This is all tracked in other tickets.