Page MenuHomePhabricator

Database lag
Closed, DeclinedPublic

Description

Some queries return outdated results.

The query https://quarry.wmflabs.org/query/2208 should show linkings to unexisting categories but in the resultset there is a not longer linked category newly deleted. The query https://quarry.wmflabs.org/query/2211 should show empty categories but it shows a not-empty category newly created.

I clicked on "submit query" a second time, but I still got the same resultset. IMHO there is a lag in the database so changes on Wikipedia are not immediately showed in the copy of the database.

Event Timeline

jcrespo subscribed.

there is a lag in the database so changes on Wikipedia are not immediately showed in the copy of the database

Not only there is lag on labs instances by design (network/labs db infrastructure problem, production problem, maintenance (scheduled or unscheduled), excessive load or production or user's queries blocking the replication process), *links tables (among others) are updated in production by the job queue, asynchronously-in the background , so they are not consistent with the actual contents. You can see the lag of the job queue here: https://grafana.wikimedia.org/dashboard/db/job-queue-health?panelId=7&fullscreen.

Lag can be seen at https://tools.wmflabs.org/replag/, or on the heartbeat tables, and your application/queries should be aware of that.