Page MenuHomePhabricator

🟡 Adjust QueryService updater batch order
Closed, ResolvedPublic

Description

We should adjust the order that queryservice updater batches are worked on ( https://github.com/wbstack/api/blob/cc8323a6e88c11919aa35dd55f3cff9bffa16176/app/Http/Controllers/Backend/QsController.php#L76 )

To ensure that we don't have a "last in first out" system where unlucky wikis end up at the back of a queue while a busy wiki is continually editing.

Refactoring of this class and adding some tests is clearly important.

The logic should could look as follows:

  • generate new batches
  • coalesce old and new batches together if possible
  • if there are already batches that are not done return the oldest of these first

We should ensure that whatever implementation we chose doesn't favour one heavily edited wiki over a smaller more lightly edited wiki.

Event Timeline

@Evelien_WMDE we (I) think this is also perhaps to be viewed as a product thing. This is probably a cause of people complaining their stuff isn't visible in their query service for ages.

We might also want to add metrics on the age of the oldest batch in the queue that hasn't been process.

When we are working on this we need to refactor the code and test this more rigorously. It's rather hard to read and I am suspicious that we accidentally drop work here

Evelien_WMDE renamed this task from Adjust QueryService updater batch order to 🟡 Adjust QueryService updater batch order.Oct 9 2023, 9:52 AM
Evelien_WMDE moved this task from Doing to To do on the Wikibase Cloud (Kanban board Q4 2023) board.
Evelien_WMDE added a subscriber: Fring.
Fring removed Fring as the assignee of this task.Oct 24 2023, 12:14 PM
Evelien_WMDE claimed this task.