Page MenuHomePhabricator

[Epic] Extract lexemes out of WDQS
Closed, DeclinedPublic

Description

As an operator of WDQS, I want ensure that the service can scale and accommodate growth due to new projects by extracting lexemes out of WDQS.

As a user of WDQS / Lexemes, I want to have a stable service that can provide reliability and performances.

Given the push to increase Lexeme usage, we need to be ready to accommodate this growth. Lexemes are a well identified subset of Wikdiata (identified by a separate namespace), so it is a good candidate to be split out as an independent sub-graph. Spinning up a new blazegraph instance seems like a reasonable strategy toward growth.

A few thoughts:

  • Since we are increasing the number of query service instances, we need to rationalize this approach so that the maintenance burden does not grow linearly with the number of instances. This includes identifying and addressing opportunities for automation, hardware allocation, etc...
  • Increasing the number of backends will require also increasing the number of UI / frontends, or rearchitecture them to support multiple backends from a single UI.
  • Updates needs to be split out of the current update flow.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
MPhamWMF moved this task from Incoming to Epics on the Wikidata-Query-Service board.
Gehel lowered the priority of this task from High to Medium.Jun 10 2021, 2:34 PM
MPhamWMF subscribed.

declining this, as splitting Lexemes from the WD graph will not significantly help our scaling situation. Any potential future work to split the WD graph will replace this epic