Page MenuHomePhabricator
Create Task
Maniphest T262828

Near zero downtime Data reload for WCQS
Closed, ResolvedPublic5 Estimated Story Points
Actions

Description

As a user I want a shortest possible downtime for WCQS in case of data reload, so that I can continue using that.

Proposed solutions:

  • Feed the second blazegraph instance, switch journals afterwards
  • The same but with two namespaces (single instance of Blazegraph)

Category reload happens this way already.

Switch can also happen via nginx.

AC:

  • data reload doesn't trigger maintenance page
  • data is available after reload.

Details

SubjectRepoBranchLines +/-
Provide active/standby namespaces for WCQSwikidata/query/rdfmaster+24 -24
Switch between active W[D|C]QS indexesoperations/puppetproduction+6 -9
Customize query in gerrit

Related Objects
Search...

Event Timeline

• Zbyszko created this task.Sep 14 2020, 3:39 PM
Restricted Application added a project: Wikidata. · View Herald TranscriptSep 14 2020, 3:39 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
• Zbyszko moved this task from Incoming to Current work on the Wikidata-Query-Service board.Sep 14 2020, 3:39 PM
• Zbyszko added a project: Discovery-Search (Current work).
Bugreporter subscribed.Sep 14 2020, 4:38 PM

In WDQS, there are several instances and they are not reloaded at the same time.

Gehel subscribed.Sep 14 2020, 5:26 PM

Using a single instance with 2 namespaces seems to be the faster way to implement this.

Gehel set the point value for this task to 5.Sep 14 2020, 5:31 PM
Gehel moved this task from Incoming to Ready for Dev -- SWE on the Discovery-Search (Current work) board.
• Zbyszko claimed this task.Sep 15 2020, 7:40 AM
• Zbyszko triaged this task as High priority.
• Zbyszko moved this task from Ready for Dev -- SWE to In Progress on the Discovery-Search (Current work) board.
gerritbot added a comment.Sep 15 2020, 12:57 PM

Change 627485 had a related patch set uploaded (by ZPapierski; owner: ZPapierski):
[wikidata/query/rdf@master] Provide active/standby indexes for WCQS

https://gerrit.wikimedia.org/r/627485

gerritbot added a project: Patch-For-Review.Sep 15 2020, 12:57 PM
gerritbot added a comment.Sep 15 2020, 1:01 PM

Change 627495 had a related patch set uploaded (by ZPapierski; owner: ZPapierski):
[operations/puppet@production] Switch between active W[D|C]QS indexes

https://gerrit.wikimedia.org/r/627495

• Zbyszko moved this task from In Progress to Needs review on the Discovery-Search (Current work) board.Sep 15 2020, 1:07 PM
gerritbot added a comment.Sep 16 2020, 8:34 AM

Change 627495 abandoned by ZPapierski:
[operations/puppet@production] Switch between active W[D|C]QS indexes

Reason:
Made obsolete.

https://gerrit.wikimedia.org/r/627495

gerritbot added a comment.Sep 21 2020, 4:06 PM

Change 627485 merged by jenkins-bot:
[wikidata/query/rdf@master] Provide active/standby namespaces for WCQS

https://gerrit.wikimedia.org/r/627485

• Zbyszko moved this task from Needs review to To Be Deployed on the Discovery-Search (Current work) board.Sep 21 2020, 5:33 PM
Gehel added a parent task: T251488: [epic] Create minimal SPARQL Endpoint for Commons on WMCS.Oct 30 2020, 1:47 PM
• Zbyszko moved this task from To Be Deployed to Needs Reporting on the Discovery-Search (Current work) board.Nov 2 2020, 6:04 PM
Gehel closed this task as Resolved.Nov 9 2020, 1:02 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits