Page MenuHomePhabricator

How do I prepare my tool for migration period?
Open, Needs TriagePublic

Description

What is the implication on queries that used to run against wb_terms during migration?

During the migration, in order to continue accessing the full data regarding entity terms as needed in your tool, one of the two schemas will have to be queried depending on some conditions or timeframes.

For property terms this is a simple switch from querying wb_terms to querying the new schema. For Item terms it requires more logic to know which schema to read from, as both will some part of the data.

In which periods do I need to be running queries against which schemas?

The following timeline shows the checkpoints of the migration in production environment and how they affect queries in general (dates are approximate atm but will be fixed 2 weeks prior to ):

... June 24th:   Property Terms migration starts
.
.                   here nothing need to be changed. all queries that fetch property terms can still read from `wb_terms` as they used to.
.
... June 26th:   Read proprty terms from new schema on Wikidata
.
.                   tools must beging querying new schema for property terms here.
.                   terms will still be written to old `wb_terms` but that is only for the sake of recoverability in case of problems.
.
... July 3rd:  Item terms migration begins
.
.                   we will migrate only the first 2,000,000 items (with their terms, that's ~1% of the total amount).
.                   nothing need to be changed in here yet. all queries that fetch item terms can still read from `wb_terms` as they used to.
.
... July 10th:  Read item terms from one of the two schemas - read one
.
.                   tools must read item terms from one of the two schemas based on the item ID:
.                   - if the integer part of Item id is less than 2,000,000 then it should be read from the new schema
.                   - otherwise, it should be read from `wb_terms`
.                   
... TBD:        Item terms migration continues for all remaning items
.
.                   this will be delayed until we have more capacity on database master node to continue the migration.
.                   this will be announced separately with dates when tools should start reading all item terms from new schema
.
... TBD:        Drop wb_terms table

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 24 2019, 12:16 PM
alaa_wmde renamed this task from How do I migrate my tool? to How do I prepare my tool for migration period?.Apr 24 2019, 12:17 PM
alaa_wmde updated the task description. (Show Details)Apr 24 2019, 12:19 PM
alaa_wmde updated the task description. (Show Details)Apr 24 2019, 12:29 PM
alaa_wmde updated the task description. (Show Details)Apr 24 2019, 12:38 PM
alaa_wmde updated the task description. (Show Details)Apr 25 2019, 5:11 PM
alaa_wmde updated the task description. (Show Details)May 15 2019, 2:47 PM
WMDE-leszek updated the task description. (Show Details)Thu, Jun 20, 3:56 PM