Change Details

Currently there are a lot of issues for evaluation and analysis, such as: T206560 - Evaluate alternatives to BG (including lots of subtasks around testing and evaluating alternatives) T306725 - Decide which BG services to migrate (assuming a migration is bound to happen) but nothing for the migration itself, which cannot be avoided. Hence this task. [Meanwhile there are more issues opening every season for fixing slowness, failures, or other inconsistencies with Blazegraph, an ongoing tax for not migrating.] The time is ripe: Blazegraph instability is slowing down data reloads on WDQS, and may prevent them altogether in the future: @Gehel [[ https://lists.wikimedia.org/hyperkitty/list/wikidata@lists.wikimedia.org/message/7QTJBRU2T3J22SNV4TGBRML4QNBGCEOU/ | wrote ]]: > TL;DR: We expect to successfully complete the recent data reload on Wikidata Query Service soon, but we've encountered multiple failures related to the size of the graph, and anticipate that this issue may worsen in the future. Although we succeeded this time, we cannot guarantee that future reload attempts will be successful given the current trend of the data reload process. Thank you for your understanding As the Query Service is the public-facing part of Wikidata in many contexts, this feels like preventing WD itself from being updated. Proposal: implement a production-quality migration before we next need to reload the query service. Even if there is a double-backend solution for a time (T290839), or the initial migration exposes a need to change the migration strategy or change the target backend, having even a temporary alternative may provide an alternative way to resolve queries. And working through the migration process would bring needed attention to this critical step in WD growth, and may help practice future recovery workflows.

Currently there are a lot of issues for evaluation and analysis, such as: T206560 - Evaluate alternatives to BG (including lots of subtasks around testing and evaluating alternatives) T306725 - Decide which BG services to migrate (assuming a migration is bound to happen) but nothing for the migration itself. It seems unavoidable, hence this task. [Meanwhile there are more issues opening every season for fixing slowness, failures, or other inconsistencies with Blazegraph, an ongoing tax for not migrating.] The time is ripe: Blazegraph instability is slowing down data reloads on WDQS, and may prevent them altogether in the future: @Gehel [[ https://lists.wikimedia.org/hyperkitty/list/wikidata@lists.wikimedia.org/message/7QTJBRU2T3J22SNV4TGBRML4QNBGCEOU/ | wrote ]]: > TL;DR: We expect to successfully complete the recent data reload on Wikidata Query Service soon, but we've encountered multiple failures related to the size of the graph, and anticipate that this issue may worsen in the future. Although we succeeded this time, we cannot guarantee that future reload attempts will be successful given the current trend of the data reload process. Thank you for your understanding As the Query Service is the public-facing part of Wikidata in many contexts, this feels like preventing WD itself from being updated. Proposal: implement a production-quality migration before we next need to reload the query service. Even if there is a double-backend solution for a time (T290839), or the initial migration exposes a need to change the migration strategy or change the target backend, having even a temporary alternative may provide an alternative way to resolve queries. And working through the migration process would bring needed attention to this critical step in WD growth, and may help practice future recovery workflows.

Currently there are a lot of issues for evaluation and analysis, such as: T206560 - Evaluate alternatives to BG (including lots of subtasks around testing and evaluating alternatives) T306725 - Decide which BG services to migrate (assuming a migration is bound to happen) but nothing for the migration itself, which cannot be . It seems unavoided. Hable, hence this task. [Meanwhile there are more issues opening every season for fixing slowness, failures, or other inconsistencies with Blazegraph, an ongoing tax for not migrating.] The time is ripe: Blazegraph instability is slowing down data reloads on WDQS, and may prevent them altogether in the future: @Gehel [[ https://lists.wikimedia.org/hyperkitty/list/wikidata@lists.wikimedia.org/message/7QTJBRU2T3J22SNV4TGBRML4QNBGCEOU/ | wrote ]]: > TL;DR: We expect to successfully complete the recent data reload on Wikidata Query Service soon, but we've encountered multiple failures related to the size of the graph, and anticipate that this issue may worsen in the future. Although we succeeded this time, we cannot guarantee that future reload attempts will be successful given the current trend of the data reload process. Thank you for your understanding As the Query Service is the public-facing part of Wikidata in many contexts, this feels like preventing WD itself from being updated. Proposal: implement a production-quality migration before we next need to reload the query service. Even if there is a double-backend solution for a time (T290839), or the initial migration exposes a need to change the migration strategy or change the target backend, having even a temporary alternative may provide an alternative way to resolve queries. And working through the migration process would bring needed attention to this critical step in WD growth, and may help practice future recovery workflows.