Cloud VPS "wikibase-registry" project Stretch deprecation
Closed, ResolvedPublic
Actions

Description

The end of life of Debian Stretch is approaching in 2022 and we need to move to Debian Bullseye (or Buster) before that date.

All instances in the wikibase-registry project need to upgrade as soon as possible. Instances not upgraded by 2022-05-01 may be subject to deletion unless prior arrangements for an extended deadline has been approved by the Cloud VPS administration team.

Remaining Debian Stretch instances (live report):

wbregistry-01.wikibase-registry.eqiad1.wikimedia.cloud

Listed administrators are:

Details

Due Date: Apr 30 2022, 11:59 PM

Event Timeline

StrikerBot triaged this task as Medium priority.Apr 13 2022, 5:00 PM

StrikerBot created this task.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 13 2022, 5:00 PM

Zabe changed the edit policy from "Custom Policy" to "All Users".Apr 13 2022, 5:31 PM

Addshore added a project: wmde-team-b-tech.May 10 2022, 12:49 PM

Flagging this with folks so that it gets prioritized

WMDE-leszek added a project: Wikibase Cloud (Launch Migration Kanban (2022)).Jun 28 2022, 9:16 AM

• toan claimed this task.Jun 28 2022, 11:18 AM

• toan moved this task from Backlog to Doing on the Wikibase Cloud (Launch Migration Kanban (2022)) board.Jun 28 2022, 11:38 AM

So, i think it's done but turns out the wdqs-updater has been broken since forever :D

https://openstack-browser.toolforge.org/project/wikibase-registry is cached now but should hopefully update and indicate that the proxies are using the new instance.

Ok the queryservice should now be updated with a fresh dump of the old instance and the queryservice-updater is running again.

How the stuff was moved between the instances.

Create a new instance on https://horizon.wikimedia.org, it now has 20gb less space than the original. however we're currently only using 31% of the disk capacity so should be good.
Install docker.io / docker-compose
tar original files (it contained some changes outside of the git repo and will remain on the new instance), docker volumes on original wikibase-registry
docker save wikibase-registry:1.32-bundle-1 > 1.32-bundle-1.tar to a tar (there is a blog post on how to do this but im lazy https://addshore.com/2019/02/creating-a-dockerfile-for-the-wikibase-registry/)
haul all the tarballs to the new instance
bring up everything docker-compose up and noticed wdqs-updater wasn't updating since a long while
dump the rdf after https://github.com/wmde/wikibase-registry/pull/4 was merged
Create a new empty volume with https://github.com/wmde/wikibase-registry/pull/5
Copy the files from wikibase container to wdqs container
munge the dump and ingest it

bash-4.4# java -cp lib/wikidata-query-tools-*-jar-with-dependencies.jar org.wikidata.query.rdf.tool.Munge --from /tmp/wikibase-registry.ttl --to /tmp/mungeOut/wikidump-%09d.ttl.gz --chunkSize 10000 -w wikibase-registry.wmflabs.org --conceptUri http://wikibase-registry.wmflabs.org
#logback.classic pattern: %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n
10:44:56.391 [main] INFO  org.wikidata.query.rdf.tool.Munge - Switching to /tmp/mungeOut/wikidump-000000001.ttl.gz
bash-4.4# ./loadData.sh -n wdq -d /tmp/mungeOut/
Processing wikidump-000000001.ttl.gz
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"><html><head><meta http-equiv="Content-Type" content="text&#47;html;charset=UTF-8"><title>blazegraph&trade; by SYSTAP</title
></head
><body<p>totalElapsed=291ms, elapsed=223ms, connFlush=0ms, batchResolve=0, whereClause=0ms, deleteClause=0ms, insertClause=0ms</p
><hr><p>COMMIT: totalElapsed=713ms, commitTime=1656499525195, mutationCount=5271</p
></html
>File wikidump-000000002.ttl.gz not found, terminating

switch over WebProxies to the new instance and seems to work

@WMDE-leszek give me the go ahead to delete the old instance and I will try to do that

• toan removed • toan as the assignee of this task.Jun 29 2022, 11:24 AM

• toan moved this task from Doing to Review on the Wikibase Cloud (Launch Migration Kanban (2022)) board.

Thanks @toan. I believe it looks all good. Please proceed with retiring the old instance.

• toan claimed this task.Jun 30 2022, 7:07 AM

• toan updated the task description. (Show Details)Jun 30 2022, 7:19 AM

Mentioned in SAL (#wikimedia-cloud) [2022-06-30T07:23:38Z] <toan> stretch instance deleted T306105

• toan moved this task from Review to Done on the Wikibase Cloud (Launch Migration Kanban (2022)) board.Jun 30 2022, 7:24 AM

Boldly resolving this task.

Cloud VPS "wikibase-registry" project Stretch deprecationClosed, ResolvedPublicActions

Description

Details

Event Timeline

Cloud VPS "wikibase-registry" project Stretch deprecation
Closed, ResolvedPublic
Actions