Page MenuHomePhabricator
Create Task
Maniphest T409528

Setup a maps staging DB
Open, Needs TriagePublic
Actions

Description

For the migration of maps to Bookworm (T381565) we had used six former Ganeti nodes as test servers.

We'll keep maps-test2001 around as a separate staging system (single PG master, no replicas, a separate role::maps::staging will be used (https://gerrit.wikimedia.org/r/c/operations/puppet/+/1201077)) to eventually point the wikikube staging endpoint to it. This will allow us to test changes like https://gerrit.wikimedia.org/r/c/mediawiki/services/kartotherian/+/1201020 or https://phabricator.wikimedia.org/T407491 more systematically w/o risk to the main production setup.

It's an old Ganeti node, but it should be good for the remainder of the FY and we will refresh it with new hardware in the next one.

Details

Related Changes in Gerrit:
Show related patches Customize query in gerrit

Related Objects

Event Timeline

MoritzMuehlenhoff created this task.Nov 7 2025, 8:03 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 7 2025, 8:03 AM
elukey added a comment.Thu, Nov 13, 2:55 PM

Created a new bucket with swift post and the Tegola AUTH credentials on thanos-fe1004:

root@thanos-fe1004:~# swift stat tegola-swift-staging-codfw-v001
                      Account: AUTH_tegola
                    Container: tegola-swift-staging-codfw-v001
                      Objects: 0
                        Bytes: 0
                     Read ACL:
                    Write ACL:
                      Sync To:
                     Sync Key:
                 Content-Type: application/json; charset=utf-8
                  X-Timestamp: 1763045396.18537
                Last-Modified: Thu, 13 Nov 2025 14:49:57 GMT
gerritbot added a comment.Thu, Nov 13, 4:02 PM

Change #1204915 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[labs/private@master] Add stub secrets for the staging role

https://gerrit.wikimedia.org/r/1204915

gerritbot added a project: Patch-For-Review.Thu, Nov 13, 4:02 PM
gerritbot added a comment.Thu, Nov 13, 4:11 PM

Change #1204915 merged by Muehlenhoff:

[labs/private@master] Add stub secrets for the staging role

https://gerrit.wikimedia.org/r/1204915

elukey added a comment.Thu, Nov 13, 4:12 PM

After a chat with Moritz we realized that the better path is probably to create another account for staging, and create the new container in there. In this way we fully disentangle prod from staging, and we don't risk to mess up prod tiles when working in staging.

Maintenance_bot removed a project: Patch-For-Review.Thu, Nov 13, 4:30 PM
gerritbot added a comment.Thu, Nov 13, 4:32 PM

Change #1204926 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[labs/private@master] Add missing secret

https://gerrit.wikimedia.org/r/1204926

gerritbot added a project: Patch-For-Review.Thu, Nov 13, 4:32 PM
gerritbot added a comment.Thu, Nov 13, 4:42 PM

Change #1204926 merged by Muehlenhoff:

[labs/private@master] Add missing secret

https://gerrit.wikimedia.org/r/1204926

Maintenance_bot removed a project: Patch-For-Review.Thu, Nov 13, 5:32 PM
gerritbot added a comment.Fri, Nov 14, 7:23 AM

Change #1205009 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Set (initially stub) Swift container for Tegola

https://gerrit.wikimedia.org/r/1205009

gerritbot added a project: Patch-For-Review.Fri, Nov 14, 7:23 AM
gerritbot added a comment.Fri, Nov 14, 7:28 AM

Change #1205009 merged by Muehlenhoff:

[operations/puppet@production] Set (initially stub) Swift container for Tegola

https://gerrit.wikimedia.org/r/1205009

Maintenance_bot removed a project: Patch-For-Review.Fri, Nov 14, 7:30 AM
gerritbot added a comment.Fri, Nov 14, 9:21 AM

Change #1205086 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Set (initially) stub Eventgate config for maps/staging

https://gerrit.wikimedia.org/r/1205086

gerritbot added a project: Patch-For-Review.Fri, Nov 14, 9:21 AM
gerritbot added a comment.Fri, Nov 14, 9:31 AM

Change #1205086 merged by Muehlenhoff:

[operations/puppet@production] Set (initially) stub Eventgate config for maps/staging

https://gerrit.wikimedia.org/r/1205086

Maintenance_bot removed a project: Patch-For-Review.Fri, Nov 14, 10:31 AM
Stashbot added a comment.Wed, Nov 19, 7:59 AM

Mentioned in SAL (#wikimedia-operations) [2025-11-19T07:59:54Z] <moritzm> started OSM import on maps-test2001 T409528

gerritbot added a comment.Mon, Nov 24, 12:58 PM

Change #1210587 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Enable imports on maps-test2001

https://gerrit.wikimedia.org/r/1210587

gerritbot added a project: Patch-For-Review.Mon, Nov 24, 12:58 PM
gerritbot added a comment.Mon, Nov 24, 2:16 PM

Change #1210598 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/mediawiki-config@master] Add a staging-specific stream for Maps tiles change

https://gerrit.wikimedia.org/r/1210598

gerritbot added a comment.Mon, Nov 24, 2:21 PM

Change #1210599 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/puppet@production] profile::thanos::swift: add tegola account for staging

https://gerrit.wikimedia.org/r/1210599

gerritbot added a comment.Wed, Nov 26, 8:29 AM

Change #1210599 merged by Elukey:

[operations/puppet@production] profile::thanos::swift: add tegola account for staging

https://gerrit.wikimedia.org/r/1210599

gerritbot added a comment.Wed, Nov 26, 9:15 AM

Change #1211609 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/deployment-charts@master] services: move tegola and kartotherian to the new staging db

https://gerrit.wikimedia.org/r/1211609

gerritbot added a comment.Wed, Nov 26, 9:30 AM

Change #1211609 merged by Elukey:

[operations/deployment-charts@master] services: move tegola and kartotherian to the new staging db

https://gerrit.wikimedia.org/r/1211609

gerritbot added a comment.Wed, Nov 26, 10:34 AM

Change #1210587 merged by Muehlenhoff:

[operations/puppet@production] Enable imports on maps-test2001

https://gerrit.wikimedia.org/r/1210587

gerritbot added a comment.Wed, Nov 26, 10:57 AM

Change #1211631 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/deployment-charts@master] services: set new caching and kafka configuration for Tegola staging

https://gerrit.wikimedia.org/r/1211631

gerritbot added a comment.Wed, Nov 26, 2:46 PM

Change #1210598 merged by jenkins-bot:

[operations/mediawiki-config@master] Add a staging-specific stream for Maps tiles change

https://gerrit.wikimedia.org/r/1210598

Stashbot added a comment.Wed, Nov 26, 2:47 PM

Mentioned in SAL (#wikimedia-operations) [2025-11-26T14:47:10Z] <elukey@deploy2002> Started scap sync-world: Backport for [[gerrit:1210598|Add a staging-specific stream for Maps tiles change (T409528)]]

Stashbot added a comment.Wed, Nov 26, 2:49 PM

Mentioned in SAL (#wikimedia-operations) [2025-11-26T14:49:28Z] <elukey@deploy2002> elukey: Backport for [[gerrit:1210598|Add a staging-specific stream for Maps tiles change (T409528)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Stashbot added a comment.Wed, Nov 26, 2:53 PM

Mentioned in SAL (#wikimedia-operations) [2025-11-26T14:53:52Z] <elukey@deploy2002> Finished scap sync-world: Backport for [[gerrit:1210598|Add a staging-specific stream for Maps tiles change (T409528)]] (duration: 06m 41s)

gerritbot added a comment.Wed, Nov 26, 3:09 PM

Change #1211631 merged by Elukey:

[operations/deployment-charts@master] services: set new caching and kafka configuration for Tegola staging

https://gerrit.wikimedia.org/r/1211631

Maintenance_bot removed a project: Patch-For-Review.Wed, Nov 26, 3:30 PM
elukey added a comment.Thu, Nov 27, 10:09 AM

Next steps:

  • Wait for imposm to catch up from OSM upstream.
  • Warm up the Tegola cache on Swift.

After the above two steps the stack will be ready to be used :)

gerritbot added a comment.Wed, Dec 3, 2:25 PM

Change #1214526 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/deployment-charts@master] services: add maps-next.w.o as FQDN for kartotherian staging

https://gerrit.wikimedia.org/r/1214526

gerritbot added a project: Patch-For-Review.Wed, Dec 3, 2:25 PM
MoritzMuehlenhoff added a comment.Thu, Dec 4, 7:45 AM

The initial imposm catchup sync after the PBF import has just completed.

gerritbot added a comment.Thu, Dec 4, 9:43 AM

Change #1215089 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/deployment-charts@master] services: add maps-next.w.o as FQDN for kartotherian staging

https://gerrit.wikimedia.org/r/1215089

gerritbot added a comment.Thu, Dec 4, 9:55 AM

Change #1215098 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/deployment-charts@master] services: add maps-next.w.o as FQDN for kartotherian staging

https://gerrit.wikimedia.org/r/1215098

gerritbot added a comment.Thu, Dec 4, 9:57 AM

Change #1214526 abandoned by Elukey:

[operations/deployment-charts@master] services: add maps-next.w.o as FQDN for kartotherian staging

https://gerrit.wikimedia.org/r/1214526

gerritbot added a comment.Thu, Dec 4, 11:33 AM

Change #1215089 abandoned by Elukey:

[operations/deployment-charts@master] services: add maps-next.w.o as FQDN for kartotherian staging

https://gerrit.wikimedia.org/r/1215089

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits