Page MenuHomePhabricator

[OSM] perform imposm3 migration in production
Open, HighPublic

Description

Background information

The final step for the imposm3 migration in production, planned for beginning of Q3/20-21.

How (imposm3 migration plan)

pre-import checks

  • Run REINDEX for the database in order to decrease disk space and remove dangling indexes
  • Is there enough free space?

Database will have a fresh start in maps1009.eqiad.wmnet

executing the import

This is a checklist for the imposm3 migration. Because it's the first time executing this functionality in production, we need SRE supervision through all the steps. The proposal migration plan avoids the need for depool and repool on the production machines because it doesn't delete data and have a rollback mechanism.

  • merge needed puppet rules https://gerrit.wikimedia.org/r/c/operations/puppet/+/644482
  • apply new puppet rules for maps::osm_master
  • enable imposm parameter in hieradata
  • disable disable_replication_service and disable_tile_generation_cron in hieradata
  • execute initial-imposm-import script using the following command
# change -d and -s values accordingly when performing the import with the most updated values
sudo -u osmupdater imposm-initial-import \
   -d 201019 \ latest dump
   -s  https://planet.openstreetmap.org/replication/hour/000/071/112.state.txt \ hourly replication
   -x webproxy.eqiad.wmnet:8080
  • monitor the full planet import
    • Is there any disk space issue?
    • Did the script finished properly?
    • Are the logs sufficient?

When the import is done, the data won't be available in production, the data will be in the import schema

  • execute check scripts?
  • deploy data to the public schema by running:
sudo -u osmupdater imposm-deploy-import
  • monitor and check the APIs, if needed rollback with the following command
sudo -u osmupdater imposm-rollback-import
  • re-enable the replication service and check for its health
  • re-enable the tile-generation cron and check for its health
  • clean-up old osm2pgsql data running the following SQL script
sudo -u postgres psql -f /etc/imposm/cleanup-old-osm2pgsql-tables.sql
sudo -u osmupdater imposm-removebackup-import

Open questions

  • Is it possible to execute the import without deleting current data? Is there enough disk space for that?
  • How much SRE intervention will be needed?

Acceptance criteria

  • imposm3 replaces osm2pgsql in the production environment

Event Timeline

MSantos created this task.Dec 7 2020, 1:39 PM
MSantos updated the task description. (Show Details)Dec 7 2020, 2:47 PM
sdkim triaged this task as High priority.Dec 7 2020, 2:48 PM
sdkim moved this task from Needs triage to Upcoming on the Product-Infrastructure-Team-Backlog board.

I think we should have an item to check if there are any scheduled tasks so when we disable replication and tile-pregeneration there are no in-flight tasks pending.

Can we easily re-use the diff script we had when we evaluated imposm to sample if the output of the new environment is valid ?

mxn added a subscriber: mxn.Jan 7 2021, 10:29 AM
MSantos assigned this task to hnowlan.Thu, Feb 4, 2:32 PM
MSantos updated the task description. (Show Details)

Can we easily re-use the diff script we had when we evaluated imposm to sample if the output of the new environment is valid ?

I think so, I'll give it a shot.

dcaro added a subscriber: dcaro.Mon, Mar 1, 9:56 AM

The puppet changes introduced in https://gerrit.wikimedia.org/r/c/operations/puppet/+/644482 are breaking the puppet runs on the clouddb host, can you advice (T276039)?