Page MenuHomePhabricator

Re-import full planet data into eqiad
Open, HighPublic

Description

Background information

Re-import full planet data into maps1009.

pre-import checks

  • Is there enough free space?

executing the import

  • disable disable_replication_service and disable_tile_generation_cron in hieradata
  • execute initial-imposm-import script using the following command
# change -d value accordingly when performing the import with the most updated values
sudo -s
screen -DR
imposm-initial-import -d 220103 -x webproxy.eqiad.wmnet:8080
  • monitor the full planet import
    • Did the script finished properly?
    • Are the logs sufficient?

When the import is done, the data won't be available in production, the data will be in the import schema

  • execute check scripts?
  • deploy data to the public schema by running:
sudo -u osmupdater imposm-deploy-import
  • monitor and check the APIs, if needed rollback with the following command
sudo -u osmupdater imposm-rollback-import
  • re-enable the replication service and check for its health
  • re-enable the tile-generation cron and check for its health
  • clean-up old data in backup
sudo -u osmupdater imposm-removebackup-import

Open questions

  • How much SRE intervention will be needed?

Acceptance criteria

  • DB is refreshed with most updated data

Event Timeline

Change 753453 had a related patch set uploaded (by MSantos; author: MSantos):

[operations/puppet@production] maps: disable every OSM cron to perform re-import of data

https://gerrit.wikimedia.org/r/753453

Change 753453 merged by Hnowlan:

[operations/puppet@production] maps: disable every OSM cron to perform re-import of data

https://gerrit.wikimedia.org/r/753453

Mentioned in SAL (#wikimedia-operations) [2022-01-12T21:28:39Z] <mbsantos> mbsantos@maps1009.eqiad.wmnet: start imposm-initial-import - full planet re-import (T299049)

MSantos updated the task description. (Show Details)