A new script needs to be written to download and import OSM data
This should also prepare the files for T157611: Add functionality to keep up to date planet and run osmborder to puppet
Updates will also handling. TBD: have one script do both with different options, or independent scripts
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T87268 Copyright license and attribution issues (tracking) | |||
| Invalid | None | T149179 Attribution and copyright tasks for the iOS app (tracking) | |||
| Open | Feature | None | T157763 Replace Apple Maps with a freely licensed alternative | ||
| Invalid | None | T155601 Stabilizing Interactive Products | |||
| Open | None | T113008 Epic: Borders aren't always marked as disputed | |||
| Resolved | Pnorman | T153282 [epic] Migrate to a new vector tile structure | |||
| Declined | None | T156682 Deploying new vector tiles to production | |||
| Resolved | Gehel | T157613 Write new import script | |||
| Resolved | Gehel | T159813 Move border creation/update to its own function |
The current initial import script: https://github.com/wikimedia/operations-puppet/blob/production/modules/role/files/maps/osm-initial-import
@Gehel what Postgres user will the import/updates be done as, and what user does the rendering?
The script that was written for this seems to be working fine at generating borders and loading them into the database. We're still testing this on the full planet.
The script I've been working on is https://gist.github.com/pnorman/17b17e2d76505c7365278cbc1c7af80f
The puppet changes needed will be
Also, post-update stuff with tile expiry might need some adjustment
Worth noting that the verbosity and logging locations will need to be changed so we have logging, notification on errors, and no undesired emails to ops.
Change 345573 had a related patch set uploaded (by Gehel):
[operations/puppet@production] postgresql - simplify creation of databases
Change 345573 merged by Gehel:
[operations/puppet@production] postgresql - simplify creation of databases
Change 378245 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] [WIP] maps: move to vector tiles and cleartables
Change 383841 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] maps: introduce the base requirements for vector tiles and cleartables
Change 383841 merged by Gehel:
[operations/puppet@production] maps: introduce the base requirements for vector tiles and cleartables
With https://phabricator.wikimedia.org/source/operations-puppet/browse/production/modules/osm/files/process-osm-data.sh we have a script, and it works.
Further changes can be their own issue.