Page MenuHomePhabricator

Set up cron task to regenerate low-zoom vector tiles
Closed, ResolvedPublic

Description

z10+ is regenerated through OSM updates, but we should also regenerate z0-z9 through a cron job.

tile.osm.org regenerates z0-z12 monthly.

Event Timeline

Pnorman updated the task description. (Show Details)
Pnorman added subscribers: jmatazzoni, Gehel, mxn.

It takes under 4 hours to do z0-z9

Vvjjkkii renamed this task from Set up cron task to regenerate low-zoom vector tiles to kwcaaaaaaa.Jul 1 2018, 1:09 AM
Vvjjkkii triaged this task as High priority.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
WhitePhosphorus renamed this task from kwcaaaaaaa to Set up cron task to regenerate low-zoom vector tiles.Jul 1 2018, 2:29 AM
WhitePhosphorus raised the priority of this task from High to Needs Triage.
WhitePhosphorus updated the task description. (Show Details)
WhitePhosphorus added a subscriber: Aklapper.
Pnorman triaged this task as High priority.Jul 3 2018, 4:10 PM

Change 447851 had a related patch set uploaded (by MSantos; owner: MSantos):
[operations/puppet@production] Set up cron task to regen low-zoom vector tiles

https://gerrit.wikimedia.org/r/447851

Change 447851 merged by Gehel:
[operations/puppet@production] Set up cron task to regen low-zoom vector tiles

https://gerrit.wikimedia.org/r/447851

Change 449719 had a related patch set uploaded (by MSantos; owner: MSantos):
[operations/puppet@production] Set up cron task to regen low-zoom vector tiles

https://gerrit.wikimedia.org/r/449719

Change 449719 merged by Gehel:
[operations/puppet@production] Set up cron task to regen low-zoom vector tiles

https://gerrit.wikimedia.org/r/449719

The tiles are being generated fine. The script works and now we just have to check the cron one more time.

But, this is not what the issue that generated this task is about. The tiles are updated at all zoom levels, but there is some problem with the language fallback that doesn't change the issue, please see the links above:

Updated Map
Snapshot img z10
Snapshot img z9 with lang vi (and the issue)
Snapshot img z9 without lang vi

I tried to purge the URL thinking that it could be some varnish stuff, but I got no luck.

Change 452467 had a related patch set uploaded (by MSantos; owner: MSantos):
[operations/puppet@production] Changing day of the cron for testing

https://gerrit.wikimedia.org/r/452467

Change 452467 merged by Gehel:
[operations/puppet@production] Changing day of the cron for testing

https://gerrit.wikimedia.org/r/452467

Log says that the script executed successfully, Tilerator UI doesn't have any new failed jobs and monitors looks normal [1] [2] [3].

[1] https://grafana.wikimedia.org/dashboard/db/service-kartotherian?orgId=1&from=1534356000000&to=1534363200000
[2] https://grafana.wikimedia.org/dashboard/db/service-tilerator?orgId=1&from=1534356000000&to=1534363200000
[3] https://logstash.wikimedia.org/app/kibana#/dashboard/cassandra-maps?_g=h@1251ff0&_a=h@5107ef2

@Jhernandez and @Mholloway, tile regeneration is fine. We just have to take a look at the real issue which seems to be related to cache, as mentioned before.

Thanks @MSantos, can you follow up on the talk page with the discussion about what we've done here and ask them to create a new ticket for the issue they are having? This task as presented should be done so it should be resolved.

Thanks for the work and the explanations 👍