Page MenuHomePhabricator

Set up cron task to regenerate low-zoom vector tiles
Closed, ResolvedPublic

Description

z10+ is regenerated through OSM updates, but we should also regenerate z0-z9 through a cron job.

tile.osm.org regenerates z0-z12 monthly.

Event Timeline

Pnorman created this task.May 15 2018, 6:35 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 15 2018, 6:35 PM
Pnorman updated the task description. (Show Details)
Pnorman added subscribers: jmatazzoni, Gehel, mxn.
Restricted Application added a project: Discovery. · View Herald TranscriptMay 15 2018, 6:38 PM

It takes under 4 hours to do z0-z9

Vvjjkkii renamed this task from Set up cron task to regenerate low-zoom vector tiles to kwcaaaaaaa.Jul 1 2018, 1:09 AM
Vvjjkkii triaged this task as High priority.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
WhitePhosphorus renamed this task from kwcaaaaaaa to Set up cron task to regenerate low-zoom vector tiles.Jul 1 2018, 2:29 AM
WhitePhosphorus raised the priority of this task from High to Needs Triage.
WhitePhosphorus updated the task description. (Show Details)
WhitePhosphorus added a subscriber: Aklapper.
Pnorman triaged this task as High priority.Jul 3 2018, 4:10 PM
Mholloway moved this task from Backlog to To-do on the Maps-Sprint board.Jul 3 2018, 4:56 PM
MSantos moved this task from To-do to In progress on the Maps-Sprint board.Jul 24 2018, 7:04 PM

Change 447851 had a related patch set uploaded (by MSantos; owner: MSantos):
[operations/puppet@production] Set up cron task to regen low-zoom vector tiles

https://gerrit.wikimedia.org/r/447851

MSantos moved this task from In progress to Needs review on the Maps-Sprint board.Jul 26 2018, 3:25 PM

Change 447851 merged by Gehel:
[operations/puppet@production] Set up cron task to regen low-zoom vector tiles

https://gerrit.wikimedia.org/r/447851

Change 449719 had a related patch set uploaded (by MSantos; owner: MSantos):
[operations/puppet@production] Set up cron task to regen low-zoom vector tiles

https://gerrit.wikimedia.org/r/449719

Change 449719 merged by Gehel:
[operations/puppet@production] Set up cron task to regen low-zoom vector tiles

https://gerrit.wikimedia.org/r/449719

MSantos moved this task from Needs review to Done on the Maps-Sprint board.Aug 2 2018, 6:58 PM

The tiles are being generated fine. The script works and now we just have to check the cron one more time.

But, this is not what the issue that generated this task is about. The tiles are updated at all zoom levels, but there is some problem with the language fallback that doesn't change the issue, please see the links above:

Updated Map
Snapshot img z10
Snapshot img z9 with lang vi (and the issue)
Snapshot img z9 without lang vi

I tried to purge the URL thinking that it could be some varnish stuff, but I got no luck.

Change 452467 had a related patch set uploaded (by MSantos; owner: MSantos):
[operations/puppet@production] Changing day of the cron for testing

https://gerrit.wikimedia.org/r/452467

Change 452467 merged by Gehel:
[operations/puppet@production] Changing day of the cron for testing

https://gerrit.wikimedia.org/r/452467

Log says that the script executed successfully, Tilerator UI doesn't have any new failed jobs and monitors looks normal [1] [2] [3].

[1] https://grafana.wikimedia.org/dashboard/db/service-kartotherian?orgId=1&from=1534356000000&to=1534363200000
[2] https://grafana.wikimedia.org/dashboard/db/service-tilerator?orgId=1&from=1534356000000&to=1534363200000
[3] https://logstash.wikimedia.org/app/kibana#/dashboard/cassandra-maps?_g=h@1251ff0&_a=h@5107ef2

@Jhernandez and @Mholloway, tile regeneration is fine. We just have to take a look at the real issue which seems to be related to cache, as mentioned before.

Jhernandez closed this task as Resolved.Aug 17 2018, 1:37 PM

Thanks @MSantos, can you follow up on the talk page with the discussion about what we've done here and ask them to create a new ticket for the issue they are having? This task as presented should be done so it should be resolved.

Thanks for the work and the explanations 👍

@Jhernandez done! Thank you for reviewing it.