Page MenuHomePhabricator

Move misc cron jobs from snapshot1003 to new server(s)
Closed, ResolvedPublic

Description

Dumps run on the new snapshot hosts (1005-1007) but the misc cron jobs do not yet. They need clean up, in some cases rewrite to use the same base script instead of several different scripts, and the puppet classes need cleanup as well. Then a role for the misc cron jobs could be applied to the new servers.

Waiting for verification that cron runs properly:

  • media list generation per project
  • page title list generation
  • cirrus search dumps
  • central auth dumps
  • generation of lists of last n good dumps
  • wikidata dumps - runs Mon

Event Timeline

ArielGlenn triaged this task as Normal priority.Apr 26 2016, 2:18 PM
ArielGlenn created this task.
ArielGlenn moved this task from Backlog to Active on the Dumps-Generation board.
ArielGlenn moved this task from Active to Up Next on the Dumps-Generation board.Apr 26 2016, 3:19 PM
ArielGlenn added a comment.EditedJul 25 2016, 10:43 AM

https://gerrit.wikimedia.org/r/#/c/300437/ fixes up onallwikis script so it can be used for more of these misc cronjobs

ArielGlenn updated the task description. (Show Details)Jul 25 2016, 11:32 AM
ArielGlenn moved this task from Up Next to Active on the Dumps-Generation board.Jul 25 2016, 12:30 PM
ArielGlenn updated the task description. (Show Details)Jul 25 2016, 12:33 PM
ArielGlenn updated the task description. (Show Details)
ArielGlenn updated the task description. (Show Details)Jul 25 2016, 12:57 PM

https://gerrit.wikimedia.org/r/#/c/300865/ merged to move cirrusdumps to snapshot1007.

ArielGlenn updated the task description. (Show Details)Jul 25 2016, 1:24 PM

Change 300878 merged by ArielGlenn:
move cron job for central auth dump to snapshot1007

https://gerrit.wikimedia.org/r/300878

ArielGlenn updated the task description. (Show Details)Jul 25 2016, 3:26 PM

Change 300889 had a related patch set uploaded (by ArielGlenn):
move generation of lists of good dumps to snapshot1007

https://gerrit.wikimedia.org/r/300889

Change 300889 merged by ArielGlenn:
move generation of lists of good dumps to snapshot1007

https://gerrit.wikimedia.org/r/300889

ArielGlenn updated the task description. (Show Details)Jul 25 2016, 3:47 PM

Change 300903 had a related patch set uploaded (by ArielGlenn):
move wikidata json, ttl dumps cron job to snapshot1007

https://gerrit.wikimedia.org/r/300903

Change 300903 merged by ArielGlenn:
move wikidata json, ttl dumps cron job to snapshot1007

https://gerrit.wikimedia.org/r/300903

ArielGlenn updated the task description. (Show Details)Jul 25 2016, 5:27 PM
ArielGlenn updated the task description. (Show Details)Jul 26 2016, 10:29 AM
ArielGlenn updated the task description. (Show Details)
ArielGlenn updated the task description. (Show Details)Jul 28 2016, 1:49 AM
ArielGlenn updated the task description. (Show Details)Jul 29 2016, 8:35 AM
ArielGlenn closed this task as Resolved.Jul 30 2016, 9:05 AM
ArielGlenn updated the task description. (Show Details)

All jobs run properly after move so this task can be closed.