Page MenuHomePhabricator
Create Task
Maniphest T126339

set up cron for second monthly dump run
Closed, ResolvedPublic
Actions

Description

The main monthly run is done from cron now. The second one gets juggled around because it's a tight schedule and it depends on failed and rerun jobs, etc. We should add the partial run to cron once we have capacity.

Details

SubjectRepoBranchLines +/-
make sure directory creation uses the dump cron job startdateoperations/puppetproduction+1 -1
extend dumps cron job to run partial dumps as welloperations/puppetproduction+49 -17
add simple variable substitution to dumpscheduler command listsoperations/dumpsmaster+32 -6
Customize query in gerrit

Related Objects
Search...

Event Timeline

ArielGlenn created this task.Feb 9 2016, 3:43 PM
ArielGlenn claimed this task.
ArielGlenn raised the priority of this task from to Medium.
ArielGlenn updated the task description. (Show Details)
ArielGlenn added a project: Dumps-Generation.
ArielGlenn subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 9 2016, 3:43 PM
ArielGlenn added a subtask: T118154: determine hardware needs for dumps in eqiad and codfw.Feb 9 2016, 3:47 PM
ArielGlenn moved this task from Backlog to Up Next on the Dumps-Generation board.
ArielGlenn moved this task from Up Next to Blocked/Stalled/Waiting for event on the Dumps-Generation board.Feb 28 2016, 9:38 PM
RobH closed subtask T118154: determine hardware needs for dumps in eqiad and codfw as Resolved.Mar 3 2016, 11:00 PM
ArielGlenn reopened subtask T118154: determine hardware needs for dumps in eqiad and codfw as Open.Mar 29 2016, 1:21 PM
ArielGlenn moved this task from Blocked/Stalled/Waiting for event to Active on the Dumps-Generation board.Jul 18 2016, 12:10 PM
gerritbot subscribed.Jul 18 2016, 12:27 PM

Change 299527 had a related patch set uploaded (by ArielGlenn):
extend dumps cron script to be able to run partial dumps

https://gerrit.wikimedia.org/r/299527

gerritbot added a project: Patch-For-Review.Jul 18 2016, 12:27 PM
ArielGlenn mentioned this in rOPUPc3ea8d81d3fe: extend dumps cron script to be able to run partial dumps.Jul 18 2016, 12:30 PM
ArielGlenn mentioned this in rOPUPafdae4e876b6: extend dumps cron job to run partial dumps as well.Jul 18 2016, 1:39 PM
gerritbot added a comment.Jul 19 2016, 12:13 PM

Change 299742 had a related patch set uploaded (by ArielGlenn):
add simple variable substitution to dumpscheduler command lists

https://gerrit.wikimedia.org/r/299742

ArielGlenn mentioned this in rOPUPe9ed5c44a502: extend dumps cron job to run partial dumps as well.Jul 19 2016, 12:20 PM
gerritbot added a comment.Jul 19 2016, 2:37 PM

Change 299742 merged by ArielGlenn:
add simple variable substitution to dumpscheduler command lists

https://gerrit.wikimedia.org/r/299742

ArielGlenn mentioned this in rOPUP67a9ceac3ec7: extend dumps cron job to run partial dumps as well.Jul 20 2016, 6:49 AM
gerritbot added a comment.Jul 20 2016, 6:57 AM

Change 299527 merged by ArielGlenn:
extend dumps cron job to run partial dumps as well

https://gerrit.wikimedia.org/r/299527

ArielGlenn mentioned this in rOPUPd99f8a887604: extend dumps cron job to run partial dumps as well.Jul 20 2016, 6:59 AM
gerritbot added a comment.Jul 20 2016, 7:04 AM

Change 299951 had a related patch set uploaded (by ArielGlenn):
make sure directory creation uses the dump cron job startdate

https://gerrit.wikimedia.org/r/299951

gerritbot added a comment.Jul 20 2016, 7:06 AM

Change 299951 merged by ArielGlenn:
make sure directory creation uses the dump cron job startdate

https://gerrit.wikimedia.org/r/299951

ArielGlenn mentioned this in rOPUP28f0354f3930: make sure directory creation uses the dump cron job startdate.Jul 20 2016, 7:09 AM
ArielGlenn added a comment.Jul 20 2016, 8:14 AM

This is running now, leaving the task open til the run completes successfully.

ArielGlenn moved this task from Active to Blocked/Stalled/Waiting for event on the Dumps-Generation board.Jul 20 2016, 8:14 AM
ArielGlenn closed subtask T126341: handle dump run start properly across multiple hosts as Resolved.Jul 25 2016, 12:27 PM
ArielGlenn removed a subtask: T118154: determine hardware needs for dumps in eqiad and codfw.
ArielGlenn closed this task as Resolved.Jul 25 2016, 12:30 PM

That hardware needs ticket must have been left around from when it was just about the new snapshot hosts, long since taken care off. This run is just about complete (last wiki doing last few jobs) so I can close this task.

ArielGlenn moved this task from Blocked/Stalled/Waiting for event to Done on the Dumps-Generation board.Jul 25 2016, 12:30 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits