Page MenuHomePhabricator
Create Task
Maniphest T161571

process-control streams to log
Closed, ResolvedPublic2 Estimated Story Points
Actions

Description

This is harder than it looks. Popen.communicate buffers the entire output, so we have to build our own non-blocking IO loop.

Solve stdout/stderr synchronization as a bonus!

And timestamps!

Details

SubjectRepoBranchLines +/-
Stream subprocess stdout and stderr to log filewikimedia/fundraising/process-controlmaster+111 -57
Customize query in gerrit

Related Objects

Event Timeline

awight created this task.Mar 27 2017, 11:16 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 27 2017, 11:16 PM
• ggellerman triaged this task as Medium priority.Mar 28 2017, 9:13 PM
• ggellerman moved this task from Triage to Sprint +1 on the Fundraising-Backlog board.
gerritbot subscribed.Mar 29 2017, 9:36 PM

Change 345424 had a related patch set uploaded (by Ejegg):
[wikimedia/fundraising/process-control@master] Stream subprocess stdout and stderr to log file

https://gerrit.wikimedia.org/r/345424

gerritbot added a project: Patch-For-Review.Mar 29 2017, 9:36 PM
Ejegg mentioned this in rWFPC537267efa827: Stream subprocess stdout and stderr to log file.Mar 29 2017, 9:37 PM
Ejegg mentioned this in rWFPCd800083bda96: Stream subprocess stdout and stderr to log file.Mar 29 2017, 9:40 PM
Ejegg mentioned this in rWFPCb4ca76766934: Stream subprocess stdout and stderr to log file.Mar 29 2017, 9:45 PM
gerritbot added a comment.Mar 29 2017, 10:03 PM

Change 345424 merged by jenkins-bot:
[wikimedia/fundraising/process-control@master] Stream subprocess stdout and stderr to log file

https://gerrit.wikimedia.org/r/345424

Ejegg claimed this task.Mar 29 2017, 10:07 PM
Ejegg moved this task from Sprint +1 to Current Sprint and Completed in Q3 2016-17 on the Fundraising-Backlog board.
Ejegg added projects: Fundraising Sprint Far Beer, Unplanned-Sprint-Work.
Ejegg updated the task description. (Show Details)
Ejegg set the point value for this task to 2.
Ejegg moved this task from Backlog to Done on the Fundraising Sprint Far Beer board.Mar 29 2017, 10:11 PM
Ejegg moved this task from Done to Pending Deployment on the Fundraising Sprint Far Beer board.
• DStrine added a project: Fundraising Sprint Gondwanaland Reunification Engine.Mar 29 2017, 10:48 PM
awight mentioned this in T161569: [Epic] Write basic process-control, something good enough to run all CRM jobs..Mar 30 2017, 6:45 PM
Ejegg moved this task from Backlog to Pending Deployment on the Fundraising Sprint Gondwanaland Reunification Engine board.Mar 30 2017, 7:01 PM
awight added a subscriber: • cwdent.Apr 3 2017, 8:26 PM

@cwdent found some libraries that might simplify our code:
https://docs.python.org/3/library/asyncio.html
https://pypi.python.org/pypi/trollius/

awight closed this task as Resolved.Apr 6 2017, 5:56 PM
awight removed a project: Patch-For-Review.
• DStrine moved this task from Current Sprint and Completed in Q3 2016-17 to Completed in Q4 2016-17 on the Fundraising-Backlog board.Apr 14 2017, 10:26 PM
• mmodell removed a subscriber: awight.Jun 22 2017, 9:32 PM
• DStrine moved this task from Pending Deployment to Done on the Fundraising Sprint Gondwanaland Reunification Engine board.Aug 29 2017, 9:09 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits