Page MenuHomePhabricator

[Volunteer] Generate.py needs to write to a separate history.json file per configured instance
Closed, ResolvedPublic

Description

Currently, there are a few separate jobs sharing limn-mobile-data's generate.py script to run ad-hoc SQL on stat1003: https://github.com/wikimedia/operations-puppet/blob/production/manifests/misc/statistics.pp#L851

But because of the way generate.py works, they are all trying to share the same history.json file: https://github.com/wikimedia/analytics-limn-mobile-data/blob/master/generate.py#L115 which just gets written to the CWD.

So, factor out the path of history.json to a parameter and update puppet to pass in an appropriate path (root of the repository is fine, we can put it in .gitignore)

Event Timeline

Milimetric raised the priority of this task from to Needs Triage.
Milimetric updated the task description. (Show Details)
Milimetric changed Security from none to None.
Milimetric subscribed.
kevinator moved this task from Incoming to Radar on the Analytics-Engineering board.
kevinator renamed this task from Generate.py needs to write to a separate history.json file per configured instance to [Volunteer] Generate.py needs to write to a separate history.json file per configured instance.Dec 9 2014, 5:10 PM

It makes more sense to store history.json in the config folder, which is already passed in as a parameter. No change to statistics.pp should be needed.

Change 180296 had a related patch set uploaded (by Fhocutt):
Save history.json in the config folder

https://gerrit.wikimedia.org/r/180296

Patch-For-Review

Change 180296 merged by Milimetric:
Save history.json in the config folder

https://gerrit.wikimedia.org/r/180296