Page MenuHomePhabricator

Make reportupdater support passing the values of an explode_by using a file path
Closed, ResolvedPublic5 Estimated Story Points

Description

We've seen that different teams use the explode by wiki feature differently. Some, need the all value, some don't, and some can not afford having a full list of all wikis, because their queries take too long to be executed 800+ times per day, and instead use a subset of them. I think it's better to make the feature more generic and let the teams specify their own lists as opposed to have different lists and parametrize each one of them. Also, we've seen teams repeating a considerably large list of values to explode in each one of the reports. This feature would also reduce that code to something more maintainable.

Example:

reports:
    some_report:
        granularity: days
        starts: 2016-01-01
        explode_by:
            wiki: list_of_wikis.txt

Maybe reportupdater could assume that a value list with a single value means a file path.

Event Timeline

Milimetric triaged this task as Medium priority.
Milimetric set the point value for this task to 5.

Change 306966 had a related patch set uploaded (by Mforns):
Support passing the exploded values by file path

https://gerrit.wikimedia.org/r/306966

Change 306968 had a related patch set uploaded (by Mforns):
Disable the deprecated option by_wiki

https://gerrit.wikimedia.org/r/306968

Change 307243 had a related patch set uploaded (by Mforns):
Make use of the new explode by file feature

https://gerrit.wikimedia.org/r/307243

Change 307244 had a related patch set uploaded (by Mforns):
Make use of the new explode by file feature

https://gerrit.wikimedia.org/r/307244

Change 307273 had a related patch set uploaded (by Mforns):
Make use of the new explode by file feature

https://gerrit.wikimedia.org/r/307273

Change 307274 had a related patch set uploaded (by Mforns):
Make use of the new explode by file feature

https://gerrit.wikimedia.org/r/307274

Ok, this can be reviewed.

  • The first patch adds the new feature to reportupdater, with backwards compat.
  • The patches 2, 3, 4 and 5 change the query repos to make use of the feature.
  • The last patch removes support to the old format from reportupdater.

They should be merged in this same order.

reviewing now. btw, to ensure changes get merged in a certain order, you can just chain them in gerrit (make them on top of each other)

Change 306966 merged by Milimetric:
Support passing the exploded values by file path

https://gerrit.wikimedia.org/r/306966

Change 307243 merged by jenkins-bot:
Make use of the new explode by file feature

https://gerrit.wikimedia.org/r/307243

Change 307244 merged by Milimetric:
Make use of the new explode by file feature

https://gerrit.wikimedia.org/r/307244

Change 307273 merged by Milimetric:
Make use of the new explode by file feature

https://gerrit.wikimedia.org/r/307273

Change 306968 merged by Milimetric:
Disable the deprecated option by_wiki

https://gerrit.wikimedia.org/r/306968

Change 307274 merged by Mforns:
Make use of the new explode by file feature

https://gerrit.wikimedia.org/r/307274