Page MenuHomePhabricator

Fix 1-week lag in executing browser reports {lama}
Closed, ResolvedPublic8 Estimated Story Points

Description

The browser reports are executing with a 1-week delay.
That is caused by a bug in reportupdater that activates when the report config is using large frequency (weeks) together with the lag option.

This causes reportupdater to:

  1. try to execute the report
  2. find out it should not execute the report yet, because of the lag
  3. storing the execution time in the history file anyway
  4. the next execution will wait one week because the history file has already one execution within this week.

Event Timeline

I edited the task's description and removed the 'solution' paragraph, that was:

Solution:
Apply the `lag` offset in a different moment in the pipeline. Not in `get_interval_reports` but in `is_time_to_execute`.

This does not work, it is not possible in the way reportupdater schedules executions.
Instead, I think the best way would be:

  1. Remove reportupdater's feature that updates the last data-point at each execution. So that, the last data point is only calculated once.
  2. Configure the browser reports with frequency=days
  3. Potentially remove the frequency parameter from the config, which is a bit confusing (but should check if some reports are making actual use of that).

Change 280201 had a related patch set uploaded (by Mforns):
[WIP] Remove dynamic last data point

https://gerrit.wikimedia.org/r/280201

I pushed a change for review. It contains some modifications to RU that will fix the problem and remove some unused complexity from the codebase.

When merging this change set, it's expected that all current RU reports in all query repositories (reportupdater-queries, limn-language-data, limn-edit-data, etc...) continue to work normally; With the exception of 1 report in limn-mobile-data (mobile-options-last-3-months) which will stop updating until some small changes are applied to its configuration. However, this report is currently not behaving as expected, so stopping its execution a couple days, won't be an issue.

Change 280201 merged by Milimetric:
Remove dynamic last data point

https://gerrit.wikimedia.org/r/280201

Nuria set the point value for this task to 8.Apr 7 2016, 4:36 PM