Page MenuHomePhabricator

Rerun pingback reports to categorize software versions correctly.
Open, HighPublic

Description

The reports backing https://pingback.wmflabs.org/#media-wiki-version need to be rerun from 2018-10-24 to correctly populate the MediaWiki version data for MediaWiki 1.33 and later.

The reports backing https://pingback.wmflabs.org/#php-version/php-version-timeseries and the associated drilldown reports need to be rerun from 2018-06-10 to correctly populate PHP version data for PHP 7.3 and later.

See discussion on https://gerrit.wikimedia.org/r/c/analytics/reportupdater-queries/+/545917 and https://gerrit.wikimedia.org/r/c/analytics/reportupdater-queries/+/548306.

Details

Related Gerrit Patches:
analytics/reportupdater-queries : masterUpdate php query to include PHP 7.3 and 7.4.
analytics/reportupdater-queries : masterMove back the start date for pingback php_drilldown

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptThu, Nov 14, 11:23 PM
mforns added a subscriber: mforns.Fri, Nov 15, 3:35 PM

Thanks for creating this task @CCicalese_WMF! I completely forgot about the reruns.
I already deleted the data that needs rerun. Reportupdater will start computing the missing data in the next hour.
As there are lots of data points to rerun and the queries are non trivial, this computation might take some days, possibly 1 week.
Let's close this task only when all data is visible and OK in the Dashboard :]
Cheers!

Ottomata assigned this task to mforns.Mon, Nov 18, 4:42 PM
Ottomata triaged this task as High priority.
Ottomata added a project: Analytics-Kanban.
Ottomata moved this task from Incoming to Data Quality on the Analytics board.

Change 551621 had a related patch set uploaded (by Cicalese; owner: Cicalese):
[analytics/reportupdater-queries@master] Update php query to include PHP 7.3 and 7.4.

https://gerrit.wikimedia.org/r/551621

It looks like rerunning the queries has completed, and they look good! However, I had forgotten that the php query is separate from the php_drilldown query. The patch linked above should fix that. Once it is merged, the php query will need to be rerun as well. Thanks!

mforns moved this task from Next Up to In Progress on the Analytics-Kanban board.Tue, Nov 19, 3:21 PM

Change 551847 had a related patch set uploaded (by Mforns; owner: Mforns):
[analytics/reportupdater-queries@master] Move back the start date for pingback php_drilldown

https://gerrit.wikimedia.org/r/551847

Change 551847 merged by Mforns:
[analytics/reportupdater-queries@master] Move back the start date for pingback php_drilldown

https://gerrit.wikimedia.org/r/551847

Change 551621 merged by Mforns:
[analytics/reportupdater-queries@master] Update php query to include PHP 7.3 and 7.4.

https://gerrit.wikimedia.org/r/551621

@CCicalese_WMF
I merged your patch, and also fixed the start date of the php_drilldown reports in another change.
This was needed for the older dates of the reports to be fixed.
It's not imediate to see, but many graphs in the dashboard have still re-run holes.
The last changes will fix them, though.
And I hope in a couple days of re-run, we can call this done :]
Thanks for the patience!

@CCicalese_WMF I think all re-runs are done!
I checked the dashboard and couldn't find any missing data point. Please do check as well!
I will move this to done, please ping if you see something odd.
Thanks!

mforns moved this task from In Progress to Done on the Analytics-Kanban board.Thu, Nov 21, 5:16 PM

@mforns Thank you! I think the reports supporting https://pingback.wmflabs.org/#php-version still need to be re-run, though. The drilldown reports are fine, but that top level graph stops at the end of 2018. The other graphs looks great!

@CCicalese_WMF Oh! Yes, actually nothing's wrong, but RU is still re-runing... I see now that the php_drilldown for v1.34 is still calculating.
After that the php one (without drilldown will re-run).
OK, will move the task back to IN PROGRESS, and check tomorrow.
Thanks!

mforns moved this task from Done to In Progress on the Analytics-Kanban board.Thu, Nov 21, 5:50 PM
mforns moved this task from In Progress to Done on the Analytics-Kanban board.Fri, Nov 22, 3:33 PM

@CCicalese
OK, *now* the dashboard looks good.
Thanks for the patience!

@mforns It looks great! Thank you!