Page MenuHomePhabricator

Bug: geoeditors (editors per country data) 2019-06 snapshot broken
Closed, ResolvedPublic5 Story Points

Description

I (@Milimetric) incorrectly understood that the cu_changes table was changing its cuc_user and cuc_user_text fields. I updated the sqoop statement accordingly and caused a bad import. The fields are not changing and need to be reverted to select as before. The import needs to be repeated for this past month and dependent jobs need to be rerun.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 11 2019, 7:05 PM

Change 522169 had a related patch set uploaded (by Milimetric; owner: Milimetric):
[analytics/refinery@master] Revert sqoop select for cu_changes

https://gerrit.wikimedia.org/r/522169

Nuria added subscribers: Asaf, Nuria.Jul 11 2019, 7:11 PM

pinging @Asaf so he knows we are rerunning data for June 2019

Change 522169 merged by Milimetric:
[analytics/refinery@master] Revert sqoop select for cu_changes

https://gerrit.wikimedia.org/r/522169

Nuria renamed this task from Bug: geoeditors 2019-06 snapshot broken to Bug: geoeditors (editors per country data) 2019-06 snapshot broken.Jul 11 2019, 7:21 PM
Ijon added a subscriber: Ijon.Jul 11 2019, 7:23 PM

Thanks!

Milimetric moved this task from Next Up to Ready to Deploy on the Analytics-Kanban board.

All jobs in the pipeline have been rerun, fresh data is accessible from superset and everywhere else. Apologies for the inconvenience.

For reference, the last job in the pipeline was the one that loads data into Druid and this is the newly restarted coordinator: https://hue.wikimedia.org/oozie/list_oozie_coordinator/0016892-190626064919032-oozie-oozi-C/

Milimetric triaged this task as High priority.Jul 15 2019, 3:13 PM
Milimetric moved this task from Incoming to Ops Week on the Analytics board.
Nuria closed this task as Resolved.Jul 18 2019, 8:39 PM
Nuria set the point value for this task to 5.
Ijon reopened this task as Open.Jul 19 2019, 1:07 PM

I have just tried to access geodata for Wikidata, and I get "no data" for "last month" and "last week". The rest works.

Nuria added a comment.Jul 19 2019, 3:04 PM

Looking into this

Nuria moved this task from Done to In Progress on the Analytics-Kanban board.Jul 19 2019, 3:05 PM
Nuria added a comment.EditedJul 19 2019, 3:29 PM

@Ijon see data under "wikidatawiki" which is the db name for wikidata (confusing i know, given project name). Did you try that wiki name?

Nuria added a comment.Jul 22 2019, 8:46 PM

pinging @Ijon to see if my comment made sense

Nuria moved this task from In Progress to Paused on the Analytics-Kanban board.Jul 22 2019, 8:46 PM
Milimetric removed Milimetric as the assignee of this task.Jul 25 2019, 4:50 PM
Nuria closed this task as Resolved.Jul 25 2019, 4:55 PM