Backfill eventlogging-sourced data
Closed, ResolvedPublic


The MySQL databases for EventLogging recently stopped replicating. No data was lost, but no data was generated for the dashboards, either. Now that we're getting _some_ of it backfilled, go through and fix up those dashboards that we can.

Ironholds_backup updated the task description. (Show Details)
Ironholds_backup raised the priority of this task from to Needs Triage.
Ironholds_backup added a subscriber: Ironholds_backup.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptJan 15 2016, 6:35 PM

Partial backfilling, to the portal data, done. Other schemas are still lagging.

mpopov added a subscriber: mpopov.Jan 18 2016, 9:57 PM

Update: I backfilled all the search/* datasets this morning.

Deskana triaged this task as Normal priority.Jan 19 2016, 9:06 PM
Deskana added a subscriber: Deskana.

@mpopov no ya didn't. The desktop sets lacked the 16th-17th. Now fixed.

Recurring bug in run() whenever I try to backfill: "Error in if (file == "") { : argument is of length zero"

It processes certain datasets/scripts but errors at some point. Can you please look into it?

That's...confusing. I don't see any if(file) check. And it shouldn't be erroring at all due to the tryCatch!

Could you provide the date that's erroring?

I got that error when running it for 2016-01-04 / 2016-01-05

It might be trying to read a file or source a script that doesn't exist.

Ironholds_backup removed Ironholds as the assignee of this task.
mpopov added a comment.EditedJan 20 2016, 11:02 PM

@mpopov no ya didn't. The desktop sets lacked the 16th-17th. Now fixed.

Actually I did but then backfilled other dates which is why you didn't see the data for 16th and 17th where you expected to see it. It's silly.

mpopov claimed this task.Jan 21 2016, 4:09 AM

Ok, seems like we have all the eventlogging-related data backfilled. OK with re-opening it if I missed something.

Deskana closed this task as Resolved.Jan 27 2016, 6:13 PM