Page MenuHomePhabricator

Beta Cluster EventLogging data is disappearing?
Closed, ResolvedPublic0 Estimate Story Points

Description

I don't have the details yet, but according to rumor, eventlogging data is not being recorded to the database.

Event Timeline

eventlogging and database have been moved to deployment-eventlogging03 instance. Needed to upgrade from Precise to Trusty. Please use that instance now, deployment-eventlogging02 will be removed.

I apologize for the lack of an announcement on this. What is the proper channel?

greg renamed this task from Betawiki EventLogging data is disappearing? to Beta Cluster EventLogging data is disappearing?.Sep 17 2015, 7:25 PM
greg removed a project: Cloud-Services.
greg set Security to None.

Many thanks @Ottomata!! K I see it now in /var/log/eventlogging/all-events.log. It's not getting through to MySQL though, but I think that's an issue with nested schemas (as per this log that @Milimetric found, not sure exactly where, though).

greg added a subscriber: greg.Sep 17 2015, 7:28 PM

What is the proper channel?

Ok, thanks. I think I migrated this service very hastily, since we were changing puppetization for production, which broke in labs because the changes didn't work on Precise.

Next time will do that. Can I email qa@lists or do I need to join?

greg added a comment.Sep 17 2015, 7:33 PM

I think it let's anyone email it. I clean the backlog daily, fwiw (thanks to listadmin).

DStrine moved this task from Pending Deployment to Done on the Fundraising Sprint Tom Waits board.
DStrine moved this task from Done to Doing on the Fundraising Sprint Tom Waits board.
greg added a subscriber: dduvall.Sep 18 2015, 10:11 PM

Please use that instance now, deployment-eventlogging02 will be removed.

@Ottomata: https://tools.wmflabs.org/sal/log/AU_igRQw1oXzWjit5mYu
"deployment-eventlogging02:/var full due to massive eventlogging_processor-server-side-0.log.1 ("(MainThread) Unable to process" errors)"

Restricted Application added a subscriber: Luke081515. · View Herald TranscriptSep 18 2015, 10:11 PM

Just deleted deployment-eventlogging02

Thanks so much! Closing... For MySQL querying of the banner history events, only T112947 is still out there... :)

AndyRussG closed this task as Resolved.Sep 21 2015, 1:46 PM
DStrine edited a custom field.Sep 30 2015, 10:36 PM
mmodell removed a subscriber: awight.Jun 22 2017, 9:44 PM
Restricted Application added a project: Analytics. · View Herald TranscriptJun 22 2017, 9:44 PM
Restricted Application added subscribers: Jay8g, TerraCodes. · View Herald Transcript