Page MenuHomePhabricator

Quiet TimeoutError in celery logging
Closed, ResolvedPublic

Description

[2016-09-25 16:37:17,695: ERROR/MainProcess] Task ores.scoring_systems.celery_queue._lookup_score_in_map[enwiki:reverted:0.1.2:741120316] raised unexpected: TimeoutError('The operation timed out.'

Event Timeline

Halfak created this task.Sep 26 2016, 5:45 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 26 2016, 5:45 PM
Halfak assigned this task to Ladsgroup.Oct 3 2016, 4:52 PM
Halfak triaged this task as Medium priority.
Halfak moved this task from Untriaged to Maintenance/cleanup on the Machine Learning Platform board.
Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptOct 3 2016, 4:53 PM
Restricted Application added a subscriber: TerraCodes. · View Herald TranscriptNov 1 2016, 7:19 PM
Ladsgroup removed Ladsgroup as the assignee of this task.Nov 1 2016, 7:34 PM
Ladsgroup added a subscriber: Ladsgroup.

I plan to mentor this in GCI

@Ladsgroup: Just an error message probably doesn't help a new contributor (~GCI student) to get what he has to do. Do you've some context for this task (e.g. why the error should be hidden, or/and a way to reproduce the problem for testing, if there's any way)? That would make the task more attracktive for students :)

It's more operational, We need to quiet them because they happen too often and beside spamming the error log they cause the system to run out of space and fail until the logrotate kicks in. Building an ORES environment is easy but making it to fail out because of timeout is not :( Reproducing it is hard but solving it is easy they need to change celery configs so it ignores Timeout errors.

Great, thanks! :) That sounds more than resoanble and like a student can figure out what he has to do :)

-> https://codein.withgoogle.com/dashboard/tasks/5083639217913856/
@Ladsgroup It seems that you're not registered as a mentor in Google code-in, do you want to do that? If not, I'll add myself as a mentor and woul love, if you would review any incoming work from GCI students :P

Dargasea claimed this task.EditedNov 28 2016, 8:51 PM
Dargasea added a subscriber: Dargasea.

I will be working on this as part of GCI 2016.

Dargasea removed Dargasea as the assignee of this task.Nov 30 2016, 1:28 AM
Dargasea closed this task as Resolved.Nov 30 2016, 2:45 AM
Dargasea assigned this task to Halfak.
This comment was removed by Dargasea.
Dargasea reopened this task as Open.Nov 30 2016, 2:47 AM

PR 179 should solve this. Can you check?

(comment removal was done by accident. content is exactly the same as original ^)

Looks alright. Merged

Looks alright. Merged

...and task in the GCI system is now also closed as resolved. Thank you everybody! :)

Dargasea closed this task as Resolved.Dec 1 2016, 8:10 AM