Page MenuHomePhabricator

Make jobs not time out in 30 seconds
Open, Needs TriagePublic

Description

As discussed in T219897: [4 hours] Event Metrics updates time out after 30 seconds, we currently run most jobs via web requests, which limits them to 30 seconds. Out of 2 solutions discussed, I'm intentionally not requiring any particular one, however something needs to be done.

Event Timeline

MaxSem created this task.Apr 9 2019, 12:15 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 9 2019, 12:15 AM
MaxSem updated the task description. (Show Details)Apr 9 2019, 12:20 AM

I thought the solution identified in T219897#5089655 and then T219897#5095493 was to put this on a cron job. But now @MaxSem now says that he and @MusikAnimal are still debating which of "2 solutions discussed" to pursue.

@aezell, can you perhaps please guide this discussion to a solution, so we can get to work on it? Gentlemen, your ideas?

The comment that apparently went unread was at T219897#5095818

I think if we use set_time_limit with say, 2 hours (2 because we might rework the job state logic), that safely prevents the process from timing out. The Apache-ran updates won't exceed more than 4 or 5 processes, and the rest automatically go to the job queue. So, between all of that and T217911: Performance improvement: Fetch pageviews asynchronously, I suspect all of our "event timed out" examples will finish in record breaking time.

MaxSem claimed this task.Apr 9 2019, 3:00 AM
MaxSem moved this task from Ready to Needs Review/Feedback on the Community-Tech-Sprint board.

I've created a PR for time limit here, meanwhile we can continue bikeshedding about Apaches being used as job runners

aezell added a comment.Apr 9 2019, 1:55 PM

@jmatazzoni Seems like was resolved for the time being.

It does seem like there are some issues with Apache processes being used like this but also issues with cron jobs which we can continue to discuss as Max said.

I hope this gets us to an OK place for this session of working on EM.

jmatazzoni moved this task from Backlog to High value ideas on the Event Metrics board.

There's no new developments I know of to test with this at this time so I'm pulling it from the board until such time as there's something to test.

Restricted Application added a project: Community-Tech. · View Herald TranscriptApr 15 2019, 11:06 PM