Page MenuHomePhabricator

"JobExecutor not loaded" error for BounceHandlerJob on wikitech.wikimedia.org
Open, LowPublic

Description

Error

Request ID: XPfF4QpAMD4AADNSoHcAAAAX

message
Exception from line 68 of /srv/mediawiki/rpc/RunSingleJob.php:

JobExecutor not loaded for job:
{ "database":"labswiki",
  "mediawiki_signature":,
  "meta":{"domain":"wikitech.wikimedia.org","dt":"2019-06-05T13:38:41+00:00","id":"request_id":"XPfF4QpAMD4AADNSoHcAAAAX","schema_uri":"mediawiki\/job\/3","topic":"mediawiki.job.BounceHandlerJob","uri":"https:\/\/placeholder.invalid\/wiki\/Special:Badtitle"},
  "page_namespace":-1,"page_title":":","params":{"email":},
  "type":"BounceHandlerJob"
}

Impact

Some kinds of user actions on Wikitech wiki are presumably not working as intended.

Notes

Breakdown of last 30 days:

Event Timeline

Krinkle created this task.Jun 5 2019, 2:04 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 5 2019, 2:04 PM
Pchelolo added a subscriber: Pchelolo.

Wikitech should not be using kafka job queue at all per T192361#4139799

In mediawiki-config we set wmgUseClusterJobqueue to false for wikitech, thus it should be using the JobQueueDB. Plus we set the wikitech group to send events of TYPE_NONE - which means no events should be produced, so no jobs should get into kafka even if the queue is incorrect.

Something is clearly not working as expected, will investigate.

I think the root cause is the same as T208922: PHP Fatal Error: Class undefined: JobExecutor (jobrunners try to run labswiki jobs), namely T208922#4766050 (global config problems when spawning inter-wiki jobs). It's just that now we detect the case when JobExecutor is not loaded instead of bluntly crashing.

WDoranWMF triaged this task as Low priority.Jul 17 2019, 8:14 PM
mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:06 PM
Krinkle moved this task from Untriaged to EventBus infra on the WMF-JobQueue board.Sep 9 2019, 9:03 PM
Krinkle removed a subscriber: Krinkle.Tue, Oct 8, 5:18 PM