UserLocator.php on line 49
Open, MediumPublicPRODUCTION ERROR
Actions

Assigned To

None

Authored By

	kostajh
	May 10 2021, 12:07 PM

Description

Error

mwversion: 1.37.0-wmf.4
reqId: 315b37df-ed8c-4504-a74a-f1622e74b5d8
Find reqId in Logstash
Find normalized_message in Logstash

normalized_message

[{reqId}] {exception_url}   TypeError: Argument 1 passed to EchoEvent::setTitle() must be an instance of Title, null given, called in /srv/mediawiki/php-1.37.0-wmf.4/extensions/Flow/includes/Notifications/UserLocator.php on line 49

exception.trace

from /srv/mediawiki/php-1.37.0-wmf.4/extensions/Echo/includes/model/Event.php(636)
#0 /srv/mediawiki/php-1.37.0-wmf.4/extensions/Flow/includes/Notifications/UserLocator.php(49): EchoEvent->setTitle(NULL)
#1 /srv/mediawiki/php-1.37.0-wmf.4/extensions/Echo/includes/controller/NotificationController.php(449): Flow\Notifications\UserLocator::locateUsersWatchingTopic(EchoEvent)
#2 /srv/mediawiki/php-1.37.0-wmf.4/extensions/Echo/includes/controller/NotificationController.php(466): EchoNotificationController::evaluateUserCallable(EchoEvent, string)
#3 /srv/mediawiki/php-1.37.0-wmf.4/extensions/Echo/includes/controller/NotificationController.php(116): EchoNotificationController::getUsersToNotifyForEvent(EchoEvent)
#4 /srv/mediawiki/php-1.37.0-wmf.4/extensions/Echo/includes/jobs/NotificationJob.php(13): EchoNotificationController::notify(EchoEvent, boolean)
#5 /srv/mediawiki/php-1.37.0-wmf.4/extensions/EventBus/includes/JobExecutor.php(79): EchoNotificationJob->run()
#6 /srv/mediawiki/rpc/RunSingleJob.php(76): MediaWiki\Extension\EventBus\JobExecutor->execute(array)
#7 {main}

Impact

One or more notifications are not sent to users watching a topic.

Notes

Did a quick look at the relevant section of NotificationController.php, UserLocator.php and Event.php, doesn't seem like anything there changed recently.

Details

Request URL: https://jobrunner.discovery.wmnet/rpc/RunSingleJob.php

Related Objects

Mentioned In: Blog Post: Production Excellence #32: May 2021
Mentioned Here: T142313: Add global information to debug logger context

Event Timeline

kostajh triaged this task as Medium priority.May 10 2021, 12:07 PM

kostajh created this task.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 10 2021, 12:07 PM

kostajh moved this task from Inbox to Needs Discussion on the Growth-Team board.May 10 2021, 12:10 PM

Krinkle moved this task from Untriaged to May 2021 on the Wikimedia-production-error board.May 12 2021, 5:47 PM

Missing notifications seems like a potentially severe error, depending on what the notification is about. Unfortunately for job queue errors like this the stack trace is usually not super useful. (Would be nice to get T142313: Add global information to debug logger context done some day to make error logs like this more useful.) If we want to fix this, or even just figure out what the impact is, we'll have to add extra logging.

Krinkle mentioned this in Blog Post: Production Excellence #32: May 2021.Jun 21 2021, 1:31 AM

There are six instances of this in the last 90 days; three of them are on May 9 and three of them are on June 9, and they follow the same pattern for time distribution:

May 9, 2021 @ 17:21:32.534
May 9, 2021 @ 17:22:32.603
May 9, 2021 @ 17:28:32.676
Jun 9, 2021 @ 21:00:23.269
Jun 9, 2021 @ 21:01:23.319
Jun 9, 2021 @ 21:07:23.382

So, failed job number two follows one minute after the first failed job, and failed job three follows 6 minutes after the second.

• Rileych moved this task from Needs Discussion to Triaged on the Growth-Team board.Jun 22 2021, 2:25 PM

Flow is getting the title from the EchoEvent, sets its own and at the end reset to original title, but that is null.

EchoEvent::getTitle is allowed to return null. Maybe there is a new notification event without title/pageid context

Umherirrender unsubscribed.Jan 2 2022, 8:44 PM

Krinkle moved this task from May 2021 to Older on the Wikimedia-production-error board.Aug 17 2023, 3:50 PM

TypeError: Argument 1 passed to EchoEvent::setTitle() must be an instance of Title, null given, called in /srv/mediawiki/php-1.37.0-wmf.4/extensions/Flow/includes/Notifications/UserLocator.php on line 49Open, MediumPublicPRODUCTION ERRORActions