It could also be that my only notifications on testwiki are very very old.

Change 163979 had a related patch set uploaded by Spage:
Don't fatal if a notification fails to initialize

https://gerrit.wikimedia.org/r/163979

Change 163979 merged by jenkins-bot:
Don't fatal if a notification fails to initialize

https://gerrit.wikimedia.org/r/163979

Gerrit 163979 logs the problem to the 'Echo' debug log. We still need to figure out why the problem occurs.

With the change to log the exceptions, the Echo debug log when I visit contains three lines

2014-10-09 02:15:02 mw1017 testwiki: EchoNotificationMapper::fetchByUser: Failed initializing event: 2325
2014-10-09 02:15:02 mw1017 testwiki: EchoNotificationMapper::fetchByUser: Failed initializing event: 2091
2014-10-09 02:15:02 mw1017 testwiki: EchoNotificationMapper::fetchByUser: Failed initializing event: 1895

while the exception.log contains three exceptions in DateTimeZone::__construct(), the same as above.

ErikB comments "the problem with spage Special:Notifications is that an unserializable string was written to the echo db. I'm not sure the exact bug number but thats probably related to a recent production problem where the igbinary serialization format was mis-configured onto a server causing other production serialization problems as well."

Changing title, because the merged bug from Reedy has an error from enwiki:

2015-01-09 14:20:58 mw1134 enwiki: [7da582d6] /w/api.php?action=query&format=json&meta=notifications&notsections=alert%7Cmessage&notgroupbysection=1&notmessageunreadfirst=1&notformat=flyout&notlimit=25&notprop=index%7Clist%7Ccount&uselang=en&_=1420813256080   Exception from line  of : DateTimeZone::__construct(): Unknown or bad timezone (+00:00)
#0 /srv/mediawiki/php-1.25wmf13/extensions/Echo/model/Event.php(225): DateTime->__wakeup()
#1 /srv/mediawiki/php-1.25wmf13/extensions/Echo/model/Event.php(285): EchoEvent->loadFromRow()
#2 /srv/mediawiki/php-1.25wmf13/extensions/Echo/model/Notification.php(184): EchoEvent::newFromRow()
#3 /srv/mediawiki/php-1.25wmf13/extensions/Echo/includes/mapper/NotificationMapper.php(183): EchoNotification::newFromRow()
#4 /srv/mediawiki/php-1.25wmf13/extensions/Echo/api/ApiEchoNotifications.php(143): EchoNotificationMapper->fetchByUser()
#5 /srv/mediawiki/php-1.25wmf13/extensions/Echo/api/ApiEchoNotifications.php(96): ApiEchoNotifications->getPropList()
#6 /srv/mediawiki/php-1.25wmf13/extensions/Echo/api/ApiEchoNotifications.php(31): ApiEchoNotifications->getSectionPropList()
#7 /srv/mediawiki/php-1.25wmf13/includes/api/ApiQuery.php(297): ApiEchoNotifications->execute()
#8 /srv/mediawiki/php-1.25wmf13/includes/api/ApiMain.php(954): ApiQuery->execute()
#9 /srv/mediawiki/php-1.25wmf13/includes/api/ApiMain.php(392): ApiMain->executeAction()
#10 /srv/mediawiki/php-1.25wmf13/includes/api/ApiMain.php(363): ApiMain->executeActionWithErrorHandling()
#11 /srv/mediawiki/php-1.25wmf13/api.php(85): ApiMain->execute()
#12 /srv/mediawiki/w/api.php(3): include()
#13 {main}

I was going to report this issue, but saw it was already done. The numbers are not too large, but they confirm it happens on several wikis:

Last 7 days: 61 occurrences - meta.wikimedia.org (48) en.wikipedia.org (11) www.mediawiki.org (2)

The referrer is Special:Watchlist.

It looks like flow-post-reply events used to have a DateTime object in event_extra, but don't any more. The newest event I could find that had one of these was at 2014-04-09T00:06:12Z. Furthermore, it appears that there was a breaking change in how DateTimeZone serializes/unserializes, because these event_extra blobs don't unserialize cleanly any more. My best suggestion is that we re-encode all these old blobs.

This would only affect requests that try to view an old (i.e. before April 2014) "someone replied to your Flow post" notification.

Wikis that had Flow enabled in April 2014:

• testwiki
• test2wiki
• mediawikiwiki
• metawiki
• enwiki

Change 288730 had a related patch set uploaded (by Legoktm):
Avoid exceptions when event_extra can't be unserialized

https://gerrit.wikimedia.org/r/288730

Change 288730 merged by jenkins-bot:
Avoid exceptions when event_extra can't be unserialized

https://gerrit.wikimedia.org/r/288730