Page MenuHomePhabricator

Make it possible to remove extensions' event data from Echo tables after undeploying them
Closed, ResolvedPublic

Description

When Flow (or any extension) is undeployed, Echo events/notifications that it generated remain behind in the database. This is harmless for most extensions (Echo hides them), but in Flow's case, they include serialized data (instances of the Flow\Model\UUID class), which can no longer be unserialized once Flow is gone, which is causing warnings to be logged (T388725#10663772), such as:

Unserializing of extra data partially failed for event <id> of type flow-post-reply
Unserializing of extra data partially failed for event <id> of type flow-new-topic
Unserializing of extra data partially failed for event {id} of type {type}

We should have a maintenance script that removes these events, and run it once Flow is undeployed.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Change #1130179 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/extensions/Echo@master] Update removeInvalidNotification maint script to handle any types

https://gerrit.wikimedia.org/r/1130179

Flow has been previously undeployed from some wikis, e.g. metawiki, and they are already emitting these warnings now. We should run the script now (once it's reviewed), and then run it again once the big sunsetting in T332022 is done.

Flow has been previously undeployed from some wikis, e.g. metawiki, and they are already emitting these warnings now. We should run the script now (once it's reviewed), and then run it again once the big sunsetting in T332022 is done.

If we resolved T383948: Echo notification expiration should be time-based, not hardcoded 2000 then such notifications will be purged and this problem will soon be moot. For now, we can treat unknown type of notifications as nonexistent since the topic pages will be no longer accessible anyway once Flow is undeployed.

They are already treated as nonexistent by the user interface, but the way Echo is coded right now, it unserializes the notification extra data before it checks its type and realizes that the notification can't be shown, so its mere existence generates warnings. I had a look and it seemed tricky to disentangle that. Updating and running the script seems easier.

One of the things we could do is to let Echo remove the Event when it cannot unserialize it. But this can cause confusion and in case of short rollbacks or disabling extensions could wipe some Notifications.

@matmarex I like the idea to fix the removal script. I'm just wondering - if we run maint scripts - we need to specify the WikiID. From what I know, we have different extensions enabled in different wikis. Do you know if just specifying the --wiki param will respect the Wiki extension list?

Change #1130179 merged by jenkins-bot:

[mediawiki/extensions/Echo@master] Update removeInvalidNotification maint script to handle any types

https://gerrit.wikimedia.org/r/1130179

Yes, WMF has a custom MediaWiki maintenance script wrapper (mwscript / MWScript.php) which ultimately handles the --wiki parameter here in order to load the right configuration before the rest of MediaWiki is loaded.

matmarex renamed this task from Remove StructuredDiscussions (Flow) event data from Echo tables after undeploying it to Make it possible to remove extensions' event data from Echo tables after undeploying them.May 8 2025, 12:17 PM

Renamed since there have been other extensions in that past that created notifications – I got someone to dry-run the script during the Wikimedia-Hackathon-2025, and it found events from e.g. MediaWiki-extensions-EducationProgram and Gather, which have been undeployed ages ago.

I scheduled the maint script to be ran today and clear out these old extensions' data, so that running it after future Flow undeployments will be smooth and quick. https://wikitech.wikimedia.org/wiki/Deployments#deploycal-item-20250508T1300

After that we can close this task.

Mentioned in SAL (#wikimedia-operations) [2025-05-08T14:34:19Z] <James_F> Running foreachwiki extensions/Echo/maintenance/removeInvalidNotification.php --remove # T389673 for MatmaRex

In total, 571068 events removed across 25 wikis. Full log here: P75884

Thanks @Jdforrester-WMF. I reviewed the log, we've removed the following invalid events from undeployed extensions across various wikis, nothing too unexpected:

matmarex claimed this task.