No known owner per mw:Maintainers..

This seems to be caused by the value being truncated:

> $title = Title::makeTitle( NS_LQT_THREAD, 'Talk:China_Task_Force/What makes Wikimedia different from Hudong & Baidu Baike' );
> $p = WikiPage::factory($title);
> $th = Threads::withRoot( $p );
> var_dump($th->id());
string(2) "64"

 select length(th_content) from thread_history where th_thread=64 order by th_timestamp desc\g
+--------------------+
| length(th_content) |
+--------------------+
|               4609 |
|               3835 |
|              65535 |
|               3055 |
|               2298 |
|              65535 |
|              65535 |
|                709 |
+--------------------+

several entries have a length of 65535, which is the maximum length for a blob (as I realized too late, this was clear by looking at the error message). Then the column was expanded to LONGBLOB in 2009, but any entry that was already truncated upon insertion remains truncated.

Depending on the resources, there are three solutions to this issue:

Quick & easy solution

Just ignore the error. It can only happen on (some) pages older than 2009. It's certainly not high priority.

Quick but hacky solution

Suppress errors from the unserialize() call, do something in case of errors. The status quo is that $rev->mThreadObj is set to false and everything moves on silently, à la typical PHP ancient style, with horrible bugs waiting to happen (possibly).

Sane solution

Serialization should be avoided (T161647), as serializing classes like this to the database is bad practice (which can quickly become horrible practice if the classes are not plain value objects). As such, a new storage format might be introduced (e.g. JSON encoding of plain arrays), a maintenance script to update old entries could be written, and this script could also do something in case of truncated blobs. All of this would require a decent amount of work.

@Daimona What is the impact on end-user experience currently? The object that we are unable to unserialize, what object is it? Is there missing content now? Or is this an optimisation or cache that has a fallback?

If the content is missing, I'm guessing it's mostly irreversible, although we may able to re-encode a few rows by hand if we can find something in the truncated blob that's left to salvage and make sense of.

In T259111#6893449, @Krinkle wrote:

@Daimona What is the impact on end-user experience currently?

It is not entirely clear from the reproduction URL. For sure, there's no fatal error of any sort. There might be missing information, but I don't have enough knowledge of LiquidThreads or that specific page to determine this easily.

The object that we are unable to unserialize, what object is it?

It's a Thread object encapsulating some information about the thread history. I haven't analyzed the long blob from the prod DB, but there seems to be a lot of data in there. It seems like every Thread object stores additional thread objects, in a way that I think might easily run OOM, although this was for blobs stored in 2009. A blob taken from strategywiki with the following query:

select th_content from thread_history where th_id=309\g

seems to have 87 (!) Thread objects serialized inside of it. While these seem to be plain value objects, it seems like there was some fast (factorial/exponential) growth in the amount of Threads that can be stored (the page history has 8 threads), but again, I don't have more detailed information. The PHP serialization format also makes it particularly hard to read.

If the content is missing, I'm guessing it's mostly irreversible, although we may able to re-encode a few rows by hand if we can find something in the truncated blob that's left to salvage and make sense of.

Correct. I had to do something similar for AbuseFilter, see https://phabricator.wikimedia.org/diffusion/EABF/browse/master/maintenance/UpdateVarDumps.php$370

Still seeing these intermittently in 1.40.0-wmf.12.

Short spike after 1.43.0-wmf.13 deployment around Jul 11, 2024 @ 08:30:00.000, again on strategywiki only just like dup T360110.

Still seen. The latest spike is from multiple different replies on this page failing to unserialize:

https://strategy.wikimedia.org/w/index.php?lqt_method=thread_history&title=Thread:Talk:Strategic_Plan/Movement_Priorities/diversity

Still seeing spikes of this error as of 1.43.0-wmf.23:

https://logstash.wikimedia.org/goto/497b6eca591edceac36a99b29c6e6572

Snapshot was taken ~5 hours after deployment of 1.43.0-wmf.23 to group0. There seems to be a significant increase in the volume for 1.43.0-wmf.23 compared to previous versions.

For anyone running into this again in the future, it looks like there's some movement in the direction of sunsetting this extension: T370722 T350164

I've added a filter to the MediaWiki New Errors dashboard to try to reduce the noise. The filter is pretty safe: specific to LQT and strategywiki, so it shouldn't risk hiding similar issues happening elsewhere.

Error

• url: https://strategy.wikimedia.org/w/index.php?lqt_method=*&title=*
• mwversion: 1.45.0-wmf.7
• reqId: 4a8be230-7889-4811-a345-31feaa89e56c
• Find reqId in Logstash

[{reqId}] {exception_url}   PHP Notice: unserialize(): Error at offset 65531 of 65535 bytes

Change #1163833 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[operations/puppet@production] logspam.pl: Consolidate ThreadRevision unserialize() errors

https://gerrit.wikimedia.org/r/1163833

Change #1163833 merged by JHathaway:

[operations/puppet@production] logspam.pl: Consolidate ThreadRevision unserialize() errors

https://gerrit.wikimedia.org/r/1163833