[RedisBagOStuff] serialization is suspect warnings started to appear after the Kubernetes Deployment for Mediawiki on 21.10.2025 by Andrew (check logs here).

Observation

• I check the redis pod health status just to make sure it's not because of redis.

kubectl get pods -n default -l app.kubernetes.io/name=redis
NAME                 READY   STATUS    RESTARTS   AGE
redis-2-master-0     1/1     Running   0          10d
redis-2-replicas-0   1/1     Running   0          10d
redis-master-0       1/1     Running   0          10d
redis-replicas-0     1/1     Running   0          10d

• redis logs also doesn't show any signs of data corruption. I check the redis master logs and looking for warnings/error about serialization but no hit.

Run these yielded nothing
# Check if Redis has any warnings about serialization
kubectl logs -n default -l app.kubernetes.io/name=redis | grep -i serial (or serialization)
# Check Redis master logs
kubectl logs -n default -l app.kubernetes.io/name=redis,app.kubernetes.io/component=master

• There was a pods rescaling, maybe it was one of the reasons
• pods deletions and pods restart. May be they were a part of the 1.43 deployment rollout

Theory
These warning appeared around the same time with recent 1.43 Mediawiki deployment via Helm. This action involves patching the deployments and changing the container image 'ghcr.io/wbstack/mediawiki:sha-7c8e51b'. Here are 2 possibilities:

• The image introduced a compatibility problem with the existing Redis data format, leading to the serialization warning, or
• it's a token mismatch due to routing(???)

I recommend we resolve https://phabricator.wikimedia.org/T407504 this first to make sure

For now I don't find any other problem

@dena and I quickly talked about it and didn't want to forget the possibility that we simply ought to set an explicit redis db for each MW version.

It's also possible that with our "bad" routing there could be instances of 1.39 MW pods trying to read cache entries written by 1.43. The opposite (1.43 pods reading 1.39 redis entries) may well be fine because there is some backwards compatibility or update path. If this is the case then perhaps fixing the routing is all we need to do.

In T408103#11335195, @Tarrow wrote:

@dena and I quickly talked about it and didn't want to forget the possibility that we simply ought to set an explicit redis db for each MW version.

I actually just looked a bit into this and it seems the database selection is not supported by mediawiki (someone attempted to patch this here, but it doesn't seem to be an option in core https://www.mediawiki.org/wiki/Topic:Wi89pk2optfhl3fz)

Also coincidentally I just read in the Redis Insight tool that using these "logical databases" in redis is apparently not a popular feature:

So I'd agree to revisit this after we implemented the correct routing, if it then still occurs.

In T408103#11338959, @Tarrow wrote:

Waiting for T409064 to be fixed up

We aren't certain what this issue is; I suspect this is routing related but we should confirm once the routing is fixed.

Only manage to look at this a tiny bit. The conclusion I immediately came to is that our error logging is missing sufficient context to understand what the error really is.

Looking at making a change to https://github.com/wbstack/mediawiki/blob/main/dist-persist/wbstack/src/Logging/CustomLogger.php#L54 to make sure we keep the error context in the JSON

Got the context with a small change to this line in my dev env.

The context is:
"context":{"cachekey":"OAUTH:mwdb_1888b402c8-mwt_02f0b58fdc_:token:46418969bf3c336e0018aeb056314da3:request:6cc043f700e5fec3d50a74c329fb1096","class":"MediaWiki\\Extension\\OAuth\\Backend\\MWOAuthToken"}

looks to me like this is thrown from https://gerrit.wikimedia.org/g/mediawiki/core/+/70cacfd6a0ae8abc0c5ea113e0bd4ade65e8fb55/includes/libs/ObjectCache/MediumSpecificBagOStuff.php#1031

Seems like we're fine to continue without much further thinking on our end but we should check if we ought to open a ticket for Extension:OAuth to not emit this in the future by changing their cache key to make it stop WARNing here.

I'm not actually sure the context here is that important but I wonder if we still want to include it at some point in this jsonPayload.

I also wonder now we look at it if we *only* want to use the error reporting log format for actual errors?

looks to me like this is thrown from https://gerrit.wikimedia.org/g/mediawiki/core/+/70cacfd6a0ae8abc0c5ea113e0bd4ade65e8fb55/includes/libs/ObjectCache/MediumSpecificBagOStuff.php#1031

Agreed.

Seems like we're fine to continue without much further thinking on our end but we should check if we ought to open a ticket for Extension:OAuth to not emit this in the future by changing their cache key to make it stop WARNing here

Agreed. Look like Extension:OAuth is already aware, see T282107: Make OAuthToken JsonDeserializable.

I also wonder now we look at it if we *only* want to use the error reporting log format for actual errors?

As in, add warning to the ignoreLevels array in LocalSetting.php?