@MusikAnimal i'll see if i can debug it.

A few things... this particular example had its pickle generated in March, so it was serving data based on the old algorithm.

However, both the new and old versions of the algorithm do badly with this particular article, and neither one correctly attributes the first words to the very first revision of the article like it ought to.

Below is the analysis that Claude Code drafted.

-Sage

Claude Code analysis

I dug into this. The bug is real and reproduces, but I think the diagnosis in the triage is backwards: the incorrect output is coming from the old Differ-based algorithm, not the new one. It's a stale-cache problem, and reverting the algorithm would make it worse.

The reported token is definitely misattributed

For Yeah Yeah Yeahs (EP) rev 1367775551, the WikiWho API itself returns o_rev_id = 1342054706 for the first bolded "Yeah", so this is in WikiWho core — WhoColor and the gadget aren't involved in this particular symptom.

And @RoySmith is right that it's wrong. '''''Yeah Yeah Yeahs''''' is byte-identical in Watagwaan's revision and its parent (1294185374). In fact that exact string is present in all 356 revisions of the article, going back to its creation in rev 1625540 (2003-10-25).

Which algorithm produced it

I replayed the article's full 356-revision history through three pinned copies of the library and compared per-token o_rev_id against what the production API currently serves:

Production is byte-for-byte identical to the old Differ algorithm across all 7999 tokens.

It's the cache, not the deployed code

To separate "prod runs old code" from "prod runs new code over an old cache", I picked a control article created after the 2026-06-28 submodule bump, so no pre-existing pickle could exist: 2026 Memphis Classic – Singles (created 2026-07-24, 160 revisions). Production had in fact never processed it — the first request returned "Requested data is not currently available in WikiWho database", then computed it on demand. Result:

So the deployed code is current. Only the cached articles are stale.

The mechanism: api/handler.py loads a per-article pickle via pickle_load() with no algorithm/version check, and api/utils_pickles.py has no version constant. After an algorithm change ships, every already-cached article keeps its old attributions indefinitely; only new revisions get appended under the new code. Watagwaan's edit was processed on 2026-03-06, months before the first bump (pin timeline: 2026-05-07 61d2174 → 2026-06-08 c3fa64d → 2026-06-28 38f4117).

This is consistent with the hybrid state you'd expect: only 7 revisions of this article postdate the deploy, they contribute 23 tokens total, and both algorithms agree on every one of them — so an old pickle with 7 new revisions appended is indistinguishable from a pure Differ replay.

A corollary worth noting: essentially every article first processed before late June 2026 is still being served old-algorithm attribution. So for most articles, users are not seeing the new matcher at all.

Recommendation

Please don't revert to difflib.Differ — the reported output is Differ's output. Reverting would guarantee this specific bug persists, and would additionally regress freshly-computed articles (which are currently correct on this token) into the same failure.

The actual fix is cache invalidation: stamp pickles with an algorithm version and rebuild when it doesn't match. The cost is real — invalidation means a full-history replay per article, which is well under a second for a small page but 40–55s for the largest ones — so a bulk regeneration needs planning.

Two caveats, in fairness

1. The new matcher isn't fully correct here either. It credits rev 748288963 (2016) rather than the true origin 1625540 (2003). This looks like a duplicate-occurrence mismatch: from 2015 to 2016 the article contained the bold string twice (the lead plus an infobox chronology field | This album = '''''Yeah Yeah Yeahs'''''), and the matcher followed the wrong lineage when the duplicate went away. PR #5 doesn't change this. So regenerating the cache will make this token much less wrong (13 years off instead of 23) but still not right, and someone checking carefully may well still report it. That's worth a separate task.
2. WhoColor PR #2 can produce similar-looking reports for an unrelated reason. That change (which I wrote) attributes an entire {{cite}} template inside <ref> tags to its first token's editor, so a citation later tweaked by someone else is still credited to whoever added it. That's a documented limitation of span-wrapping, not this bug — but if any of the "multiple articles" cases involve clicking on references rather than prose, that's the likelier explanation and should be triaged separately.

Method

Replays used the reference wikiwho.py driven over full histories captured from the Action API, comparing per-token o_rev_id rather than aggregate counts (token counts and structural state can match exactly while attribution diverges). Happy to share the fixtures and comparison scripts if useful.

I'm looking at the examples from the Meta talk page as well, and did find one regression that was introduced by the recent algorithm changes, but I think it should be practical to fix it. (The bug is that in internal link with a namespace prefix, like File:, breaks highlighting, so anything after that link doesn't get any highlighting data.)

Thanks for looking into it!

The actual fix is cache invalidation: stamp pickles with an algorithm version and rebuild when it doesn't match. The cost is real — invalidation means a full-history replay per article, which is well under a second for a small page but 40–55s for the largest ones — so a bulk regeneration needs planning.

This may be doable for many/most pickles, but definitely not all. We should not wholesale invalidate pickles, or else very large articles like [[Barack Obama]] may never return – as it's just too many revisions to comb through with the action API. We'd need a fresh XML dump to process from instead. Re-importing from XML for all pickles generated before 2026-06-28 will take many weeks of tedious work and probably involve some downtime. I am not personally interested in such a grand undertaking. I had thought the differences in the algorithm were not large enough such that a mix of an old pickle with new revisions on top would make that much of a difference.

In T434097#12189462, @Ragesoss wrote:

I'm looking at the examples from the Meta talk page as well, and did find one regression that was introduced by the recent algorithm changes, but I think it should be practical to fix it. (The bug is that in internal link with a namespace prefix, like File:, breaks highlighting, so anything after that link doesn't get any highlighting data.)

Awesome! It'd be great to get that fixed. I'll happily review it when a PR is ready.

I agree, we shouldn't regenerate the pickles.

The PR is ready at https://github.com/wikimedia/WhoColor/pull/3

It's just a small regex tweak.

In T434097#12189351, @Ragesoss wrote:

The new matcher isn't fully correct here either. It credits rev 748288963 (2016) rather than the true origin 1625540 (2003). This looks like a duplicate-occurrence mismatch: from 2015 to 2016 the article contained the bold string twice (the lead plus an infobox chronology field | This album = '''''Yeah Yeah Yeahs'''''), and the matcher followed the wrong lineage when the duplicate went away. PR #5 doesn't change this. So regenerating the cache will make this token much less wrong (13 years off instead of 23) but still not right, and someone checking carefully may well still report it. That's worth a separate task.

Let me take a look into this and see if we can fix it in WikiWho PR#5.

In T434097#12189499, @MusikAnimal wrote:

This may be doable for many/most pickles, but definitely not all. We should not wholesale invalidate pickles, or else very large articles like [[Barack Obama]] may never return – as it's just too many revisions to comb through with the action API. We'd need a fresh XML dump to process from instead. Re-importing from XML for all pickles generated before 2026-06-28 will take many weeks of tedious work and probably involve some downtime. I am not personally interested in such a grand undertaking. I had thought the differences in the algorithm were not large enough such that a mix of an old pickle with new revisions on top would make that much of a difference.

Well, you have to consider the possibility that the old pickle is already faulty, which is the case here. But I agree, generating full pickles is too costly. since it only concerns one article, you can pick it out and only regenerate that single one article, and similarly in the future, only regenerate the faulty articles when reported.

Well, you have to consider the possibility that the old pickle is already faulty, which is the case here. But I agree, generating full pickles is too costly. since it only concerns one article, you can pick it out and only regenerate that single one article, and similarly in the future, only regenerate the faulty articles when reported.

I had thought in the past about creating an admin interface to delete pickles (even if it's just browsing to a route with no UI). I've filed T434163: Add OAuth and route to delete pickle files.

In T434097#12190087, @MusikAnimal wrote:

I had thought in the past about creating an admin interface to delete pickles (even if it's just browsing to a route with no UI). I've filed T434163: Add OAuth and route to delete pickle files.

That's a great idea! A dedicated route to delete pickles would be very useful.

In T434097#12190009, @Supergrey1 wrote:

Let me take a look into this and see if we can fix it in WikiWho PR#5.

WikiWho PR#5 are some correction fixes I have already tested previously. These can be merged separately.

I am still testing different matcher fixes for the "Yeah Yeah Yeahs" article. That'll be PR#6 when I figured out a proper fix to the issue (without regressions on correctness or time complexity).

musikanimal opened https://gitlab.wikimedia.org/repos/modtools/wikiwho_api/-/merge_requests/7

Bump WhoColor to the latest

I have created https://github.com/wikimedia/WikiWho/pull/6 after I am satisfied with a proper fix to the matcher algorithm. This one does not regress on any cases I've tested (I have a ~4k-test-cases benchmark), including all test cases on the current golden fixtures, with an acceptable runtime (~4.3%) and memory (~11%) increase. I believe making the attribution right is more important, so this PR#6 implementation is the current result I am in favor of.

musikanimal merged https://gitlab.wikimedia.org/repos/modtools/wikiwho_api/-/merge_requests/7

Bump WhoColor to the latest

@RoySmith I think your issue is fixed now. Can you confirm?

Same with the issues mentioned on the talk page. They sounded like multiple problems, but we've only deployed the fix for T434097#12189462, which is not an issue with the core WikiWho algorithm, rather only with WhoColor.

@MusikAnimal it's getting closer. In the same example I cited originally, hovering on the first word now highlights this:

clicking on that takes you to https://en.wikipedia.org/w/index.php?diff=748288963 which (ignoring a lot of citation template changes) has this:

which is kind of the right diff, but it doesn't include the

'''''Yeah Yeah Yeahs'''''

part, so I'm not sure if it's quite correct.

@RoySmith I just tested and reproduced the issue you talked about. The fix is in WikiWho PR#6 so you might have to wait till that PR is merged.

Ah, I missed that above. Makes sense, thanks.

Related: T434587: Merge WikiWho and WhoColor repos into monorepo

Given we're continually re-adding to every pickle (i.e. article) out there, I kindly request swift review on WikiWho PR #5 and #6. If I understand correctly, with each growing day, matters will be made worse. All the while I'm not certain the new fixes will make things stable or not :/

This is not a criticism of the hard work and efforts into improving the algorithm, which I greatly appreciate. The issue is from a product standpoint, and also a technical one, in the sense we need to invalidate pickles to get the same results consistently. Some of these cannot be invalidated currently and will forever be broken until we add auto-XML imports (see T434163) – and even then we're running into uncharted waters.

Please understand that important decisions are historically or even currently being made using WikiWho data, given its tenure as a public, reliable, authorship attribution service. So when we introduce regressions or other significant difference in behaviour, wrong decisions may follow.

In T434097#12189131, @MusikAnimal wrote:

The algorithm was recently changed for T342805 (see WikiWho/pull/2 and WikiWho/pull/3). I highly suspect this is the cause, assuming this issue seems to be relatively new.

There was also a change in WhoColor that might be at fault (WhoColor/pull/2).

Pinging authors @Supergrey1 @Ragesoss. Not sure if either of you have any ideas, or the appetite to debug this, but if not I plan to start with reverting back to the old algorithm with the difflib.Differ-based word matcher (c3fa64d569).

I hate to be such a stickler, but I sorta kinda wanta go back to just reverting, at least until we can sandbox things and test at scale like production? The prod pickles I don't think should be the guinea pigs :(

Don't be discouraged. I kind of get how it feels to be unsure if things are actually better or worse, and the limited test suite in the WikiWho repo probably doesn't help with that uneasiness.

I've been maintaining a benchmark with over 7k test cases that I put together while working on algorithm improvements. The old Differ-based version struggles with a bunch of them, but the newer one does better without any correctness regressions, including the recently discussed cases (and certainly faster than the old version, e.g. on [[Google Play]]; though the newest ones may be slightly slower than the previous new matchers). Hopefully this puts your mind at ease a little bit.