PDF report

UpdateSubscriptions seems to not need any updates.

https://docs.google.com/document/d/1pXiTT_IMoArOOdsmaQ1zYKHOBLAZTvDuIq8YOawUc_Y/edit?tab=t.plset6blnjgm

solution at https://gitlab.wikimedia.org/repos/wmde/wikidata-integration/external-rc-sampler/-/merge_requests/21

Results can be tracked down from: https://docs.google.com/document/d/1ORyD_uQwBvZyqaL29TsOmq0E9W0Pyo8vq-oln8dDFuE/edit?usp=sharing

Danny will get in touch with Taavi

Suzie will send out an email to Chris

Task may need to be broken down further into smaller tasks

Biggest part is done, a comment from Arabic wiki went with a string instead of the team approach. Danny still needs to learn if this will not break anything

This wmfdata package is still breaking implementation

In T431814#12115114, @Ifeatu_Nnaobi_WMDE wrote:

Is there a way to see further categories that are not only language link added so that we can find out the various proportions of other potentially annoying notifications that don't create a diff

In T430281#12114286, @awight wrote:

This has been run in production now!

This is blocked for now because its awaiting the text diffing functionality

DOM related diffs turned out to be inconsistent due to the DOM misalignment from the parsing html process which rendered it useless to the main goal.

Other related patch: https://gitlab.wikimedia.org/repos/wmde/wikidata-integration/external-rc-sampler/-/merge_requests/15/diffs

Solution: https://gitlab.wikimedia.org/repos/wmde/wikidata-integration/external-rc-sampler/-/merge_requests/14/diffs

• https://github.com/GerHobbelt/google-diff-match-patch
• https://pypi.org/project/html-diff/

• This aligns well with the https://docs.python.org/3/library/difflib.html#difflib.HtmlDiff

• Comparing screenshots using playwright. https://playwright.dev/docs/test-snapshots
• https://xmldiff.readthedocs.io/en/stable/index.html. This takes the DOM into consideration during its comparison. So I went with this.

[insert, /div[1], p, 3]
[update-text, /div/p[2], "i", null]
[delete, /div/h2[1]]

• Choose suitable Library for diff with mostly python native modules.
• Return changes value for indirect value if not inner html changes and inner html changed return verdict if something changed.
• Store everything in a single directory, rev, re-parsed rev and diff
• Use Pytest to test everything.

The graphs

Add Adam to the mailing list.

Can we attach a document showing what we have learned and the things we need to follow up on.

@SuzanneWood-WMDE , According to my look through codesearch, I identified changes potentially needed in:

Solution can be found here https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1260687/17..18 for RC where cache updates before the RefreshLinks Job is run

Some of the change scenarios can be caught here and everything is summarized in this doc. The resolutions are also included in the later document

Current Grafana Board: https://grafana-rw.wikimedia.org/d/ni4446j/wit-recent-changes-tracker?forceLogin=true&from=now-90d&orgId=1&timezone=Europe%2FBerlin&to=now

Find the reviews correspondence in this document.

Pausing https://phabricator.wikimedia.org/T426386 till this is done.

Find the changes on Patch https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1289392