• Choose suitable Library for diff with mostly python native modules.
• Return changes value for indirect value if not inner html changes and inner html changed return verdict if something changed.
• Store everything in a single directory, rev, re-parsed rev and diff
• Use Pytest to test everything.

Options:

• https://github.com/GerHobbelt/google-diff-match-patch
• https://pypi.org/project/html-diff/

• This aligns well with the https://docs.python.org/3/library/difflib.html#difflib.HtmlDiff

• Comparing screenshots using playwright. https://playwright.dev/docs/test-snapshots
• https://xmldiff.readthedocs.io/en/stable/index.html. This takes the DOM into consideration during its comparison. So I went with this.

[insert, /div[1], p, 3]
[update-text, /div/p[2], "i", null]
[delete, /div/h2[1]]

also html

<div xmlns:diff="http://namespaces.shoobx.com/diff">
    <h1>Hello</h1>
    <h2 diff:delete="">Hello</h2>
    <p>Next something</p>
<diff:insert>    </diff:insert><p diff:insert="">i</p><diff:insert>
</diff:insert></div>

We noticied this unfortunate difference, caused by unexplained Enterprise things: T430788: Enterprise snapshot rendering may be missing some MediaWiki extensions

Solution: https://gitlab.wikimedia.org/repos/wmde/wikidata-integration/external-rc-sampler/-/merge_requests/14/diffs

Other related patch: https://gitlab.wikimedia.org/repos/wmde/wikidata-integration/external-rc-sampler/-/merge_requests/15/diffs

DOM related diffs turned out to be inconsistent due to the DOM misalignment from the parsing html process which rendered it useless to the main goal.

The decision was to resort to visible text comparisons comparing two text blocks to find differences instead of relying on html. So the non-visible changes and changes in links or images will be done at a much later stage.

This has been run in production now!

In T430281#12114286, @awight wrote:

This has been run in production now!

Yippeee!