Some diffs are not generated correctly due to mismatched line endings that vary across systems ("\r\n" compared to '\n"). We should use TextContent::normalizeLineEndings() in calculating the proposed content diff.
Change #1214447 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):
[mediawiki/extensions/WikimediaEvents@master] hCaptchaEditAttempt logging: Normalize line endings
Change #1214447 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] hCaptchaEditAttempt logging: Normalize line endings
Change #1214469 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):
[mediawiki/extensions/WikimediaEvents@wmf/1.46.0-wmf.4] hCaptchaEditAttempt logging: Normalize line endings
Change #1214470 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):
[mediawiki/extensions/WikimediaEvents@wmf/1.46.0-wmf.5] hCaptchaEditAttempt logging: Normalize line endings
Change #1214469 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@wmf/1.46.0-wmf.4] hCaptchaEditAttempt logging: Normalize line endings
Change #1214470 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@wmf/1.46.0-wmf.5] hCaptchaEditAttempt logging: Normalize line endings
Mentioned in SAL (#wikimedia-operations) [2025-12-03T10:27:03Z] <kharlan@deploy2002> Started scap sync-world: Backport for [[gerrit:1214469|hCaptchaEditAttempt logging: Normalize line endings (T411578)]], [[gerrit:1214470|hCaptchaEditAttempt logging: Normalize line endings (T411578)]]
Mentioned in SAL (#wikimedia-operations) [2025-12-03T10:29:22Z] <kharlan@deploy2002> kharlan: Backport for [[gerrit:1214469|hCaptchaEditAttempt logging: Normalize line endings (T411578)]], [[gerrit:1214470|hCaptchaEditAttempt logging: Normalize line endings (T411578)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
Mentioned in SAL (#wikimedia-operations) [2025-12-03T10:34:59Z] <kharlan@deploy2002> Finished scap sync-world: Backport for [[gerrit:1214469|hCaptchaEditAttempt logging: Normalize line endings (T411578)]], [[gerrit:1214470|hCaptchaEditAttempt logging: Normalize line endings (T411578)]] (duration: 07m 56s)
Seems like this may still be an issue, so I'm moving it to ready, for investigation and follow-up fixes.
Change #1218397 had a related patch set uploaded (by Samuel (WMF); author: Samuel (WMF)):
[mediawiki/extensions/WikimediaEvents@master] hCaptchaEditAttempt: Fix diff generation for proposed_content
Change #1233186 had a related patch set uploaded (by Harroyo-wmf; author: Harroyo-wmf):
[mediawiki/extensions/WikimediaEvents@master] hcaptcha: Delegate computing content diffs to the DifferenceEngine
This is being solved by https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaEvents/+/1233186 (associated with T406865: hCaptcha: Implement mechanism to log about-to-be-published content when challenge is presented), which will save the diffs as HTML.
Change #1218397 abandoned by Kosta Harlan:
[mediawiki/extensions/WikimediaEvents@master] hCaptchaEditAttempt: Fix diff generation for proposed_content
https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaEvents/+/1233186 is being merged now, so moving this task to QA
Change #1233186 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] hcaptcha: Delegate computing content diffs to the DifferenceEngine
This was solved by storing the diff as an HTML table, and we've verified that table is being saved (truncating the HTML if needed) while testing T406865, see T406865#11728727:
Verified that this information is being captured in Superset, and that the data can be used to print an HTML table for the diff associated with the edit that triggered the challenge (the images below correspond to a diff in enwiki).
