Page MenuHomePhabricator

Inspect Reply Tool diffs for instances of corruption
Closed, ResolvedPublic

Description

This task is about inspecting a sample of Reply Tool diffs to ensure there are no instances of content corruption.

Background.

622176 will, ideally, resolve "Issue 2" described in the task description of T259855.

Because "Issue 2" cannot be easily reproduced in a testing environment, it will be difficult for us to know the extent to which this issue persists prior to the patch intended to resolve it [i] being deployed. As such, once the patch [i] is deployed, we will inspect a sample of Reply Tool diffs to ensure there are no instances of content corruption [ii] using the run the script @matmarex wrote and shared in T260393#6384098

Timing

This task should be worked on after 2-September, assuming T252558 is deployed on 2-September as we plan for it to be. [iii]

Done

  • Reply Tool diffs are inspected using the script @matmarex wrote and shared in T260393#6384098
  • Any instances of Reply Tool edits causing changes other than content being added to the page are documented on this ticket

i. https://gerrit.wikimedia.org/r/622176
ii. These are two examples of the kind of content corruption we would expect to see more of if the "Issue 2" is NOT resolved:

iii. T252558#6428684

Event Timeline

ppelberg created this task.Aug 26 2020, 3:29 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 26 2020, 3:29 PM
ppelberg updated the task description. (Show Details)Sep 1 2020, 11:47 PM

Timing
Updated the task description's "Timing" section

This comment was removed by matmarex.

[sorry, I messed up my time zones initially]

Minor update to the script so that it only checks recent changes more recent than the deployment (around 2 September, 20:00 UTC 18:00 UTC):

I just ran it and there are no corrupted edits so far (although there are very few edits overall).

Checking en.wikipedia.org...                                                                                           
* 2020-09-02T23:14:21Z https://en.wikipedia.org/w/index.php?oldid=976442208&diff=prev                                  
* 2020-09-02T19:37:37Z https://en.wikipedia.org/w/index.php?oldid=976412065&diff=prev                                  
* en.wikipedia.org: 2/26 look suspicious                                                                               
Checking www.mediawiki.org...                                                                                          
* www.mediawiki.org: 0/0 look suspicious                                                                               
Checking ar.wikipedia.org...                                                                                           
* ar.wikipedia.org: 0/6 look suspicious                                                                                
Checking ca.wikipedia.org...                                                                                           
* ca.wikipedia.org: 0/0 look suspicious                                                                                
Checking ckb.wikipedia.org...                                                                                          
* ckb.wikipedia.org: 0/1 look suspicious                                                                               
Checking cs.wikipedia.org...                                                                                           
* cs.wikipedia.org: 0/2 look suspicious                                                                                
Checking fr.wikipedia.org...                                                                                           
* fr.wikipedia.org: 0/9 look suspicious                                                                                
Checking hu.wikipedia.org...                                                                                           
* hu.wikipedia.org: 0/0 look suspicious                                                                                
Checking ka.wikipedia.org...                                                                                           
* ka.wikipedia.org: 0/3 look suspicious                                                                                
Checking ko.wikipedia.org...                                                                                           
* ko.wikipedia.org: 0/2 look suspicious                                                                                
Checking nl.wikipedia.org...                                                                                           
* nl.wikipedia.org: 0/5 look suspicious                                                                                
Checking sr.wikipedia.org...                                                                                           
* sr.wikipedia.org: 0/0 look suspicious                                                                                
Checking sv.wikipedia.org...                                                                                           
* sv.wikipedia.org: 0/3 look suspicious                                                                                
Checking zh.wikipedia.org...                                                                                           
* zh.wikipedia.org: 0/0 look suspicious

(the suspicious edits on en.wp only unexpectedly remove some leading/trailing spaces)

matmarex claimed this task.Sep 3 2020, 12:30 AM

I'll check again later this week

matmarex added a comment.EditedSep 3 2020, 7:17 PM

Ran again today, there is no corruption like in T259855, but there are a few interesting dirty diffs:

(I don't think any of these require any action from us, but you might find it interesting)

Full output:

Checking en.wikipedia.org...
* 2020-09-03T12:55:29Z https://en.wikipedia.org/w/index.php?oldid=976527796&diff=prev
* 2020-09-03T04:05:14Z https://en.wikipedia.org/w/index.php?oldid=976475170&diff=prev
* 2020-09-02T23:14:21Z https://en.wikipedia.org/w/index.php?oldid=976442208&diff=prev
* 2020-09-02T19:37:37Z https://en.wikipedia.org/w/index.php?oldid=976412065&diff=prev
* en.wikipedia.org: 4/33 look suspicious
Checking www.mediawiki.org...
* 2020-09-03T06:57:53Z https://www.mediawiki.org/w/index.php?oldid=4074102&diff=prev
* www.mediawiki.org: 1/3 look suspicious
Checking ar.wikipedia.org...
* ar.wikipedia.org: 0/22 look suspicious
Checking ca.wikipedia.org...
* ca.wikipedia.org: 0/0 look suspicious
Checking ckb.wikipedia.org...
* ckb.wikipedia.org: 0/1 look suspicious
Checking cs.wikipedia.org...
* cs.wikipedia.org: 0/10 look suspicious
Checking fr.wikipedia.org...
* 2020-09-03T13:18:22Z https://fr.wikipedia.org/w/index.php?oldid=174398363&diff=prev
* 2020-09-03T10:09:34Z https://fr.wikipedia.org/w/index.php?oldid=174393706&diff=prev
* fr.wikipedia.org: 2/33 look suspicious
Checking hu.wikipedia.org...
* hu.wikipedia.org: 0/4 look suspicious
Checking ka.wikipedia.org...
* ka.wikipedia.org: 0/12 look suspicious
Checking ko.wikipedia.org...
* 2020-09-03T13:14:40Z https://ko.wikipedia.org/w/index.php?oldid=27483571&diff=prev
* 2020-09-03T09:18:23Z https://ko.wikipedia.org/w/index.php?oldid=27482051&diff=prev
* 2020-09-03T09:03:25Z https://ko.wikipedia.org/w/index.php?oldid=27481948&diff=prev
* 2020-09-03T06:43:17Z https://ko.wikipedia.org/w/index.php?oldid=27480891&diff=prev
* 2020-09-03T04:17:37Z https://ko.wikipedia.org/w/index.php?oldid=27479801&diff=prev
* 2020-09-03T04:12:46Z https://ko.wikipedia.org/w/index.php?oldid=27479784&diff=prev
* 2020-09-03T03:17:47Z https://ko.wikipedia.org/w/index.php?oldid=27479283&diff=prev
* 2020-09-03T02:34:03Z https://ko.wikipedia.org/w/index.php?oldid=27478691&diff=prev
* 2020-09-03T01:46:34Z https://ko.wikipedia.org/w/index.php?oldid=27478406&diff=prev
* 2020-09-03T01:46:10Z https://ko.wikipedia.org/w/index.php?oldid=27478403&diff=prev
* ko.wikipedia.org: 10/42 look suspicious
Checking nl.wikipedia.org...
* nl.wikipedia.org: 0/19 look suspicious
Checking sr.wikipedia.org...
* sr.wikipedia.org: 0/0 look suspicious
Checking sv.wikipedia.org...
* sv.wikipedia.org: 0/9 look suspicious
Checking zh.wikipedia.org...
* 2020-09-03T08:00:25Z https://zh.wikipedia.org/w/index.php?oldid=61474788&diff=prev
* zh.wikipedia.org: 1/6 look suspicious

(anything I didn't mention is just trailing/leading spaces)

Ran again today, there is no corruption like in T259855, but there are a few interesting dirty diffs...

Having a low effort and reliable way to identify [potentially] problematic diffs increases my confidence in the tool. It leads me to wonder whether we ought to make this a consistent practice as it becomes used in a greater variety of environments. Here is a task for us to advance this thought: T261998.

  • @matmarex: would it be accurate for me to understand this as the `{{RMCategory|匯知中學|萬鈞匯知中學}} category being moved to a different location on the page?
  • @matmarex: would it be accurate for me to understand this as the `{{RMCategory|匯知中學|萬鈞匯知中學}} category being moved to a different location on the page?

Yeah, I think that's right… I haven't looked previously to see what is generated by the template, but it looks like it's just a category.

It wasn't exactly "moved to a different location", but rather the reply was inserted before the template, which was on the same line in wikitext as an existing comment, which necessitates putting the template on a separate line. I'm not sure if this is correct, or if the template should have been treated as a part of the comment. But I don't think this matters much.

It doesn't surprise me, but it does annoy me. :)

matmarex added a comment.EditedSep 8 2020, 2:23 AM

One more run. I think we can safely conclude that the issues from T259855 are not reappearing.

Interesting diffs (excluding whitespace changes, and the </sup> signature on ko.wp):

Full output:

Checking en.wikipedia.org...
* 2020-09-07T23:46:53Z https://en.wikipedia.org/w/index.php?oldid=977280557&diff=prev
* 2020-09-07T23:14:42Z https://en.wikipedia.org/w/index.php?oldid=977276876&diff=prev
* 2020-09-07T07:23:02Z https://en.wikipedia.org/w/index.php?oldid=977158736&diff=prev
* 2020-09-07T02:27:14Z https://en.wikipedia.org/w/index.php?oldid=977124090&diff=prev
* 2020-09-07T01:48:04Z https://en.wikipedia.org/w/index.php?oldid=977119075&diff=prev
* 2020-09-04T00:07:55Z https://en.wikipedia.org/w/index.php?oldid=976614331&diff=prev
* 2020-09-03T22:13:26Z https://en.wikipedia.org/w/index.php?oldid=976602142&diff=prev
* 2020-09-03T12:55:29Z https://en.wikipedia.org/w/index.php?oldid=976527796&diff=prev
* 2020-09-03T04:05:14Z https://en.wikipedia.org/w/index.php?oldid=976475170&diff=prev
* 2020-09-02T23:14:21Z https://en.wikipedia.org/w/index.php?oldid=976442208&diff=prev
* 2020-09-02T19:37:37Z https://en.wikipedia.org/w/index.php?oldid=976412065&diff=prev
* en.wikipedia.org: 11/115 look suspicious
Checking www.mediawiki.org...
* 2020-09-03T06:57:53Z https://www.mediawiki.org/w/index.php?oldid=4074102&diff=prev
* www.mediawiki.org: 1/10 look suspicious
Checking ar.wikipedia.org...
* 2020-09-06T19:09:57Z https://ar.wikipedia.org/w/index.php?oldid=50253617&diff=prev
* 2020-09-05T10:40:42Z https://ar.wikipedia.org/w/index.php?oldid=50226176&diff=prev
* ar.wikipedia.org: 2/148 look suspicious
Checking ca.wikipedia.org...
* ca.wikipedia.org: 0/2 look suspicious
Checking ckb.wikipedia.org...
* ckb.wikipedia.org: 0/3 look suspicious
Checking cs.wikipedia.org...
* 2020-09-07T19:10:32Z https://cs.wikipedia.org/w/index.php?oldid=18979516&diff=prev
* 2020-09-06T17:32:08Z https://cs.wikipedia.org/w/index.php?oldid=18976582&diff=prev
* 2020-09-05T22:09:54Z https://cs.wikipedia.org/w/index.php?oldid=18973191&diff=prev
* 2020-09-05T14:00:04Z https://cs.wikipedia.org/w/index.php?oldid=18972249&diff=prev
* cs.wikipedia.org: 4/65 look suspicious
Checking fr.wikipedia.org...
* 2020-09-07T18:59:34Z https://fr.wikipedia.org/w/index.php?oldid=174521932&diff=prev
* 2020-09-07T16:56:14Z https://fr.wikipedia.org/w/index.php?oldid=174518306&diff=prev
* 2020-09-05T23:56:56Z https://fr.wikipedia.org/w/index.php?oldid=174467549&diff=prev
* 2020-09-05T20:37:17Z https://fr.wikipedia.org/w/index.php?oldid=174463162&diff=prev
* 2020-09-05T16:39:17Z https://fr.wikipedia.org/w/index.php?oldid=174457664&diff=prev
* 2020-09-05T13:25:51Z https://fr.wikipedia.org/w/index.php?oldid=174452720&diff=prev
* 2020-09-04T14:53:58Z https://fr.wikipedia.org/w/index.php?oldid=174427454&diff=prev
* 2020-09-04T14:52:20Z https://fr.wikipedia.org/w/index.php?oldid=174427403&diff=prev
* 2020-09-03T13:18:22Z https://fr.wikipedia.org/w/index.php?oldid=174398363&diff=prev
* 2020-09-03T10:09:34Z https://fr.wikipedia.org/w/index.php?oldid=174393706&diff=prev
* fr.wikipedia.org: 10/207 look suspicious
Checking hu.wikipedia.org...
* 2020-09-07T23:46:30Z https://hu.wikipedia.org/w/index.php?oldid=23025431&diff=prev
* 2020-09-05T05:03:45Z https://hu.wikipedia.org/w/index.php?oldid=23014207&diff=prev
* hu.wikipedia.org: 2/65 look suspicious
Checking ka.wikipedia.org...
* ka.wikipedia.org: 0/19 look suspicious
Checking ko.wikipedia.org...
* 2020-09-07T07:41:48Z https://ko.wikipedia.org/w/index.php?oldid=27512111&diff=prev
* 2020-09-07T03:13:14Z https://ko.wikipedia.org/w/index.php?oldid=27509937&diff=prev
* 2020-09-06T07:29:26Z https://ko.wikipedia.org/w/index.php?oldid=27504320&diff=prev
* 2020-09-04T10:03:03Z https://ko.wikipedia.org/w/index.php?oldid=27489239&diff=prev
* 2020-09-04T03:18:43Z https://ko.wikipedia.org/w/index.php?oldid=27487012&diff=prev
* 2020-09-04T03:18:27Z https://ko.wikipedia.org/w/index.php?oldid=27487008&diff=prev
* 2020-09-04T03:05:13Z https://ko.wikipedia.org/w/index.php?oldid=27486943&diff=prev
* 2020-09-04T03:03:11Z https://ko.wikipedia.org/w/index.php?oldid=27486934&diff=prev
* 2020-09-04T02:40:22Z https://ko.wikipedia.org/w/index.php?oldid=27486769&diff=prev
* 2020-09-04T02:35:40Z https://ko.wikipedia.org/w/index.php?oldid=27486727&diff=prev
* 2020-09-04T02:34:04Z https://ko.wikipedia.org/w/index.php?oldid=27486712&diff=prev
* 2020-09-04T02:32:11Z https://ko.wikipedia.org/w/index.php?oldid=27486696&diff=prev
* 2020-09-03T13:14:40Z https://ko.wikipedia.org/w/index.php?oldid=27483571&diff=prev
* 2020-09-03T09:18:23Z https://ko.wikipedia.org/w/index.php?oldid=27482051&diff=prev
* 2020-09-03T09:03:25Z https://ko.wikipedia.org/w/index.php?oldid=27481948&diff=prev
* 2020-09-03T06:43:17Z https://ko.wikipedia.org/w/index.php?oldid=27480891&diff=prev
* 2020-09-03T04:17:37Z https://ko.wikipedia.org/w/index.php?oldid=27479801&diff=prev
* 2020-09-03T04:12:46Z https://ko.wikipedia.org/w/index.php?oldid=27479784&diff=prev
* 2020-09-03T03:17:47Z https://ko.wikipedia.org/w/index.php?oldid=27479283&diff=prev
* 2020-09-03T02:34:03Z https://ko.wikipedia.org/w/index.php?oldid=27478691&diff=prev
* 2020-09-03T01:46:34Z https://ko.wikipedia.org/w/index.php?oldid=27478406&diff=prev
* 2020-09-03T01:46:10Z https://ko.wikipedia.org/w/index.php?oldid=27478403&diff=prev
* ko.wikipedia.org: 22/277 look suspicious
Checking nl.wikipedia.org...
* 2020-09-07T16:43:03Z https://nl.wikipedia.org/w/index.php?oldid=57078729&diff=prev
* 2020-09-05T10:28:52Z https://nl.wikipedia.org/w/index.php?oldid=57059437&diff=prev
* 2020-09-05T09:16:22Z https://nl.wikipedia.org/w/index.php?oldid=57058995&diff=prev
* nl.wikipedia.org: 3/114 look suspicious
Checking sr.wikipedia.org...
* sr.wikipedia.org: 0/1 look suspicious
Checking sv.wikipedia.org...
* 2020-09-07T10:38:19Z https://sv.wikipedia.org/w/index.php?oldid=48202797&diff=prev
* 2020-09-04T12:02:57Z https://sv.wikipedia.org/w/index.php?oldid=48191929&diff=prev
* 2020-09-04T07:44:29Z https://sv.wikipedia.org/w/index.php?oldid=48191097&diff=prev
* sv.wikipedia.org: 3/56 look suspicious
Checking zh.wikipedia.org...
* 2020-09-05T04:33:01Z https://zh.wikipedia.org/w/index.php?oldid=61500272&diff=prev
* 2020-09-03T08:00:25Z https://zh.wikipedia.org/w/index.php?oldid=61474788&diff=prev
* zh.wikipedia.org: 2/56 look suspicious
This comment was removed by Esanders.
This comment was removed by Esanders.

Never mind, it's not worrying, I just realized that my fix for T261715 is not deployed yet. It will be worrying if it continues happening next week.

kaldari added a subscriber: kaldari.Sep 9 2020, 3:15 PM

I'd like to hear more thoughts about https://ko.wikipedia.org/w/index.php?oldid=27478403&diff=prev&diffmode=source and https://en.wikipedia.org/w/index.php?oldid=977124090&diff=prev. For me, the most annoying feature/bug of VE is its tendency to add unexpected <nowiki> tags (although its much better than it used to be). If this problem also occurs with DiscussionTools, I imagine it will affect community buy-in for the feature. Is the addition of <nowiki> tags strictly necessary? When is such addition considered "correct" or "incorrect"? Is it possible to fully eliminate "incorrect" <nowiki> additions or is the problem too complex to decisively resolve? Sorry for my ignorance on this and if it's already been discussed elsewhere.

@kaldari - in the linked examples we are adding <nowiki>'s to other people's comments, so that is definitely something that selser should be catching. Hopefully the parsing team will be able to address that (T262411).

For when <nowiki>s are added to your own comments, this is done by Parsoid and as far as I know is always necessary based on the input given to them by the editor. Generally this is because someone typed or pasted some character in visual mode that has meaning in wikitext, e.g. '' [ |, http:// etc. There are things that can be done in the UI to warn users that they are in visual mode and not wikitext (as that is sometimes the cause) and we do have warnings in place where possible. We also do things like automatically parsing any wikitext that is pasted in to visual mode (provided we can detect it is wikitext).

I think users will be far more forgiving on talk pages though, because the wikitext output is not a curated artefact that users need to keep clean to aide subsequent edits, it just needs to work well enough and be legible until the conversation is eventually archived.

@Esanders - Thanks for the clarification. That is very helpful information.

Related tickets were created so this task is done. See T257651

(I updated my previous comments with links to the tasks Ed and I have filed about the dirty diffs)

This task was supposed to be only about verifying that T259855: DiscussionTools touched unrelated parts of the page was fixed, and it definitely is.

We accidentally identified some other issues and they are now tracked as subtasks of T262408: [TRACKING] Selser issues on talk pages.

We're calling it done for now, but we're thinking about a less "ad-hoc" way of reviewing edits made using the tool for dirty diffs: T261998: Make DiscussionTool dirty diff detection more accessible.

ppelberg closed this task as Resolved.Sep 11 2020, 11:17 PM