https://de.wikipedia.org/w/index.php?title=National_Register_of_Historic_Places_in_Arizona&curid=9675366&diff=171746522&oldid=171745883 has a paragraph which is indicated as moved, but the paragraph wasn't moved, since all paragraphs between the "old" and the "new" place have been deleted in the edit.
Description
Description
Related Objects
Related Objects
Event Timeline
Comment Actions
And the other way round, too: https://de.wikipedia.org/w/index.php?title=Maria_Canins&curid=5138507&diff=171746679&oldid=171745939
Most of the "moved" paragraphs weren't moved, the diff totally obscures the fact that only a) headlines were inserted, and b) paragraphs have been split.
The only real move in this example is the block with the references.
Comment Actions
It even happens for paragraphs with nothing between them: https://de.wikipedia.org/w/index.php?title=Frankenhardt&curid=502187&diff=171771634&oldid=170255294
Comment Actions
I had a look at the provided examples and compared what the table at https://wmde-wikidiff2-unpatched.wmflabs.org/core/index.php/Main_Page#Test_Pages provides in the "old" and "new" columns:
- The diff for "National Register of Historic Places in Arizona" looks better than before. Instead of deleting |Navigationsleiste Listen der National Historic Places der Vereinigten Staaten and inserting {{Navigationsleiste Listen der National Historic Places der Vereinigten Staaten}}, the algorithm successfully detects that these two lines are almost identical (except for the special characters), and links them.
- I'm afraid I don't understand what's wrong with the "Maria Canins" diff. When I compare old and new algorithm, the new seems to be so much more helpful. Large paragraphs are linked, instead of just appearing deleted and inserted. The appearance of the headlines is not very different between the two, so I really can't tell what is "obscured" there.
- The diff for "Frankenhardt" is just … a bit different. The amount of information I can see in the two diffs is pretty much identical. The old algorithm linked paragraphs purely based on their position, which was ok for this particular diff. The new algorithm links paragraphs when they are significantly similar. This results in a diff that's a little more spread out, but not worse, from what I can see.
I'm afraid there is nothing we can do here at the moment. Can you help us understand the problem better?
Comment Actions
Old diff (1):
Current diff (2):
Best diff (3):
Of course, (2) is better than (1), but that is no reason why it shouldn't be (3), which is even better than (2), and that's what this task is about.
Comment Actions
Ah, I see. Thanks for the clarification! The problem here is the way two separate algorithms work together: The first line-matching algorithm does it's best to put identical lines side by side. Only then the second moved paragraph algorithm (this is the one we introduced recently) tries to re-match disjoint lines and links them together. Having these two algorithms separated is required for situations where lines have been moved.
What we see in screenshot (2) is an edge-case where the second algorithm is a bit more clever in finding matches than the first. This is by design. The line-matcher is intentionally trivial and does not try to match "almost identical" lines, but only lines that are either truly identical, or at the same position. Neither is the case in the provided example. Only then the second algorithm kicks in, finds a match, but does not move anything. This is also by design.
Yes, it's technically possible to introduce special-case handling for situations like this. I can imaging two approaches:
- Try to make the line-matcher "softer" so it can find vague matches the same way the moved paragraph algorithm does. The biggest problem here is: this is heavily CPU intensive and probably nothing we can do.
- Apply a third "clean-up" algorithm after everything else was done. This "clean-up" could look for situations like shown in the screenshot where a move was detected, and there is an empty gap in the diff that allows us to actually move the paragraph.
I need to point out this is not a Regression but a new feature. We will see if we find the time to work on this.
Comment Actions
Here are more examples that would benefit from the same optimization:
- https://wmde-wikidiff2-patched.wmflabs.org/core/index.php?diff=9360
- https://wmde-wikidiff2-patched.wmflabs.org/core/index.php?diff=9440
- https://wmde-wikidiff2-patched.wmflabs.org/core/index.php?diff=9973
- https://wmde-wikidiff2-patched.wmflabs.org/core/index.php?diff=11393
- https://wmde-wikidiff2-patched.wmflabs.org/core/index.php?diff=11757
- https://wmde-wikidiff2-patched.wmflabs.org/core/index.php?diff=11962
- https://wmde-wikidiff2-patched.wmflabs.org/core/index.php?diff=12310
- https://wmde-wikidiff2-patched.wmflabs.org/core/index.php?diff=13380
- https://wmde-wikidiff2-patched.wmflabs.org/core/index.php?diff=13467
All these cases are already better than before, so again this is not a bug in what we did so far. Compare by replacing "patched" with "unpatched" in the URL.
Comment Actions
This bug seems to describe a more general issue with the visualization of diffs that is already covered in another ticket. I'll merge it accordingly.