Page MenuHomePhabricator

Adding or editing citations using VisualEditor causes major formatting issues involving pipes, equals signs and nowiki tags
Closed, ResolvedPublicBUG REPORT

Description

This bug was first reported in other users' edits by User:Jessicapierce on 27 June 2019.

Steps to Reproduce:

Unknown as of yet, but occurs on some occasions when adding or editing citations in an article using VisualEditor.

Actual Results:

Numerous strings and alphanumerical characters are substituted into the affected text from neighbouring sentences, including their own citations, resulting in garbled text with some letters substituted by pipes and equals signs. Nowiki tags are placed all across the mess, sometimes corresponding to non-markup text (sentences) that has been implanted but sometimes not.

  • Example 1 – the editor removes a sentence and replaces it with a paragraph with a new citation at the end, but this garbles the unrelated citation preceding the removed sentence by substituting various pipes and equals signs inside it, as well as substituting the 2 first words of the removed sentence. All subsequent citations in the entire article are garbled by similar removals of neighbouring sentences and/or citations.
  • Example 2 – the editor adds a new citation to a sentence with 3 existing citations, but this garbles all previous citations (not the new one) by substituting a garbled version of the sentence after the new citation (with equals signs and pipes substituted inside it at random) and substituting various parts of the existing citations into the ones previous.
  • Example 3 – the editor replaces a citation with one used previously in the article, but this causes all of the characters of the previous citation to remain as some sort of ghost placeholders and their contents be replaced by a garbled version of the citation used in the unrelated next sentence.
  • Example 4 – the editor copies a citation already present in the article and cites it again somewhere else, but this garbles the sentences before and after the new citation, substituting some of their letters with curly brackets, pipes and equals signs.
  • Example 5 – too hectic to determine details

Expected Results:

The citation is added or edited and no other disruptive changes are made to the markup.

Details

Related Gerrit Patches:

Event Timeline

SUM1 created this task.Jul 3 2019, 6:06 PM
Restricted Application added subscribers: jeblad, Aklapper. · View Herald TranscriptJul 3 2019, 6:06 PM
SUM1 updated the task description. (Show Details)Jul 3 2019, 6:15 PM
SUM1 updated the task description. (Show Details)Jul 3 2019, 6:21 PM
SUM1 updated the task description. (Show Details)Jul 3 2019, 6:29 PM
SUM1 updated the task description. (Show Details)
SUM1 updated the task description. (Show Details)Jul 3 2019, 6:39 PM
SUM1 updated the task description. (Show Details)Jul 3 2019, 6:45 PM
SUM1 renamed this task from Adding citations using VisualEditor causes major formatting issues involving pipes, equals signs and nowiki tags to Adding or editing citations using VisualEditor causes major formatting issues involving pipes, equals signs and nowiki tags.Jul 3 2019, 7:10 PM
SUM1 updated the task description. (Show Details)
SUM1 updated the task description. (Show Details)
Jdforrester-WMF triaged this task as Unbreak Now! priority.Jul 3 2019, 9:36 PM
Jdforrester-WMF added a project: Parsoid.
Jdforrester-WMF added a subscriber: Jdforrester-WMF.

Have encountered this myself, but can't isolate.

Possibly a Parsoid bug? DOM corruption like this is very unlikely to be from VE, given that it involves rewriting elements to be random offset strings.

Restricted Application added a subscriber: Liuxinyu970226. · View Herald TranscriptJul 3 2019, 9:36 PM
cscott added a subscriber: cscott.Jul 3 2019, 9:43 PM

Looks like the DSR offsets are getting scrambled and/or the base text is changing, and selser is substituting regions from a different wikitext and/or from an offset region. Seems to require that a parameter of the ref is changed, so that selser isn't skipping over the entire ref but is instead trying to do a finer-grainer selective serialization of each parameter.

(Ignore the <nowiki> cruft, it's just what's happening when we attempt to wikitext escape a corrupted string with lots of wiki metacharacters in it.)

It's *possible* that VE is to blame, if for some reason it's not marking subtrees as 'changed' properly, but I agree that this is probably a Parsoid selser bug.

ssastry added a subscriber: ssastry.EditedJul 3 2019, 9:44 PM

It's *possible* that VE is to blame, if for some reason it's not marking subtrees as 'changed' properly, but I agree that this is probably a Parsoid selser bug.

VE won't mark diffs .. we run dom-diff and mark changed regions. So, I thnk this is likely Parsoid.

@cscott: one way to narrow this down is to run the deployed version of Parsoid on the revision that was edited and try a selser edit.

@SUM1, can you let us know if the reporting user was switching back and forth between Wikitext and VisualEditor?

(Only the fifth example has a "Visual edit: Switched" tag)

SUM1 added a comment.EditedJul 3 2019, 10:53 PM

@ssastry The reporting user didn't experience the bug herself (to my knowledge) but encountered numerous other people's edits being affected.

I just tested switching back and forth between Wikitext and VisualEditor in the sandbox, and yes, this induced the bug. I'll update the description.

@ssastry The reporting user didn't experience the bug herself (to my knowledge) but encountered numerous other people's edits being affected.

Are there other instances besides the five you reported above?

I just tested switching back and forth between Wikitext and VisualEditor in the sandbox, and yes, this induced the bug. I'll update the description.

Oh great! We've been trying all kinds of things and are struggling to reproduce it. So, it would be helpful to provide steps to reproduce this. Thanks!

SUM1 added a comment.Jul 3 2019, 10:59 PM

@ssastry I'm back at a standstill, can't reproduce the bug again. I'll keep testing.

And not yet, but I can go searching for affected edits.

@ssastry I'm back at a standstill, can't reproduce the bug again. I'll keep testing.
And not yet, but I can go searching for affected edits.

@SUM1 .. so, Example 2 is your edit. Do you know if you switched between VE & wikitext or was it a straight up VE-only edit?

(Only the fifth example has a "Visual edit: Switched" tag)

One theory we were considering was if it was VE <-> wikitext <-> VE .. so not a switch. Presumably that tag is added only if the editor switched from VE to Wikitext.

SUM1 added a comment.Jul 3 2019, 11:09 PM

@ssastry I don't recall switching at all when I made my Prion edit.

Presumably that tag is added only if the editor switched from VE to Wikitext.

Apparently not. All my edits have that tag, including the majority in which I didn't switch editors.

SUM1 added a comment.Jul 3 2019, 11:16 PM

I can't reproduce the edit at all in my sandbox, and I believe my first report was a false alarm caused by confusing markup diff appearance.

I review all my edits before I submit them, and I can't see why I would've missed a glaring red warning on the page editor or the garbled mess in the diffs. I have a feeling this bug only occurs after the edit is submitted.

SUM1 updated the task description. (Show Details)Jul 3 2019, 11:20 PM

@SUM1, Thanks for your help .. we may have found some leads to track this down.

Change 520652 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/services/parsoid@master] Set top frame's source text when parsing from a stash

https://gerrit.wikimedia.org/r/520652

Change 520652 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Set top frame's source text when parsing from a stash

https://gerrit.wikimedia.org/r/520652

Instructions from @cscott to reproduce this bug:

1. https://en.wikipedia.beta.wmflabs.org/wiki/Martyrs_of_Gorkum?veaction=edit
2. click VE 'cite' button, insert manual citation.  i used "website", then URL "https://cscott.net" and title "title"
3. now switch to wikitext editor, confirm inserted reference is sane
4. now switch back to VE, click the reference, click edit.
5. the url will be corrupt

I confirmed the bug with these steps.

These steps will probably work on any production wiki as well.

Mentioned in SAL (#wikimedia-operations) [2019-07-04T00:15:02Z] <cscott@deploy1001> Started deploy [parsoid/deploy@af5fd0e]: Updating Parsoid to d355bc90 (deploy-20170703 branch, T227216)

Confirmed fixed for me.

These steps will probably work on any production wiki as well.

Yes, confirmed on [[enwiki:Hampi]]. Now, waiting for deploy to complete before I can re-try that and verify it is fixed.

Mentioned in SAL (#wikimedia-operations) [2019-07-04T00:21:50Z] <cscott@deploy1001> Finished deploy [parsoid/deploy@af5fd0e]: Updating Parsoid to d355bc90 (deploy-20170703 branch, T227216) (duration: 06m 48s)

These steps will probably work on any production wiki as well.

Yes, confirmed on [[enwiki:Hampi]]. Now, waiting for deploy to complete before I can re-try that and verify it is fixed.

Confirmed fixed after the deploy there.

Jdforrester-WMF closed this task as Resolved.Jul 4 2019, 12:25 AM
Jdforrester-WMF assigned this task to c_schmitz.
Restricted Application added a project: User-Ryasmeen. · View Herald TranscriptJul 4 2019, 12:25 AM
Jdforrester-WMF added a subscriber: c_schmitz.

(Note that after you've edited the page once Visual Editor will helpfully "restore your saved edits", reintroducing corruption. That makes it hard to figure out if you're still triggering the bug or if it has been fixed. The wise @Jdforrester-WMF says, "if you try to do this a second time you'll need to switch to the 'read tab', click 'discard edits', then click back to the 'edit' tab".)

Mentioned in SAL (#wikimedia-operations) [2019-07-24T21:00:17Z] <cscott@deploy1001> Started deploy [parsoid/deploy@abd05ab]: Updating Parsoid to df1af404 (T227216, T226523, T226451)

Mentioned in SAL (#wikimedia-operations) [2019-07-24T21:18:52Z] <cscott@deploy1001> Finished deploy [parsoid/deploy@abd05ab]: Updating Parsoid to df1af404 (T227216, T226523, T226451) (duration: 18m 35s)