Update Flow for Parsoid changes re data-mw
Open, Needs TriagePublic

Description

See blocking tasks.

ssastry created this task.Jan 26 2016, 10:03 PM
ssastry updated the task description. (Show Details)
ssastry raised the priority of this task from to Needs Triage.
ssastry added a subscriber: ssastry.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald Transcript

I was under the impression that Flow stored HTML, and needs to convert to wikitext for editing. Is that still the case?

The stashing functionality in RB is useful to preserve wikitext when temporarily switching to HTML editing. Does this apply here?

The stashing functionality in RB is useful to preserve wikitext when temporarily switching to HTML editing. Does this apply here?

I was under the impression that stashing was the exact feature lacking in RB for Flow to adopt it.

I was under the impression that Flow stored HTML, and needs to convert to wikitext for editing. Is that still the case?

Yes (although the HTML can also be edited with VE).

Also, we have a switcher function (HTML <-> wikitext <-> HTML) (which we should make more performant). You can switch at any time (i.e. mid-edit, etc.)

The specific reason we stopped using RESTBase's VirtualRestService is problems with data-parsoid being removed. See rEFLWe359e9c166df: Temporarily disable RESTbase support to avoid data-parsoid issues. The linked tasks from that are T115236: Flow posts being serialized from HTML -> WT without providing Parsoid data-parsoid attributes?, T112350: [Regression] In betalabs "Due to a technical error, this post could not be retrieved." for entries with triple curly brackets, T113044: Converting {{{foo}}} from wikitext to html to wikitext returns a 500 error.

Okay, so it sounds like the issue is really getting data-parsoid and data-mw from the wt2html end point, and storing it along the HTML in Flow storage.

I see two main options:

1) Provide pagebundle end points for wt2html and html2wt.

These are JSON blobs containing data-parsoid, html & later data-mw, and could be stored in Flow storage.

Difficulty on the RESTBase side: easy.
Difficulty on the Flow side: easy-moderate; does not solve change propagation.

2) Move Flow storage to RESTBase, as discussed in T94574.

While the storage itself is fairly straightforward, we would not be able to use much of the Parsoid logic due to its assumption that content is available on-wiki. The change propagation issues mentioned on the task are a separate issue altogether.

Difficulty on RESTBase side: Moderate-complex, especially change propagation.
Difficulty on Flow side: moderate?

Mattflaschen-WMF renamed this task from Migrate Flow to talk with RESTBase instead of Parsoid to Update Flow for Parsoid changes re data-mw.EditedFeb 4 2016, 6:26 PM
Mattflaschen-WMF edited projects, added Parsoid; removed RESTBase.
Mattflaschen-WMF set Security to None.

After meeting, we decided to still talk to Parsoid: https://etherpad.wikimedia.org/p/Flow-Restbase-2016-02-04

Arlolra moved this task from Backlog to Non-Parsoid Tasks on the Parsoid board.Feb 5 2016, 5:24 AM
StjnVMF triaged this task as High priority.May 18 2018, 5:17 PM
JJMC89 closed subtask T194980: Triage as Invalid.May 18 2018, 5:42 PM
JJMC89 raised the priority of this task from High to Needs Triage.May 18 2018, 5:46 PM
Restricted Application added a project: Growth-Team. · View Herald TranscriptSep 2 2018, 10:14 AM
SBisson closed this task as Resolved.Sep 7 2018, 2:09 PM
SBisson claimed this task.
SBisson added a subscriber: SBisson.

No activity for 2.5 years. I assume this is not a problem anymore.

ssastry reopened this task as Open.Sep 7 2018, 3:07 PM

No, this will need to be done when (not if) data-mw split happens, unless we explicitly resolve we will maintain a separate flag / endpoint for Flow to preserve the old format.

@ssastry My bad. Do you have an idea when that will be needed?

@ssastry My bad. Do you have an idea when that will be needed?

Not yet .. we've delayed doing this till it gets close to when we absolutely have to do this. We'll provide sufficient advance notice before we actually do this.

SBisson moved this task from Inbox to Triaged but Future on the Growth-Team board.Sep 7 2018, 3:53 PM

@ssastry Thanks. I'll put it in "future" until you tell us it is needed.