Page MenuHomePhabricator

Switch Flow from ExternalStore to RESTBase
Closed, DeclinedPublic

Description

Switching to RESTBase makes sense if it allows us to solve the problems we're having with ExternalStore, such as:

RESTBase has a PHP extension that listens to updates and automatically tells Parsoid to regenerate the HTML and store the updated version in RESTBase. We would want these to also take into templatelinks, etc. dependencies for Flow. Flow might need to hook in somehow to tell it how our data model works.


See T125857: Change propagation for Flow for the canonical task about the actual problem here.

Event Timeline

Any ETA on this?

Note that the Parsoid v1 Varnish cache still exists, but there are no update jobs running any more for it, meaning that cached content still exists, but it's stale.

Any ETA on this?

Note that the Parsoid v1 Varnish cache still exists, but there are no update jobs running any more for it, meaning that cached content still exists, but it's stale.

We don't use the Parsoid content of standard wiki pages (except https://gerrit.wikimedia.org/r/#/c/230648/18 will, but is not currently using RESTBase, though I believe it could).

We convert back and forth using the POST API (https://git.wikimedia.org/blob/mediawiki%2Fextensions%2FFlow.git/9b180d3af89dd484494c615622ac53118772cf9a/includes%2FParsoid%2FUtils.php#L68).

This might be a bigger change than it sounds like, since we have our own revision model and links table (in addition to the standard link tables).

It sounds like we may actually not be affected at all, unless Varnish is caching POST requests, until you actually want to kill this Parsoid API entirely. If so, we should probably meet.

Yes, we do want to kill Parsoid's v1 API. Maintaining multiple versions of the API adds to maintenance burden. While we've refactored the code internally over time to minimize this burden, it is still undesirable. So, once the v3 API (T100680, and is much closer in url form to the RESTBase API) is well tested, we would encourage you to switch over to using it. It is going live today.

The POST API still exists; the URL scheme is just slightly different. Flow should use the VRS service for Parsoid like everyone else; see https://gerrit.wikimedia.org/r/214351 and https://gerrit.wikimedia.org/r/233439.

Even $wmgParsoidURL and $wgParsoidWikiPrefix are going away.

Okay, sounds like next step (soon, and before conversion to RESTBase) is to convert to VRS and Parsoid v3 (T110721: Switch Flow to use Parsoid v3 API with VirtualRESTService).

Change 234667 had a related patch set uploaded (by Cscott):
Use the VirtualRESTService to configure Flow

https://gerrit.wikimedia.org/r/234667

@Mattflaschen Once Flow uses VRS, switching to RESTBase will happen magically, since that's the default VRS configuration for production.

@Mattflaschen Once Flow uses VRS, switching to RESTBase will happen magically, since that's the default VRS configuration for production.

This task is about storing Flow content in RESTBase and automatically updating it in response to template changes, etc., not just using the RESTBase API for wikitext <=> HTML conversions.

Mattflaschen-WMF renamed this task from Switch Flow to RESTBase to Switch Flow from ExternalStore to RESTBase.Aug 28 2015, 8:09 PM

Relevant to this, we are preparing to migrate from one External Store cluster to another Flow-specific External Store (same technology, just Flow-specific DB) (T106363: Migrate Flow content to new separate logical External Store in production). But if this makes more sense, we could do it instead.

See T124837: Update Flow for Parsoid changes re data-mw.

GWicke lowered the priority of this task from High to Low.Oct 12 2016, 5:44 PM
GWicke added a project: Services (later).
Pchelolo subscribed.

No movement in 3 years clearly indicates this is not needed. Please reopen if I'm mistaken.