Page MenuHomePhabricator

Make Flow database available / accessible on Labs/Tools
Open, MediumPublic

Description

Author: metatron

Description:
As mentioned in the docs, Flow is designed to be a cross wiki database
https://www.mediawiki.org/wiki/Flow/Database

Please make this db and all its tables available accessible on Labs/Tools/Cloud

The key thing here is to properly redact the data respecting Flow's moderation model.


Version: unspecified
Severity: normal

Details

Reference
bz67397

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 3:25 AM
bzimport added a project: Toolforge.
bzimport set Reference to bz67397.

metatron wrote:

To allow Tool developers to adapt to the new format and create custom analysis.

coren removed coren as the assignee of this task.Nov 17 2015, 2:40 PM
coren added a subscriber: coren.
bd808 moved this task from Backlog to Wiki replicas on the Data-Services board.

@Ariel I wonder if importing from the dumps would be a good way to solve this.

It's not the only approach, but it would solve the moderation/redaction issues without us doing that specifically for this use case.

There's already replication to WMF Cloud for most dbs; I'd prefer that we add the Flow db to that workflow. Among the benefits will be that its data will always be up-to-date, rather than relying on twice-monthly imports.

@Mattflaschen-WMF P.S. The 'Ariel' user exists only for purposes of the CoC committee; they needed a non WMF account for their purposes. Use the ArielGlenn one instead.

This seems rather blocked on replicating extension1 cluster, which is currently declined T121742. Unless that gets reconsidered, this bug is not going anywhere

This seems rather blocked on replicating extension1 cluster, which is currently declined T121742. Unless that gets reconsidered, this bug is not going anywhere

I think this task was opened specifically about replicating the flow tables to the Cloud Services Wiki Replicas. Is the flow redaction model wildly different than the redaction model for page revisions? Mostly I'm wondering if somehow we can't run these tables through the same sanitarium + filtered view workflow that we use for other wiki content. I would imagine that we would not be talking about replicating the actual Flow revision content from External Store, just the various metadata tables. There is some description of the suppression model at https://www.mediawiki.org/wiki/Extension:StructuredDiscussions/Moderation but we would want a Flow expert to help make sure we design the correct view logic.

This seems rather blocked on replicating extension1 cluster, which is currently declined T121742. Unless that gets reconsidered, this bug is not going anywhere

I think this task was opened specifically about replicating the flow tables to the Cloud Services Wiki Replicas. Is the flow redaction model wildly different than the redaction model for page revisions? Mostly I'm wondering if somehow we can't run these tables through the same sanitarium + filtered view workflow that we use for other wiki content. I would imagine that we would not be talking about replicating the actual Flow revision content from External Store, just the various metadata tables. There is some description of the suppression model at https://www.mediawiki.org/wiki/Extension:StructuredDiscussions/Moderation but we would want a Flow expert to help make sure we design the correct view logic.

Well all the flow metadata is in extension1, so I would assume that a precursor to that is replicating extension1 to sanitarium.

I'm not familar with the flow redaction model, but I can't imagine its really all that different, and can probably be handled the same way.