Page MenuHomePhabricator

Make Flow database available / accessible on Labs/Tools
Open, NormalPublic

Description

Author: metatron

Description:
As mentioned in the docs, Flow is designed to be a cross wiki database
https://www.mediawiki.org/wiki/Flow/Database

Please make this db and all its tables available accessible on Labs/Tools/Cloud

The key thing here is to properly redact the data respecting Flow's moderation model.


Version: unspecified
Severity: normal

Details

Reference
bz67397

Event Timeline

bzimport raised the priority of this task from to Normal.Nov 22 2014, 3:25 AM
bzimport added a project: Toolforge.
bzimport set Reference to bz67397.
bzimport created this task.Jul 1 2014, 10:15 PM

metatron wrote:

To allow Tool developers to adapt to the new format and create custom analysis.

coren moved this task from Triage to Backlog on the Toolforge board.Nov 25 2014, 4:11 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 24 2015, 8:02 PM
coren removed coren as the assignee of this task.Nov 17 2015, 2:40 PM
coren added a subscriber: coren.
bd808 edited projects, added Data-Services; removed Toolforge.Jul 27 2017, 12:46 AM
bd808 moved this task from Backlog to Wiki replicas on the Data-Services board.

@Ariel I wonder if importing from the dumps would be a good way to solve this.

It's not the only approach, but it would solve the moderation/redaction issues without us doing that specifically for this use case.

ArielGlenn removed a subscriber: Ariel.Jul 29 2017, 7:44 AM

There's already replication to WMF Cloud for most dbs; I'd prefer that we add the Flow db to that workflow. Among the benefits will be that its data will always be up-to-date, rather than relying on twice-monthly imports.

@Mattflaschen-WMF P.S. The 'Ariel' user exists only for purposes of the CoC committee; they needed a non WMF account for their purposes. Use the ArielGlenn one instead.

This seems rather blocked on replicating extension1 cluster, which is currently declined T121742. Unless that gets reconsidered, this bug is not going anywhere

This seems rather blocked on replicating extension1 cluster, which is currently declined T121742. Unless that gets reconsidered, this bug is not going anywhere

I think this task was opened specifically about replicating the flow tables to the Cloud Services Wiki Replicas. Is the flow redaction model wildly different than the redaction model for page revisions? Mostly I'm wondering if somehow we can't run these tables through the same sanitarium + filtered view workflow that we use for other wiki content. I would imagine that we would not be talking about replicating the actual Flow revision content from External Store, just the various metadata tables. There is some description of the suppression model at https://www.mediawiki.org/wiki/Extension:StructuredDiscussions/Moderation but we would want a Flow expert to help make sure we design the correct view logic.

This seems rather blocked on replicating extension1 cluster, which is currently declined T121742. Unless that gets reconsidered, this bug is not going anywhere

I think this task was opened specifically about replicating the flow tables to the Cloud Services Wiki Replicas. Is the flow redaction model wildly different than the redaction model for page revisions? Mostly I'm wondering if somehow we can't run these tables through the same sanitarium + filtered view workflow that we use for other wiki content. I would imagine that we would not be talking about replicating the actual Flow revision content from External Store, just the various metadata tables. There is some description of the suppression model at https://www.mediawiki.org/wiki/Extension:StructuredDiscussions/Moderation but we would want a Flow expert to help make sure we design the correct view logic.

Well all the flow metadata is in extension1, so I would assume that a precursor to that is replicating extension1 to sanitarium.

I'm not familar with the flow redaction model, but I can't imagine its really all that different, and can probably be handled the same way.

Restricted Application added a project: Growth-Team. · View Herald TranscriptJul 18 2018, 6:53 PM
SBisson moved this task from Inbox to Triaged but Future on the Growth-Team board.Jul 20 2018, 6:00 PM