With the introduction of partial blocks, ipblocks_restrictions table was added to the schema. This table does not contain any private data and it should be exposed to the replicas.
- Mentioned In
- T209549: Add ipblocks_restrictions table to Data Lake
- Mentioned Here
- T210002: Oversighters should only be able to suppress a username for sitewide blocks (not partial blocks) via "Hide username from edits and lists" checkbox
T209031: Not able to scoop comment table in labs for mediawiki reconstruction process
T209549: Add ipblocks_restrictions table to Data Lake
@TBolliger, I don't know enough about this table to know whether it's good for it to be available on the cloud replicas. If it's ok, then once this task is done it would be relatively easy for us to sqoop the table and make it available in Hadoop as well.
However, if this table should NOT be publicly available in the cloud replicas, then we can sqoop it from our production replicas separately. Then it would be available only in Hadoop. We might not have time to do this before the end of this quarter, but we'll get it done afterwards. I'll tag T209549 as well to keep these discussions in sync.
I thought it'd be useful to clarify a bit. Importantly, the point I'm making here about filtered-tables has no relation to the different discussion about sanitizing data in Hadoop, happening in T209031.
Relevant to this discussion, we prefer to give cloud replicas a chance first. Because this way everyone can have access to the same data, whether they work with the WMF directly or not. If that doesn't work, we'll import the data directly into Hadoop.
Confirmed. No private data or similar. One is a link to ipblocks, the other two are just simple numbers
-- -- Partial Block Restrictions -- CREATE TABLE /*_*/ipblocks_restrictions ( -- The ipb_id from ipblocks ir_ipb_id int NOT NULL, -- The restriction type id. ir_type tinyint(1) NOT NULL, -- The restriction id that corrposponds to the type. Typically a Page ID or a -- Namespace ID. ir_value int NOT NULL, PRIMARY KEY (ir_ipb_id, ir_type, ir_value) ) /*$wgDBTableOptions*/; -- Index to query restrictions by the page or namespace. CREATE INDEX /*i*/ir_type_value ON /*_*/ipblocks_restrictions (ir_type, ir_value);