Page MenuHomePhabricator

update the risk observatory usage of the ipblocks
Open, LowPublic

Description

The risk observatory depends on the ipblocks table, which has been moved to the private replica servers. The usage of that table in research-datasets needs to be migrated to the new table mediawiki_private_ipblocks.

From the announcement: The ipblocks table has recently been removed from the wikireplicas in: T390767 Remove the compatibility layer of block schema in wikireplicas. As a result, we have had to change the way that we ingest this table into the data lake during our monthly sqoop. This data is now being ingested from the private mediawiki replica servers.

Event Timeline

Thanks so much for creating this task! @fkaelin @Pablo how urgent is this task? I.e. is this breaking the most important plots (as in: most viewed by users) in the dashboard? Also is this a major change - how long would it take to make this change?

Thank you both!

As far as I know, the plots in the main tab are rarely used. The Risk Observatory is primarily used because of the revert risk prediction data (e.g., the Moderators Tools team when calibrating Automoderator, myself for the patrolling dataset T392210). That said, Trust & Safety recently told me they still use the secondary tabs for various tasks (in particular, focusing on key current events to see the quantity of high risk revisions that were happening on certain pages).

I would not mark this as urgent, but the sooner it is fixed, the better :)