Page MenuHomePhabricator

Update mediawiki_user_blocks_change to log partial block parameters
Closed, ResolvedPublic

Description

The event.mediawiki_user_blocks_change table in the Data Lake does not log partial block parameters. This means that it's impossible to determine if a block is sitewide or partial, and if it's the latter what pages/namespaces and such were blocked.

Here's an example from Italian Wikipedia:

In the logging table in MediaWiki:

| 51086528 | block    | block      | 20190622231903 |             2 | Marco1233333 |             44 | a:4:{s:11:"5::duration";s:7:"8 hours";s:8:"6::flags";s:8:"nocreate";s:8:"sitewide";b:0;s:15:"7::restrictions";a:1:{s:10:"namespaces";a:1:{i:0;s:1:"0";}}} |           0 |       474 |        0 |

In event.mediawiki_user_blocks_change, this event is logged as follows:

blocks	comment	database	meta	performer	user_groups	user_text	prior_stateuser_id	_schema	datacenter	year	month	day	hour
{"account_create":true,"email":false,"expiry_dt":"2019-06-23T07:19:03Z","name":false,"user_talk":false}	Ripetuto inserimento di contenuto senza senso	itwiki	{"domain":"it.wikipedia.org","dt":"2019-06-22T23:19:03Z","id":"1f7b5c06-9544-11e9-a4e1-b083fecf1287","request_id":"XQ63ZwpAMFYAAHxlPasAAABF","schema_uri":null,"topic":null,"uri":"https://it.wikipedia.org/wiki/Utente:Marco1233333","stream":"mediawiki.user-blocks-change"}	{"user_edit_count":29461,"user_groups":["sysop","*","user","autoconfirmed"],"user_id":1217526,"user_is_bot":false,"user_registration_dt":"2015-05-21T21:55:43Z","user_text":"Dan Kenshi"}	["*","user"]	Marco1233333	{"blocks":{"account_create":null,"email":null,"expiry_dt":null,"name":null,"user_talk":null}}	1814692	/mediawiki/user/blocks-change/1.0.0	eqiad	2019	6	22	23

With partial blocks being available on all wikis, it would be meaningful to be able to use event.mediawiki_user_blocks_change as a data source for user block data (see for instance T252225#6125157).

Event Timeline

fdans moved this task from Incoming to Data Quality on the Analytics board.

cc @WDoranWMF and @Pchelolo events sent by MediaWiki need to be augmented with the new parameters.

Change 599447 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[schemas/event/primary@master] Add sitewide and restrictions properties to user-blocks-change schema

https://gerrit.wikimedia.org/r/599447

Change 599447 merged by Ottomata:
[schemas/event/primary@master] Add sitewide and restrictions properties to user-blocks-change schema

https://gerrit.wikimedia.org/r/599447

Change 603585 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[schemas/event/primary@master] user/blocks-change - Add missing type for page_restrictions type field

https://gerrit.wikimedia.org/r/603585

Change 603585 merged by Ottomata:
[schemas/event/primary@master] user/blocks-change - Add missing type for page_restrictions type field

https://gerrit.wikimedia.org/r/603585

Change 603687 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[schemas/event/primary@master] user-block-change: rename page_restrictions to restrictions.

https://gerrit.wikimedia.org/r/603687

Change 603689 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[schemas/event/primary@master] user-blocks-change: rename namespace to ns restriction type

https://gerrit.wikimedia.org/r/603689

Change 603687 merged by Ppchelko:
[schemas/event/primary@master] user-block-change: rename page_restrictions to restrictions.

https://gerrit.wikimedia.org/r/603687

Change 603689 merged by Ppchelko:
[schemas/event/primary@master] user-blocks-change: rename namespace to ns restriction type

https://gerrit.wikimedia.org/r/603689

Change 603691 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[mediawiki/extensions/EventBus@master] Emit user-block-change/1.1.0 event

https://gerrit.wikimedia.org/r/603691

Change 603691 merged by jenkins-bot:
[mediawiki/extensions/EventBus@master] Emit user-block-change/1.1.0 event

https://gerrit.wikimedia.org/r/603691

Verified that the new fields are being emitted now.

Change 660886 had a related patch set uploaded (by Paladox; owner: Ppchelko):
[mediawiki/extensions/EventBus@REL1_35] Emit user-block-change/1.1.0 event

https://gerrit.wikimedia.org/r/660886

Change 660886 merged by jenkins-bot:
[mediawiki/extensions/EventBus@REL1_35] Emit user-block-change/1.1.0 event

https://gerrit.wikimedia.org/r/660886