Page MenuHomePhabricator

Restore polling page-properties-change
Closed, DeclinedPublic

Description

We've disabled polling page-properties-change channel for Kafka poller for performance reasons, because it nearly doubles the workload. We should find a way to re-enable it.

The event looks like this:

{"added_properties": {"wb-claims": 22}, "database": "wikidatawiki", "meta": {"domain": "www.wikidata.org", "dt": "2018-10-26T05:10:55+00:00", "id": "8412e0e1-d8dd-11e8-8284-1866da98024d", "request_id": "9c065d2d-6c85-4b84-9c31-74825c1d339d", "schema_uri": "mediawiki/page/properties-change/2", "topic": "mediawiki.page-properties-change", "uri": "https://www.wikidata.org/wiki/Q50949694"}, "page_id": 51876966, "page_is_redirect": false, "page_namespace": 0, "page_title": "Q50949694", "performer": {"user_edit_count": 3975626, "user_groups": ["bot", "*", "user", "autoconfirmed"], "user_id": 2991187, "user_is_bot": true, "user_registration_dt": "2018-07-15T00:21:35Z", "user_text": "Citationgraph bot 2"}, "removed_properties": {"wb-claims": "21"}, "rev_id": 773455651}

Looking at added_properties and removed_properties, we can process this event without reloading the whole entity, though it would probably require some special handling.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Smalyshev triaged this task as Medium priority.Dec 22 2018, 2:08 AM
Gehel subscribed.

This is going to be obsolete with the new streaming updater