According to @MMiller_WMF , the PrefUpdate schema used to record thousands of events per day on frwiki, until May 11th, when it started recording ~10 events per day. This corresponds to the deployment of 1.35.0-wmf.31, and I believe this is caused by T253149: Regression: UserSaveOptions hook no longer exposes old preference values

Details

Related Changes in Gerrit:

	Subject	Author	Repo	Branch	Lines +/-
	Use $originalOptions in UserSaveOptions hook	Ppchelko	mediawiki/extensions/WikimediaEvents	master	+8 -5

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		• Pchelolo	T253149 Regression: UserSaveOptions hook no longer exposes old preference values
		Resolved		None	T253151 PrefUpdate schema no longer receiving (most) events

Event Timeline

Catrope created this task.May 19 2020, 7:33 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 19 2020, 7:33 PM

Catrope added a parent task: T253149: Regression: UserSaveOptions hook no longer exposes old preference values.May 19 2020, 7:34 PM

Catrope mentioned this in T253149: Regression: UserSaveOptions hook no longer exposes old preference values.

• Niharika subscribed.May 19 2020, 8:59 PM

kzimmerman added a project: Product-Analytics.May 19 2020, 10:03 PM

Mayakp.wiki subscribed.May 19 2020, 10:14 PM

MNeisler subscribed.May 19 2020, 11:04 PM

Can confirm that only a limited amount of data starts flowing in on 2019-05-11. Per the two queries below, I find the drop to start from 17:00 UTC on 2019-05-11, and continues from then on.

SELECT substring(dt, 1, 13) AS log_hour, count(*) AS num_events
FROM event.prefupdate
WHERE year = 2020
AND month = 5
AND day = 11
GROUP BY substring(dt, 1, 13)
LIMIT 25;

Output:

log_hour	num_events
2020-05-11T00	2812
2020-05-11T01	2480
2020-05-11T02	2380
2020-05-11T03	2213
2020-05-11T04	2586
2020-05-11T05	2611
2020-05-11T06	3432
2020-05-11T07	3730
2020-05-11T08	4449
2020-05-11T09	4551
2020-05-11T10	5161
2020-05-11T11	5075
2020-05-11T12	5434
2020-05-11T13	5817
2020-05-11T14	6450
2020-05-11T15	6094
2020-05-11T16	5645
2020-05-11T17	492
2020-05-11T18	207
2020-05-11T19	273
2020-05-11T20	236
2020-05-11T21	208
2020-05-11T22	147
2020-05-11T23	109

SELECT log_date, num_events
FROM (
    SELECT TO_DATE(dt) AS log_date, count(*) AS num_events
    FROM event.prefupdate
    WHERE year = 2020
    AND month >= 5
    GROUP BY TO_DATE(dt)
) AS pfe
ORDER BY log_date
LIMIT 50;

Output:

log_date	num_events
2020-05-01	97795
2020-05-02	95602
2020-05-03	96518
2020-05-04	102007
2020-05-05	101922
2020-05-06	99562
2020-05-07	89037
2020-05-08	94462
2020-05-09	90715
2020-05-10	94350
2020-05-11	72592
2020-05-12	4850
2020-05-13	4590
2020-05-14	5366
2020-05-15	4980
2020-05-16	5049
2020-05-17	4971
2020-05-18	4787
2020-05-19	4077

This issue does not affect any currently running Growth Team experiments, as far as I know. It'll affect future ones, though, as we've been using this schema to identify users who self-select in/out of our experiment groups.

LGoto moved this task from Triage to Tracking on the Product-Analytics board.May 26 2020, 5:12 PM

feel free to tag us if this is not an instrumentation problem

Change 599373 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[mediawiki/extensions/WikimediaEvents@master] Use $originalOptions in UserSaveOptions hook

https://gerrit.wikimedia.org/r/599373

gerritbot added a project: Patch-For-Review.May 28 2020, 4:17 PM

Change 599373 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Use $originalOptions in UserSaveOptions hook

https://gerrit.wikimedia.org/r/599373

Maintenance_bot removed a project: Patch-For-Review.May 28 2020, 5:11 PM

Since this task is on the Product Analytics tracking board, and we in the team were wondering what's going on, I looked into it. The patch rode the wmf.35 train last week, and data is flowing in again. We can see this on the output from the last of my queries above, which run now produces the following:

log_date	num_events
2020-05-01	97795
2020-05-02	95602
2020-05-03	96518
2020-05-04	102007
2020-05-05	101922
2020-05-06	99562
2020-05-07	89037
2020-05-08	94462
2020-05-09	90715
2020-05-10	94350
2020-05-11	72592
2020-05-12	4850
2020-05-13	4590
2020-05-14	5366
2020-05-15	4980
2020-05-16	5049
2020-05-17	4971
2020-05-18	4787
2020-05-19	4737
2020-05-20	4720
2020-05-21	4392
2020-05-22	4027
2020-05-23	4944
2020-05-24	4442
2020-05-25	4907
2020-05-26	4793
2020-05-27	4593
2020-05-28	5389
2020-05-29	4916
2020-05-30	4516
2020-05-31	4287
2020-06-01	4500
2020-06-02	4996
2020-06-03	7318
2020-06-04	51510
2020-06-05	147511
2020-06-06	149787
2020-06-07	145379
2020-06-08	151910

I also went and changed some of my preferences on cswiki and confirmed that these triggered PrefUpdate events in kafka. Based on this, I'm expecting this task and its parent to get closed relatively soon.

• nettrom_WMF mentioned this in T256178: Data missing in event_prefupdate in Druid.Jun 23 2020, 8:28 PM

MNeisler mentioned this in T249386: Calculate adoption of Replying tool (Beta Features).Jun 30 2020, 11:03 PM

Krinkle closed this task as Resolved.Oct 30 2021, 12:47 AM

PrefUpdate schema no longer receiving (most) eventsClosed, ResolvedPublicActions

Description

Details

Related ObjectsSearch...

Event Timeline

PrefUpdate schema no longer receiving (most) events
Closed, ResolvedPublic
Actions

Related Objects
Search...