Page MenuHomePhabricator

PrefUpdate Event Platform Migration
Closed, ResolvedPublic

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Ottomata renamed this task from PrefUpdate to Event Platform Migration to PrefUpdate Event Platform Migration.Nov 5 2020, 4:48 PM

@jlinehan
Let us know if this schema needs client IP and/or geocoded data? If not, it will be removed as part of this migration.

@mpopov Let us know if this schema needs client IP and/or geocoded data? If not, it will be removed as part of this migration.

I'd like to migrate this schema soon. If possible this week! If not, next week is fine.

According to @nettrom_WMF and @MNeisler, client_ip and geocoded data can be removed

This schema will be migrated starting the week of Feb 1 - Feb 5 2021.

Change 661706 had a related patch set uploaded (by Mforns; owner: Mforns):
[schemas/event/secondary@master] Add PrefUpdate schema to analytics/legacy

https://gerrit.wikimedia.org/r/661706

Change 661706 merged by Mforns:
[schemas/event/secondary@master] Add PrefUpdate schema to analytics/legacy

https://gerrit.wikimedia.org/r/661706

Change 661709 had a related patch set uploaded (by Mforns; owner: Mforns):
[operations/mediawiki-config@master] Migrate PrefUpdate from EventLogging to EventGate on testwiki

https://gerrit.wikimedia.org/r/661709

Change 661709 merged by jenkins-bot:
[operations/mediawiki-config@master] Migrate PrefUpdate from EventLogging to EventGate on testwiki

https://gerrit.wikimedia.org/r/661709

Change 661746 had a related patch set uploaded (by Mforns; owner: Mforns):
[operations/mediawiki-config@master] Migrate PrefUpdate to EventPlatform on testwiki (2nd trial)

https://gerrit.wikimedia.org/r/661746

Change 661746 merged by Ottomata:
[operations/mediawiki-config@master] Migrate PrefUpdate to EventPlatform on testwiki (2nd trial)

https://gerrit.wikimedia.org/r/661746

Mentioned in SAL (#wikimedia-operations) [2021-02-04T16:46:52Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Migrate PrefUpdate schema to Event Platform on testwiki - T267348 (duration: 01m 08s)

Change 661758 had a related patch set uploaded (by Mforns; owner: Mforns):
[operations/mediawiki-config@master] Migrate PrefUpdate schema to EventGate on all wikis

https://gerrit.wikimedia.org/r/661758

Change 661758 merged by Ottomata:
[operations/mediawiki-config@master] Migrate PrefUpdate schema to EventGate on all wikis

https://gerrit.wikimedia.org/r/661758

Mentioned in SAL (#wikimedia-operations) [2021-02-04T17:51:46Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Migrate PrefUpdate schema to Event Platform on all wikis - T267348 (duration: 01m 01s)

Change 661766 had a related patch set uploaded (by Mforns; owner: Mforns):
[operations/mediawiki-config@master] Rollback migration of PrefUpdate to EventGate on all wikis

https://gerrit.wikimedia.org/r/661766

Change 661766 merged by Ottomata:
[operations/mediawiki-config@master] Rollback migration of PrefUpdate to EventGate on all wikis

https://gerrit.wikimedia.org/r/661766

Mentioned in SAL (#wikimedia-operations) [2021-02-04T18:05:01Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: revert - Migrate PrefUpdate schema to Event Platform on all wikis - leave on testwiki only, seeing validation errors. T267348 (duration: 01m 01s)

Oh my, this schema is produced from the Eventlogging PHP client, which does not yet work. We deployed and rolled back today, but there was a period of about 15 minutes where we dropped events because they were invalid.

This was my fault for telling Marcel to proceed with this migration, I had assumed that these were JS emitted events.

I was able to backfill these events into eventlogging_PrefUpdate from the eventgate-analytics-external.error.validation stream.

@mforns i'm stealing this one from you. we can do it now that PHP based events are working.

Change 668769 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Migrate PrefUpdate to EventGate on all wikis

https://gerrit.wikimedia.org/r/668769

Will migrate this to all wikis on Monday.

Change 668769 merged by Ottomata:
[operations/mediawiki-config@master] Migrate PrefUpdate to EventGate on all wikis

https://gerrit.wikimedia.org/r/668769

Mentioned in SAL (#wikimedia-operations) [2021-03-08T15:07:18Z] <otto@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Migrate PrefUpdate to EventGate on all wikis - T267348 (duration: 00m 59s)

Change 669867 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Finalize PrefUpdate schema ingestion migration to Event Platform

https://gerrit.wikimedia.org/r/669867

Change 669867 merged by Ottomata:
[operations/puppet@production] Finalize PrefUpdate schema ingestion migration to Event Platform

https://gerrit.wikimedia.org/r/669867

Change 669868 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[mediawiki/extensions/WikimediaEvents@master] Finalize migration of PrefUpdate to Event Platform

https://gerrit.wikimedia.org/r/669868

Change 669868 merged by Ottomata:
[mediawiki/extensions/WikimediaEvents@master] Finalize migration of PrefUpdate to Event Platform

https://gerrit.wikimedia.org/r/669868

Change 673075 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Remove schema overrides for 6 finished EL migrations

https://gerrit.wikimedia.org/r/673075

Change 673075 merged by Ottomata:
[operations/mediawiki-config@master] Remove schema overrides for 6 finished EL migrations

https://gerrit.wikimedia.org/r/673075

Mentioned in SAL (#wikimedia-operations) [2021-03-23T18:45:05Z] <otto@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Remove schema overrides for 6 finished EL migrations - T267347 T271164 T267351 T267348 T267343 T267353 (duration: 01m 07s)