Page MenuHomePhabricator

LandingPageImpression Event Platform Migration
Closed, ResolvedPublic

Description

Client IP and consequently geocoded data should be collected for this event data after migration.

  • 1. Pick a schema to migrate
  • 2. Create a new task to track this schema's migration
  • 3. Create /analytics/legacy/landingpageimpression/current.yaml schema
  • 4. Edit-protect the metawiki Schema page at https://meta.wikimedia.org/wiki/Schema:LandingPageImpression
  • 5. Manually evolve the Hive table to use new schema
  • 6. Add entry to wgEventStreams, wgEventLoggingStreamNames and wgEventLoggingSchemas in operations/mediwiki-config
  • 7. Once the legacy stream's data is fully produced through EventGate, switch to using Refine job that uses schema repo instead of meta.wm.org
  • 8. Edit the producer extension.json and set EventLoggingSchemas to the new schema URI
  • 9. Once the producer extension.json is fully deployed, edit wgEventLoggingSchemas in operations/mediawiki-config InitialiseSettings.php and remove the schema's entry.
  • 10. Mark the schema as migrated in the EventLogging Schema Migration Audit spreadsheet

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

@AndyRussG Do you need to continue recording client IP and geolocation data in Hive for these events, or is that already handled at a different layer of the fundraising data pipeline, for example through the explicit country parameter in the event data?

Update: just noticed that you already stated this requirement, thank you!

Change 691095 had a related patch set uploaded (by Awight; author: Awight):

[schemas/event/secondary@master] [WIP] Add LandingPageImpression schema to analytics/legacy

https://gerrit.wikimedia.org/r/691095

odimitrijevic moved this task from Incoming to Event Platform on the Analytics board.
odimitrijevic moved this task from Next Up to In Code Review on the Analytics-Kanban board.

Change 691095 merged by jenkins-bot:

[schemas/event/secondary@master] Add LandingPageImpression schema to analytics/legacy

https://gerrit.wikimedia.org/r/691095

I've copied the checklist from the parent task. I don't have permissions to do step (4), and (5) scares me, so leaving those to Analytics. Once completed, I can create patches for the remaining steps.

Change 698993 had a related patch set uploaded (by Ottomata; author: Ottomata):

[operations/mediawiki-config@master] Migrate LandingPageImpression to Event Platform

https://gerrit.wikimedia.org/r/698993

Change 698993 merged by Ottomata:

[operations/mediawiki-config@master] Migrate LandingPageImpression to Event Platform

https://gerrit.wikimedia.org/r/698993

Change 698998 had a related patch set uploaded (by Ottomata; author: Ottomata):

[operations/mediawiki-config@master] Migrate LandingPageImpression to Event Platform on testwiki

https://gerrit.wikimedia.org/r/698998

Change 698998 merged by Ottomata:

[operations/mediawiki-config@master] Migrate LandingPageImpression to Event Platform on testwiki

https://gerrit.wikimedia.org/r/698998

Mentioned in SAL (#wikimedia-operations) [2021-06-09T14:23:02Z] <otto@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Migrate LandingPageImpression schema to EventPlatform on testwiki - T282855 (duration: 01m 07s)

Change 699000 had a related patch set uploaded (by Ottomata; author: Ottomata):

[operations/mediawiki-config@master] Migrate LandingPageImpression to Event Platform on all wikis

https://gerrit.wikimedia.org/r/699000

Change 699000 merged by Ottomata:

[operations/mediawiki-config@master] Migrate LandingPageImpression to Event Platform on all wikis

https://gerrit.wikimedia.org/r/699000

Mentioned in SAL (#wikimedia-operations) [2021-06-09T14:33:06Z] <otto@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Migrate LandingPageImpression schema to EventPlatform on all wikis - T282855 (duration: 01m 06s)

Change 699002 had a related patch set uploaded (by Ottomata; author: Ottomata):

[operations/puppet@production] Finalize backend EP migration of 4 EL schemas

https://gerrit.wikimedia.org/r/699002

Change 699005 had a related patch set uploaded (by Ottomata; author: Ottomata):

[mediawiki/extensions/FundraiserLandingPage@master] Finalize migration of LandngPageImpression to Event Platform

https://gerrit.wikimedia.org/r/699005

@awight I've already done a bunch of the main migration steps, now we're just up to finalizing.

I'll merge the puppet patch that does step 7 either tomorrow or Monday. After that perhaps you can do the rest?

https://gerrit.wikimedia.org/r/c/mediawiki/extensions/FundraiserLandingPage/+/699005 (step 8) can be merged anytime.

Ottomata updated Other Assignee, added: awight.
Ottomata moved this task from In Code Review to Ready to Deploy on the Analytics-Kanban board.

Change 699002 merged by Ottomata:

[operations/puppet@production] Finalize backend EP migration of 4 EL schemas

https://gerrit.wikimedia.org/r/699002

Change 699005 merged by Ottomata:

[mediawiki/extensions/FundraiserLandingPage@master] Finalize migration of LandngPageImpression to Event Platform

https://gerrit.wikimedia.org/r/699005

Change 706689 had a related patch set uploaded (by Ottomata; author: Ottomata):

[operations/mediawiki-config@master] Finalize several EventLogging -> Event Platfom migrations

https://gerrit.wikimedia.org/r/706689

Change 706689 merged by jenkins-bot:

[operations/mediawiki-config@master] Finalize several EventLogging -> Event Platfom migrations

https://gerrit.wikimedia.org/r/706689

Mentioned in SAL (#wikimedia-operations) [2021-07-22T19:26:12Z] <otto@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Finalize several EventLogging -> Event Platfom migrations - T282855 T238138 T282562 T271168 (duration: 00m 55s)