Page MenuHomePhabricator
Create Task
Maniphest T282855

LandingPageImpression Event Platform Migration
Closed, ResolvedPublic
Actions

Description

Client IP and consequently geocoded data should be collected for this event data after migration.

  • 1. Pick a schema to migrate
  • 2. Create a new task to track this schema's migration
  • 3. Create /analytics/legacy/landingpageimpression/current.yaml schema
  • 4. Edit-protect the metawiki Schema page at https://meta.wikimedia.org/wiki/Schema:LandingPageImpression
  • 5. Manually evolve the Hive table to use new schema
  • 6. Add entry to wgEventStreams, wgEventLoggingStreamNames and wgEventLoggingSchemas in operations/mediwiki-config
  • 7. Once the legacy stream's data is fully produced through EventGate, switch to using Refine job that uses schema repo instead of meta.wm.org
  • 8. Edit the producer extension.json and set EventLoggingSchemas to the new schema URI
  • 9. Once the producer extension.json is fully deployed, edit wgEventLoggingSchemas in operations/mediawiki-config InitialiseSettings.php and remove the schema's entry.
  • 10. Mark the schema as migrated in the EventLogging Schema Migration Audit spreadsheet

Details

Other Assignee
awight
SubjectRepoBranchLines +/-
Finalize several EventLogging -> Event Platfom migrationsoperations/mediawiki-configmaster+0 -7
Finalize migration of LandngPageImpression to Event Platformmediawiki/extensions/FundraiserLandingPagemaster+1 -1
Finalize backend EP migration of 4 EL schemasoperations/puppetproduction+12 -0
Migrate LandingPageImpression to Event Platform on all wikisoperations/mediawiki-configmaster+1 -1
Migrate LandingPageImpression to Event Platform on testwikioperations/mediawiki-configmaster+1 -1
Migrate LandingPageImpression to Event Platformoperations/mediawiki-configmaster+9 -0
Add LandingPageImpression schema to analytics/legacyschemas/event/secondarymaster+505 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

awight created this task.May 14 2021, 9:18 AM
Restricted Application added a project: Analytics. · View Herald TranscriptMay 14 2021, 9:18 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
awight added a comment.EditedMay 14 2021, 9:24 AM

@AndyRussG Do you need to continue recording client IP and geolocation data in Hive for these events, or is that already handled at a different layer of the fundraising data pipeline, for example through the explicit country parameter in the event data?

Update: just noticed that you already stated this requirement, thank you!

gerritbot added a comment.May 14 2021, 9:32 AM

Change 691095 had a related patch set uploaded (by Awight; author: Awight):

[schemas/event/secondary@master] [WIP] Add LandingPageImpression schema to analytics/legacy

https://gerrit.wikimedia.org/r/691095

gerritbot added a project: Patch-For-Review.May 14 2021, 9:32 AM
awight updated the task description. (Show Details)May 14 2021, 12:26 PM
Ottomata added a comment.May 14 2021, 4:25 PM

@awight you are a true hero :)

Ottomata awarded a token.May 14 2021, 4:25 PM
odimitrijevic triaged this task as High priority.May 17 2021, 3:42 PM
odimitrijevic moved this task from Incoming to Event Platform on the Analytics board.
odimitrijevic added a project: Analytics-Kanban.
odimitrijevic moved this task from Next Up to In Code Review on the Analytics-Kanban board.
Ottomata mentioned this in T259163: Migrate legacy metawiki schemas to Event Platform.Jun 9 2021, 1:31 PM
gerritbot added a comment.Jun 9 2021, 1:39 PM

Change 691095 merged by jenkins-bot:

[schemas/event/secondary@master] Add LandingPageImpression schema to analytics/legacy

https://gerrit.wikimedia.org/r/691095

awight updated the task description. (Show Details)EditedJun 9 2021, 1:43 PM

I've copied the checklist from the parent task. I don't have permissions to do step (4), and (5) scares me, so leaving those to Analytics. Once completed, I can create patches for the remaining steps.

gerritbot added a comment.Jun 9 2021, 1:51 PM

Change 698993 had a related patch set uploaded (by Ottomata; author: Ottomata):

[operations/mediawiki-config@master] Migrate LandingPageImpression to Event Platform

https://gerrit.wikimedia.org/r/698993

gerritbot added a comment.Jun 9 2021, 2:01 PM

Change 698993 merged by Ottomata:

[operations/mediawiki-config@master] Migrate LandingPageImpression to Event Platform

https://gerrit.wikimedia.org/r/698993

Maintenance_bot removed a project: Patch-For-Review.Jun 9 2021, 2:10 PM
gerritbot added a comment.Jun 9 2021, 2:19 PM

Change 698998 had a related patch set uploaded (by Ottomata; author: Ottomata):

[operations/mediawiki-config@master] Migrate LandingPageImpression to Event Platform on testwiki

https://gerrit.wikimedia.org/r/698998

gerritbot added a project: Patch-For-Review.Jun 9 2021, 2:19 PM
gerritbot added a comment.Jun 9 2021, 2:21 PM

Change 698998 merged by Ottomata:

[operations/mediawiki-config@master] Migrate LandingPageImpression to Event Platform on testwiki

https://gerrit.wikimedia.org/r/698998

Stashbot added a comment.Jun 9 2021, 2:23 PM

Mentioned in SAL (#wikimedia-operations) [2021-06-09T14:23:02Z] <otto@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Migrate LandingPageImpression schema to EventPlatform on testwiki - T282855 (duration: 01m 07s)

gerritbot added a comment.Jun 9 2021, 2:25 PM

Change 699000 had a related patch set uploaded (by Ottomata; author: Ottomata):

[operations/mediawiki-config@master] Migrate LandingPageImpression to Event Platform on all wikis

https://gerrit.wikimedia.org/r/699000

gerritbot added a comment.Jun 9 2021, 2:27 PM

Change 699000 merged by Ottomata:

[operations/mediawiki-config@master] Migrate LandingPageImpression to Event Platform on all wikis

https://gerrit.wikimedia.org/r/699000

Stashbot added a comment.Jun 9 2021, 2:33 PM

Mentioned in SAL (#wikimedia-operations) [2021-06-09T14:33:06Z] <otto@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Migrate LandingPageImpression schema to EventPlatform on all wikis - T282855 (duration: 01m 06s)

gerritbot added a comment.Jun 9 2021, 2:42 PM

Change 699002 had a related patch set uploaded (by Ottomata; author: Ottomata):

[operations/puppet@production] Finalize backend EP migration of 4 EL schemas

https://gerrit.wikimedia.org/r/699002

gerritbot added a comment.Jun 9 2021, 2:44 PM

Change 699005 had a related patch set uploaded (by Ottomata; author: Ottomata):

[mediawiki/extensions/FundraiserLandingPage@master] Finalize migration of LandngPageImpression to Event Platform

https://gerrit.wikimedia.org/r/699005

Ottomata updated the task description. (Show Details)Jun 9 2021, 2:44 PM

@awight I've already done a bunch of the main migration steps, now we're just up to finalizing.

I'll merge the puppet patch that does step 7 either tomorrow or Monday. After that perhaps you can do the rest?

https://gerrit.wikimedia.org/r/c/mediawiki/extensions/FundraiserLandingPage/+/699005 (step 8) can be merged anytime.

Ottomata claimed this task.Jun 9 2021, 2:52 PM
Ottomata updated Other Assignee, added: awight.
Ottomata moved this task from In Code Review to Ready to Deploy on the Analytics-Kanban board.
gerritbot added a comment.Jun 14 2021, 1:24 PM

Change 699002 merged by Ottomata:

[operations/puppet@production] Finalize backend EP migration of 4 EL schemas

https://gerrit.wikimedia.org/r/699002

Ottomata updated the task description. (Show Details)Jun 14 2021, 1:38 PM
gerritbot added a comment.Jun 14 2021, 1:40 PM

Change 699005 merged by Ottomata:

[mediawiki/extensions/FundraiserLandingPage@master] Finalize migration of LandngPageImpression to Event Platform

https://gerrit.wikimedia.org/r/699005

Ottomata updated the task description. (Show Details)Jun 14 2021, 1:40 PM
ReleaseTaggerBot added a project: MW-1.37-notes (1.37.0-wmf.11; 2021-06-21).Jun 14 2021, 2:00 PM
Ottomata moved this task from Backlog to In Progress Before Value Streams Kickoff (August 15th) on the Event-Platform board.Jun 14 2021, 8:54 PM
gerritbot added a comment.Jul 22 2021, 7:14 PM

Change 706689 had a related patch set uploaded (by Ottomata; author: Ottomata):

[operations/mediawiki-config@master] Finalize several EventLogging -> Event Platfom migrations

https://gerrit.wikimedia.org/r/706689

gerritbot added a comment.Jul 22 2021, 7:24 PM

Change 706689 merged by jenkins-bot:

[operations/mediawiki-config@master] Finalize several EventLogging -> Event Platfom migrations

https://gerrit.wikimedia.org/r/706689

Stashbot added a comment.Jul 22 2021, 7:26 PM

Mentioned in SAL (#wikimedia-operations) [2021-07-22T19:26:12Z] <otto@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Finalize several EventLogging -> Event Platfom migrations - T282855 T238138 T282562 T271168 (duration: 00m 55s)

Stashbot mentioned this in T282562: WMDEBanner* Event Platform Migration.Jul 22 2021, 7:26 PM
Stashbot mentioned this in T271168: CentralNoticeBannerHistory and CentralNoticeImpression Event Platform Migration.
Stashbot mentioned this in T238138: VirtualPageView Event Platform Migration.
Ottomata updated the task description. (Show Details)Jul 22 2021, 8:03 PM
Ottomata moved this task from Ready to Deploy to Done on the Analytics-Kanban board.
Ottomata closed this task as Resolved.Jul 27 2021, 7:21 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL