Page MenuHomePhabricator

Modern Event Platform: Stream Intake Service: Migrate eventlogging-service-eventbus events to eventgate-main
Open, NormalPublic

Description

This includes making the minor schema changes needed, i.e. meta.schema_uri -> $schema and meta.topic -> meta.stream.

Event streams to migrate in order of lowest to highest risk:

  • user-blocks-change
  • revision-tags-change
  • page-properties-change
  • page-links-change
  • page-restrictions-change
  • campaign-{create,change,delete}
  • page-move
  • page-delete/undelete
  • page-create
  • recentchange - public
  • revision-visibility-change - security implications
  • revision-create - this is basically the starter event that is beginning most of the update chains for change-prop
  • resource-change (Mediawiki EventBus)
  • resource-change (RESTBase)
  • revision-score (change-prop)

Before we can decom eventlogging-service-eventbus, we also need to complete:

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Mentioned in SAL (#wikimedia-operations) [2019-06-25T15:13:37Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Produce page-properties-change stream to eventgate-main - T211248 (duration: 00m 58s)

Change 520019 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Produce centralnotice.campaign-* streams to eventgate-main

https://gerrit.wikimedia.org/r/520019

Change 521293 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Migrate page-* events to eventgate-main

https://gerrit.wikimedia.org/r/521293

daniel removed a subscriber: daniel.Jul 8 2019, 3:18 PM

Change 521293 merged by jenkins-bot:
[operations/mediawiki-config@master] Migrate page-* events to eventgate-main

https://gerrit.wikimedia.org/r/521293

Mentioned in SAL (#wikimedia-operations) [2019-07-08T18:43:50Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Produce page-* streams to eventgate-main - T211248 (duration: 00m 50s)

Ottomata updated the task description. (Show Details)Jul 8 2019, 6:46 PM

Change 521328 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Refine mediawiki_page* with schema aware Refine job

https://gerrit.wikimedia.org/r/521328

Change 521328 merged by Ottomata:
[operations/puppet@production] Refine mediawiki_page* with schema aware Refine job

https://gerrit.wikimedia.org/r/521328

Change 521517 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Migrate mediawiki.recentchange stream to eventgate-main

https://gerrit.wikimedia.org/r/521517

Change 521528 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Move wgRCFeeds settings from CommonSettings to InitialiseSettings

https://gerrit.wikimedia.org/r/521528

Change 521528 merged by jenkins-bot:
[operations/mediawiki-config@master] Use wgEventServiceStreamConfig to configure wgRCFeeds['eventbus']

https://gerrit.wikimedia.org/r/521528

Mentioned in SAL (#wikimedia-operations) [2019-07-10T19:26:28Z] <otto@deploy1001> Synchronized wmf-config/CommonSettings.php: Use wgEventServiceStreamConfig to configure wgRCFeeds eventbus. No-op in prod. - T211248 (duration: 00m 58s)

Change 521517 merged by jenkins-bot:
[operations/mediawiki-config@master] Migrate mediawiki.recentchange stream to eventgate-main

https://gerrit.wikimedia.org/r/521517

Krinkle removed a subscriber: Krinkle.Jul 10 2019, 7:29 PM

Mentioned in SAL (#wikimedia-operations) [2019-07-10T19:32:22Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Produce recentchange stream to eventgate-main - T211248 (duration: 00m 57s)

Ottomata updated the task description. (Show Details)Jul 10 2019, 7:34 PM

@DStrine, I need some help from someone who knows how to make changes to centralnotice campaigns to test that these events work after migration. Who should I ask?

@DStrine, I need some help from someone who knows how to make changes to centralnotice campaigns to test that these events work after migration. Who should I ask?

Hi! I can help with this, sorry for the delay in replying via other channels...! Thanks!!

Change 523180 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Produce revision-visibility-change event to eventgate-main

https://gerrit.wikimedia.org/r/523180

Change 523180 merged by jenkins-bot:
[operations/mediawiki-config@master] Produce revision-visibility-change event to eventgate-main

https://gerrit.wikimedia.org/r/523180

Mentioned in SAL (#wikimedia-operations) [2019-07-15T16:57:13Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Produce revision-visibility-change stream to eventgate-main - T211248 (duration: 00m 49s)

Change 523232 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Produce revision-create stream to eventgate-main

https://gerrit.wikimedia.org/r/523232

Ottomata claimed this task.Jul 16 2019, 4:20 PM

Change 520019 merged by Ottomata:
[operations/mediawiki-config@master] Produce centralnotice.campaign-* streams to eventgate-main

https://gerrit.wikimedia.org/r/520019

Mentioned in SAL (#wikimedia-operations) [2019-07-16T17:39:40Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Produce centralnotice.campaign-* streams to eventgate-main - T211248 (duration: 00m 55s)

Ottomata updated the task description. (Show Details)Jul 16 2019, 5:47 PM

Change 523787 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Move revision_visibility_change Refine job to schema aware version

https://gerrit.wikimedia.org/r/523787

Ottomata updated the task description. (Show Details)Jul 16 2019, 5:49 PM

Change 523787 merged by Ottomata:
[operations/puppet@production] Move revision_visibility_change Refine job to schema aware version

https://gerrit.wikimedia.org/r/523787

Change 523232 merged by Ottomata:
[operations/mediawiki-config@master] Produce revision-create stream to eventgate-main

https://gerrit.wikimedia.org/r/523232

Mentioned in SAL (#wikimedia-operations) [2019-07-16T18:22:59Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Produce revision-create stream to eventgate-main - T211248 (duration: 00m 54s)

Mentioned in SAL (#wikimedia-operations) [2019-07-16T18:46:59Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: retry - Produce revision-create stream to eventgate-main - T211248 (duration: 00m 54s)

Ottomata updated the task description. (Show Details)Jul 16 2019, 6:53 PM

Change 523791 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Refine mediawiki_revision_create events using schema aware Refine job

https://gerrit.wikimedia.org/r/523791

Change 523794 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[mediawiki/extensions/EventBus@master] Use proper schema url for resource_change event

https://gerrit.wikimedia.org/r/523794

Change 523794 merged by Ottomata:
[mediawiki/extensions/EventBus@master] Use proper schema url for resource_change event

https://gerrit.wikimedia.org/r/523794

Ottomata renamed this task from Modern Event Platform: Stream Intake Service: Migrate Mediawiki Eventbus events to eventgate-main to Modern Event Platform: Stream Intake Service: Migrate eventlogging-service-eventbus events to eventgate-main.Jul 17 2019, 8:28 PM
Ottomata updated the task description. (Show Details)

Change 523791 merged by Ottomata:
[operations/puppet@production] Refine mediawiki_revision_create events using schema aware Refine job

https://gerrit.wikimedia.org/r/523791

Change 524055 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[mediawiki/services/change-propagation/deploy@master] Switch revision-create to eventgate.

https://gerrit.wikimedia.org/r/524055

Change 524055 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[mediawiki/services/change-propagation/deploy@master] Switch revision-score to eventgate.

https://gerrit.wikimedia.org/r/524055

Change 524055 merged by Ppchelko:
[mediawiki/services/change-propagation/deploy@master] Switch revision-score to eventgate.

https://gerrit.wikimedia.org/r/524055

Mentioned in SAL (#wikimedia-operations) [2019-07-22T19:07:30Z] <ppchelko@deploy1001> Started deploy [changeprop/deploy@3f8aad2]: Switch revision-score to eventgate T211248

Mentioned in SAL (#wikimedia-operations) [2019-07-22T19:09:01Z] <ppchelko@deploy1001> Finished deploy [changeprop/deploy@3f8aad2]: Switch revision-score to eventgate T211248 (duration: 01m 31s)

Pchelolo updated the task description. (Show Details)Mon, Jul 22, 7:27 PM

Mentioned in SAL (#wikimedia-operations) [2019-07-22T21:04:57Z] <ppchelko@deploy1001> Started deploy [restbase/deploy@ea10fa5]: Switch event production to eventgate T211248

Change 524922 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/deployment-charts@master] eventgate-main configure change-prop.transcludes.resource-change stream

https://gerrit.wikimedia.org/r/524922

Change 524922 merged by Ottomata:
[operations/deployment-charts@master] eventgate-main configure change-prop.transcludes.resource-change stream

https://gerrit.wikimedia.org/r/524922

Mentioned in SAL (#wikimedia-operations) [2019-07-22T21:21:11Z] <ppchelko@deploy1001> Finished deploy [restbase/deploy@ea10fa5]: Switch event production to eventgate T211248 (duration: 16m 14s)

Mentioned in SAL (#wikimedia-operations) [2019-07-22T21:25:40Z] <ppchelko@deploy1001> Started deploy [restbase/deploy@9a99b17]: Rollback: Switch event production to eventgate T211248

Mentioned in SAL (#wikimedia-operations) [2019-07-22T21:38:39Z] <ppchelko@deploy1001> Finished deploy [restbase/deploy@9a99b17]: Rollback: Switch event production to eventgate T211248 (duration: 13m 01s)

Mentioned in SAL (#wikimedia-operations) [2019-07-23T14:39:34Z] <ppchelko@deploy1001> Started deploy [restbase/deploy@ea10fa5]: Switch event production to eventgate T211248, attempt 2

Mentioned in SAL (#wikimedia-operations) [2019-07-23T14:52:42Z] <ppchelko@deploy1001> Finished deploy [restbase/deploy@ea10fa5]: Switch event production to eventgate T211248, attempt 2 (duration: 13m 08s)

Change 525110 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/deployment-charts@master] Use proper main-codfw Kafka cluster for eventgate-main in codfw

https://gerrit.wikimedia.org/r/525110

Change 525110 merged by Ottomata:
[operations/deployment-charts@master] Use proper main-codfw Kafka cluster for eventgate-main in codfw

https://gerrit.wikimedia.org/r/525110

Pchelolo updated the task description. (Show Details)Tue, Jul 23, 3:19 PM

Change 525114 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[operations/puppet@production] Clean up eventlogging_service_uri from RESTBase profile.

https://gerrit.wikimedia.org/r/525114

Change 525121 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[operations/puppet@production] Clean up eventlogging_service_uri from maps.

https://gerrit.wikimedia.org/r/525121

Change 525121 abandoned by Ppchelko:
Clean up eventlogging_service_uri from maps.

Reason:
Superseded by I06a90a199c5832a7fe2505bffa5c5cf40281ff98

https://gerrit.wikimedia.org/r/525121

Change 525121 restored by Ppchelko:
Clean up eventlogging_service_uri from maps.

https://gerrit.wikimedia.org/r/525121

Change 525114 merged by Ottomata:
[operations/puppet@production] Clean up eventlogging_service_uri from RESTBase profile.

https://gerrit.wikimedia.org/r/525114

Change 525121 merged by Ottomata:
[operations/puppet@production] Clean up eventlogging_service_uri from maps.

https://gerrit.wikimedia.org/r/525121

Change 525854 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[operations/mediawiki-config@master] [EventBus] Switch resource_change event to eventgate.

https://gerrit.wikimedia.org/r/525854

Change 525854 merged by Ottomata:
[operations/mediawiki-config@master] [EventBus] Switch resource_change event to eventgate.

https://gerrit.wikimedia.org/r/525854

Mentioned in SAL (#wikimedia-operations) [2019-07-29T15:30:46Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Produce resource_change stream to eventgate-main - T211248 (duration: 00m 47s)

Mentioned in SAL (#wikimedia-operations) [2019-07-29T15:35:26Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Retry - Produce resource_change stream to eventgate-main - T211248 (duration: 00m 46s)

Pchelolo updated the task description. (Show Details)Mon, Jul 29, 3:50 PM

Change 526180 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Use schema aware refine for revision score and resource change

https://gerrit.wikimedia.org/r/526180

Change 526180 merged by Ottomata:
[operations/puppet@production] Use schema aware refine for revision score and resource change

https://gerrit.wikimedia.org/r/526180

Change 526435 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Remove unsed refine job refine_eventlogging_eventbus

https://gerrit.wikimedia.org/r/526435

Change 526435 merged by Ottomata:
[operations/puppet@production] Remove unsed refine job refine_eventlogging_eventbus

https://gerrit.wikimedia.org/r/526435

Ottomata updated the task description. (Show Details)Tue, Jul 30, 3:08 PM