Page MenuHomePhabricator

Modern Event Platform: Stream Intake Service: Migrate change-prop events to new (EventGate) style schemas
Closed, ResolvedPublic

Description

change-prop produces to Kafka directly instead of using EventGate. This is fine, but the events it produces still need to be migrated to the newer EventGate style schema format: draft-07, with $schema and meta.stream fields.

Event Timeline

fdans triaged this task as Medium priority.Jul 1 2019, 3:44 PM
fdans moved this task from Incoming to Event Platform on the Analytics board.

Change 523792 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Point changeprop events at eventgate-main in beta

https://gerrit.wikimedia.org/r/523792

Change 524057 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[operations/puppet@production] Switch RESTBase evvnt production to eventgate. Step 1.

https://gerrit.wikimedia.org/r/524057

Change 524058 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[mediawiki/services/restbase/deploy@master] Switch event production to eventgate.

https://gerrit.wikimedia.org/r/524058

Change 524060 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[operations/deployment-charts@master] Switch event service to eventgate.

https://gerrit.wikimedia.org/r/524060

Ok, I've seem to have been using a wrong ticket number in my gerrit patches...

After our meeting today we decided to put the eventlogging-service-eventbus migration tasks in T211248 (I was getting confused as to what was actually using it).

Change 523792 merged by Ottomata:
[operations/puppet@production] Add change-prop event_service_uri and point at eventgate-main

https://gerrit.wikimedia.org/r/523792

Change 524832 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[operations/puppet@production] Remove changeprop::eventbus_uri variable.

https://gerrit.wikimedia.org/r/524832

Change 524832 merged by Ottomata:
[operations/puppet@production] Remove changeprop::eventbus_uri variable.

https://gerrit.wikimedia.org/r/524832

Mentioned in SAL (#wikimedia-operations) [2019-07-23T16:58:46Z] <ppchelko@deploy1001> Started deploy [changeprop/deploy@894f735]: Switch internal events to the new schema T226522

Mentioned in SAL (#wikimedia-operations) [2019-07-23T17:00:17Z] <ppchelko@deploy1001> Finished deploy [changeprop/deploy@894f735]: Switch internal events to the new schema T226522 (duration: 01m 30s)

Change 525130 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[mediawiki/services/change-propagation/deploy@master] Remove old-style resource_change event support.

https://gerrit.wikimedia.org/r/525130

Change 525130 merged by Ppchelko:
[mediawiki/services/change-propagation/deploy@master] Remove old-style resource_change event support.

https://gerrit.wikimedia.org/r/525130

Mentioned in SAL (#wikimedia-operations) [2019-07-23T17:36:35Z] <ppchelko@deploy1001> Started deploy [changeprop/deploy@6c5c0a3]: Switch internal events to the new schema T226522, step 2

Mentioned in SAL (#wikimedia-operations) [2019-07-23T17:38:11Z] <ppchelko@deploy1001> Finished deploy [changeprop/deploy@6c5c0a3]: Switch internal events to the new schema T226522, step 2 (duration: 01m 37s)

All internal change-prop events are following the new schema now. Resolving.