This project was created before any of the utilities function were available, we should adapt the WDQS Streaming Updater flink to use these so that it better integrates with the event-platform.
AC:
| Title | Reference | Author | Source Branch | Dest Branch | |
|---|---|---|---|---|---|
| Bump to 0.3.154 | repos/search-platform/flink-rdf-streaming-updater!26 | dcausse | bump-to-0_3_154 | main | |
| Bump to 0.3.154-82c97965 (snapshot) | repos/search-platform/flink-rdf-streaming-updater!25 | dcausse | bump-to-0_3_154-82c97965 | main | |
| Bump to 0.3.153 | repos/search-platform/flink-rdf-streaming-updater!24 | dcausse | bump-to-0_3_153 | main | |
| Bump to 0.3.152 | repos/search-platform/flink-rdf-streaming-updater!23 | dcausse | bump_to_0_3_150 | main | |
| Bump to 0.3.150 | repos/search-platform/flink-rdf-streaming-updater!22 | dcausse | bump-0.3.150 | main |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Duplicate | Feature | None | T330521 Make WDQS update stream public | ||
| Open | None | T294133 [EPIC] Expose rdf-streaming-updater.mutation content through EventStreams | |||
| Resolved | dcausse | T374919 Adapt the rdf-streaming-updater flink job to use wikimedia-eventutilities-flink |
Change #1078991 had a related patch set uploaded (by DCausse; author: DCausse):
[wikidata/query/rdf@master] Add support for EventDataStreamFactory for input streams
Change #1079538 had a related patch set uploaded (by DCausse; author: DCausse):
[wikidata/query/rdf@master] Add support for setting a start timestamp
Change #1081221 had a related patch set uploaded (by DCausse; author: DCausse):
[wikidata/query/rdf@master] Add support for event platform in output streams
Change #1078991 merged by jenkins-bot:
[wikidata/query/rdf@master] Add support for EventDataStreamFactory for input streams
Change #1079538 merged by jenkins-bot:
[wikidata/query/rdf@master] Add support for setting a start timestamp
Change #1081221 merged by jenkins-bot:
[wikidata/query/rdf@master] Add support for EventDataStreamFactory for output streams
Change #1091290 had a related patch set uploaded (by DCausse; author: DCausse):
[wikidata/query/deploy@master] deploy version 0.3.150
dcausse opened https://gitlab.wikimedia.org/repos/search-platform/flink-rdf-streaming-updater/-/merge_requests/22
Bump to 0.3.150
dcausse merged https://gitlab.wikimedia.org/repos/search-platform/flink-rdf-streaming-updater/-/merge_requests/22
Bump to 0.3.150
Change #1091306 had a related patch set uploaded (by DCausse; author: DCausse):
[operations/deployment-charts@master] rdf-streaming-updater: bump to 0.3.150
Change #1092191 had a related patch set uploaded (by DCausse; author: DCausse):
[operations/deployment-charts@master] rdf-streaming-updater: produce rdf_change v2 events
Change #1091290 merged by Bking:
[wikidata/query/deploy@master] deploy version 0.3.150
Change #1092191 merged by jenkins-bot:
[operations/deployment-charts@master] rdf-streaming-updater: produce rdf_change v2 events
Change #1099656 had a related patch set uploaded (by DCausse; author: DCausse):
[wikidata/query/rdf@master] Add canary event filtering for the streaming-updater-consumer
Change #1099727 had a related patch set uploaded (by DCausse; author: DCausse):
[operations/mediawiki-config@master] rdf-streaming-updater: add wdqs udpater streams in event stream config
Change #1102319 had a related patch set uploaded (by DCausse; author: DCausse):
[wikidata/query/rdf@master] Add support for versioned streams
Change #1099727 merged by jenkins-bot:
[operations/mediawiki-config@master] rdf-streaming-updater: add wdqs udpater streams in event stream config
Mentioned in SAL (#wikimedia-operations) [2024-12-17T14:05:33Z] <dcausse@deploy2002> Started scap sync-world: Backport for [[gerrit:1099727|rdf-streaming-updater: add wdqs udpater streams in event stream config (T374919)]], [[gerrit:1104598|cirrussearch: increase shard count for cebwiki_content (T379002)]]
Mentioned in SAL (#wikimedia-operations) [2024-12-17T14:14:55Z] <dcausse@deploy2002> dcausse: Backport for [[gerrit:1099727|rdf-streaming-updater: add wdqs udpater streams in event stream config (T374919)]], [[gerrit:1104598|cirrussearch: increase shard count for cebwiki_content (T379002)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)
Mentioned in SAL (#wikimedia-operations) [2024-12-17T14:25:41Z] <dcausse@deploy2002> Finished scap sync-world: Backport for [[gerrit:1099727|rdf-streaming-updater: add wdqs udpater streams in event stream config (T374919)]], [[gerrit:1104598|cirrussearch: increase shard count for cebwiki_content (T379002)]] (duration: 20m 07s)
Change #1109450 had a related patch set uploaded (by DCausse; author: DCausse):
[wikidata/query/rdf@master] Rename outputMutationSchema to outputMutationSchemaVersion
Change #1109450 merged by jenkins-bot:
[wikidata/query/rdf@master] Rename outputMutationSchema to outputMutationSchemaVersion
Change #1102319 merged by jenkins-bot:
[wikidata/query/rdf@master] Add support for versioned streams
Change #1099656 merged by jenkins-bot:
[wikidata/query/rdf@master] Add canary event filtering for the streaming-updater-consumer
Change #1112251 had a related patch set uploaded (by DCausse; author: DCausse):
[wikidata/query/deploy@master] deploy version 0.3.152
dcausse opened https://gitlab.wikimedia.org/repos/search-platform/flink-rdf-streaming-updater/-/merge_requests/23
Bump to 0.3.152
dcausse merged https://gitlab.wikimedia.org/repos/search-platform/flink-rdf-streaming-updater/-/merge_requests/23
Bump to 0.3.152
Change #1112762 had a related patch set uploaded (by DCausse; author: DCausse):
[operations/deployment-charts@master] wdqs: bump image version to 0.3.152
Change #1112763 had a related patch set uploaded (by DCausse; author: DCausse):
[operations/deployment-charts@master] wdqs: enable new event stream api config in staging
Change #1112762 merged by jenkins-bot:
[operations/deployment-charts@master] wdqs: bump image version to 0.3.152
Change #1112763 merged by jenkins-bot:
[operations/deployment-charts@master] wdqs: enable new event stream api config in staging
Change #1113080 had a related patch set uploaded (by DCausse; author: DCausse):
[operations/deployment-charts@master] wdqs: add missing page_change_content_models config entry
Change #1113080 merged by jenkins-bot:
[operations/deployment-charts@master] wdqs: add missing page_change_content_models config entry
Change #1113082 had a related patch set uploaded (by DCausse; author: DCausse):
[operations/deployment-charts@master] wdqs: add missing config entry main_output_stream
Change #1113082 merged by jenkins-bot:
[operations/deployment-charts@master] wdqs: add missing config entry main_output_stream
Change #1113150 had a related patch set uploaded (by DCausse; author: DCausse):
[operations/deployment-charts@master] wdqs: fix staging stream names
Change #1113150 merged by jenkins-bot:
[operations/deployment-charts@master] wdqs: fix staging stream names
Change #1113165 had a related patch set uploaded (by DCausse; author: DCausse):
[wikidata/query/rdf@master] streaming-updater-producer: fix versionned streams support
Change #1113165 merged by jenkins-bot:
[wikidata/query/rdf@master] streaming-updater-producer: fix versionned streams support
dcausse opened https://gitlab.wikimedia.org/repos/search-platform/flink-rdf-streaming-updater/-/merge_requests/24
Bump to 0.3.153
dcausse merged https://gitlab.wikimedia.org/repos/search-platform/flink-rdf-streaming-updater/-/merge_requests/24
Bump to 0.3.153
Change #1113184 had a related patch set uploaded (by DCausse; author: DCausse):
[operations/deployment-charts@master] wdqs: bump image to 0.3.153
Change #1113184 merged by jenkins-bot:
[operations/deployment-charts@master] wdqs: bump image to 0.3.153
dcausse opened https://gitlab.wikimedia.org/repos/search-platform/flink-rdf-streaming-updater/-/merge_requests/25
Bump to 0.3.154-82c97965 (snapshot)
dcausse merged https://gitlab.wikimedia.org/repos/search-platform/flink-rdf-streaming-updater/-/merge_requests/25
Bump to 0.3.154-82c97965 (snapshot)
Change #1113204 had a related patch set uploaded (by DCausse; author: DCausse):
[wikidata/query/rdf@master] streaming-updater-producer: use long for sequence fields
Change #1113204 merged by jenkins-bot:
[wikidata/query/rdf@master] streaming-updater-producer: use long for sequence fields
dcausse opened https://gitlab.wikimedia.org/repos/search-platform/flink-rdf-streaming-updater/-/merge_requests/26
Bump to 0.3.154
dcausse merged https://gitlab.wikimedia.org/repos/search-platform/flink-rdf-streaming-updater/-/merge_requests/26
Bump to 0.3.154
Change #1113466 had a related patch set uploaded (by DCausse; author: DCausse):
[operations/deployment-charts@master] wdqs: bump to 0.3.154 and enable event utilities APIs
Change #1113743 had a related patch set uploaded (by DCausse; author: DCausse):
[operations/deployment-charts@master] wdqs: bump to 0.3.154 and enable event utilities APIs (2/3)
Change #1113744 had a related patch set uploaded (by DCausse; author: DCausse):
[operations/deployment-charts@master] wdqs: bump to 0.3.154 and enable event utilities APIs (Step 3/3)
Change #1113745 had a related patch set uploaded (by DCausse; author: DCausse):
[operations/deployment-charts@master] wdqs: cleanup unused settings
Change #1113466 merged by jenkins-bot:
[operations/deployment-charts@master] wdqs: bump to 0.3.154 and enable event utilities APIs (1/3)
Change #1113743 merged by jenkins-bot:
[operations/deployment-charts@master] wdqs: bump to 0.3.154 and enable event utilities APIs (2/3)
Change #1113744 merged by jenkins-bot:
[operations/deployment-charts@master] wdqs: bump to 0.3.154 and enable event utilities APIs (3/3)
Change #1113745 merged by jenkins-bot:
[operations/deployment-charts@master] wdqs: cleanup unused settings
Change #1112251 merged by Ryan Kemper:
[wikidata/query/deploy@master] deploy version 0.3.155