As an operator, I want to have automation or documented procedures to standard operations so that I can react to issues in a timely manner.
AC:
The following cookbooks are to be adapted, or verified that they still work:
The following documentation is reviewed:
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Gehel | T244590 [Epic] Rework the WDQS updater as an event driven application | |||
| Resolved | • Zbyszko | T276469 Cookbooks and / or operation procedures are adapted for the new Flink based WDQS Streaming Updater |
Note that there's a WIP patch to ease the transfer of kafka offsets (while transferring a journal from one node to another) that will no longer maintained in the triple store. The solution is up for discussion and especially determining the priority of the source of offsets. Should we always pick kafka maintained offsets or the ones in the file or something smarter like take the most recent ones if they're both valid?
Change 721857 had a related patch set uploaded (by ZPapierski; author: ZPapierski):
[operations/puppet@production] Add kafka clusters' brokers to spicerack config
Change 723214 had a related patch set uploaded (by ZPapierski; author: ZPapierski):
[operations/software/spicerack@master] Added spicerack.kafka with offset transfer function
Just making sure yall have considered using consumer offset sync as supported by MirrorMaker 2.
Upgrading to MM2 would be a big deal, but we haven't done it because no one has needed it! If there is a need it would be a good reason to upgrade.
Change 721857 merged by Gehel:
[operations/puppet@production] Add kafka clusters' brokers to spicerack config
Change 727021 had a related patch set uploaded (by ZPapierski; author: ZPapierski):
[operations/cookbooks@master] [WIP] Add kafka position transfer to wdqs cookbooks
Change 723214 merged by jenkins-bot:
[operations/software/spicerack@master] Added spicerack.kafka with offset transfer function
Change 727021 merged by Gehel:
[operations/cookbooks@master] sre.wdqs: Add kafka position transfer to wdqs cookbooks