Page MenuHomePhabricator

Run the rdf-streaming-updater from k8s@codfw
Closed, ResolvedPublic2 Estimated Story Points

Description

This is a followup on the incident T314835.

To mitigate the issue the flink job was started from yarn, we should move it back to k8s.

We should coordinate with the cleanup done in T316003:
Option1:

  • keep using the T314835 subfolder (even for the flink_ha storage folder)
  • once the cleanup is done, fully redeploy the service under the root folder

Option2:

  • wait for the cleanup to be done and deploy the service using the root folder

Option1 should be preferred if the cleanup is going to take several weeks.

AC:

  • WDQS & WCQS flink jobs are running from k8s@codfw

Event Timeline

Gehel set the point value for this task to 2.Aug 29 2022, 3:35 PM
Gehel moved this task from Incoming to Prioritized on the Discovery-Search (Current work) board.

Will be picked up once the cleanup is done (T316031).

Change 833991 had a related patch set uploaded (by DCausse; author: DCausse):

[operations/deployment-charts@master] rdf-streaming-updater: use rdf-streaming-updater-codfw swift container

https://gerrit.wikimedia.org/r/833991

Change 833991 merged by jenkins-bot:

[operations/deployment-charts@master] rdf-streaming-updater: use rdf-streaming-updater-codfw swift container

https://gerrit.wikimedia.org/r/833991