[M] Write CronJob configuration
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	kostajh
	Sep 20 2023, 6:41 AM

Description

To run regular, automated imports and updates from the external data source, we will need a Kubernetes CronJob definition for the ipoid Chart in operations/deployment-charts.

Prior art that can be helpful:

Some notes:

we should switch concurrency to 0
we probably want a backoffLimit of something like 2 or 3, to avoid retrying a bunch of times on failure

Acceptance criteria:

A cronjob.yaml template is in the ipoid charts directory, and contains the command invocation to import/update data on a daily basis

Details

Subject	Repo	Branch	Lines +/-
ipoid: disable emptyDir	operations/deployment-charts	master	+4 -20
ipoid: Set MYSQL_HOST	operations/deployment-charts	master	+1 -1
ipoid: add cronjobs for initialImport and dailyUpdate	operations/deployment-charts	master	+158 -91
ipoid: Enable the cronjob	operations/deployment-charts	master	+1 -1
ipoid: Update cronjob definition	operations/deployment-charts	master	+75 -23
[WIP] ipoid: Set an initialImport cron job	operations/deployment-charts	master	+20 -0

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
In Progress	Niharika	T324492 Temporary accounts - MVP
Open	None	T340895 [Epic] IP Info accommodations for temporary accounts
Open	STran	T341395 Display Spur data on IPInfo infobox
Resolved	kostajh	T339284 Deploy ipoid
Resolved	kostajh	T346861 [M] Write CronJob configuration
Resolved	jijiki	T350903 Rename FEED_API_KEY

Event Timeline

kostajh created this task.Sep 20 2023, 6:41 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 20 2023, 6:41 AM

kostajh mentioned this in T336165: Create Wikitech page documenting the iPoid service.Oct 15 2023, 8:03 PM

kostajh claimed this task.Oct 17 2023, 6:43 AM

Change 966813 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[operations/deployment-charts@master] ipoid: Update cronjob definition

https://gerrit.wikimedia.org/r/966813

gerritbot added a project: Patch-For-Review.Oct 18 2023, 8:31 AM

Change 967243 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[operations/deployment-charts@master] ipoid: Enable the cronjob

https://gerrit.wikimedia.org/r/967243

Change 967245 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[operations/deployment-charts@master] ipoid: Set an initialImport cron job

https://gerrit.wikimedia.org/r/967245

jijiki subscribed.Oct 23 2023, 4:33 PM

Tchanders moved this task from Backlog to Deployment blockers on the iPoid-Service board.Oct 23 2023, 6:17 PM

Tchanders added a project: Trust and Safety Product Sprint.Oct 27 2023, 10:37 AM

kostajh moved this task from Backlog to Sprint Bodhrán on the Trust and Safety Product Sprint board.Oct 30 2023, 4:55 PM

kostajh edited projects, added Trust and Safety Product Sprint (Sprint Bodhrán); removed Trust and Safety Product Sprint.

kostajh moved this task from Priority Backlog to In Progress on the Trust and Safety Product Sprint (Sprint Bodhrán) board.

kostajh moved this task from Deployment blockers to In Progress on the iPoid-Service board.Nov 1 2023, 2:36 PM

Tchanders renamed this task from Write CronJob configuration to [M] Write CronJob configuration.Nov 1 2023, 6:51 PM

kostajh moved this task from In Progress to Needs review on the Trust and Safety Product Sprint (Sprint Bodhrán) board.Nov 3 2023, 10:35 AM

Change 971933 had a related patch set uploaded (by Effie Mouzeli; author: Effie Mouzeli):

[operations/deployment-charts@master] (WIP) ipoid: add cronjobs for initialImport and dailyUpdate

https://gerrit.wikimedia.org/r/971933

Change 967245 abandoned by Effie Mouzeli:

[operations/deployment-charts@master] [WIP] ipoid: Set an initialImport cron job

Reason:

in favour of I662d33f1a9e0f0230f3247d8443505427864dc92

https://gerrit.wikimedia.org/r/967245

Change 966813 abandoned by Effie Mouzeli:

[operations/deployment-charts@master] ipoid: Update cronjob definition

Reason:

in favour of I662d33f1a9e0f0230f3247d8443505427864dc92

https://gerrit.wikimedia.org/r/966813

Change 967243 abandoned by Effie Mouzeli:

[operations/deployment-charts@master] ipoid: Enable the cronjob

Reason:

in favour of I662d33f1a9e0f0230f3247d8443505427864dc92

https://gerrit.wikimedia.org/r/967243

@STran @Tchanders @kostajh I am merging the first iteration of the jobs as I want to run a few tests on staging. Feel free to make new patches for any other changes we may need here.