Migrate and re-deploy eventstreams using service-utils
Closed, ResolvedPublic8 Estimated Story Points
Actions

Assigned To

Authored By

	Ahoelzl
	Apr 4 2024, 12:25 AM

Project Tags

Referenced Files

None

Subscribers

Description

In our effort to standardize our Nodejs service implementation stack, eventstreams should be migrated and re-deployed to the new service-utils library.

https://phabricator.wikimedia.org/T360924

Details

Related Changes in Gerrit:

	Subject	Author	Repo	Branch	Lines +/-
	Eventstreams: Bump image to v0.14.0	TChin	operations/deployment-charts	master	+2 -2
	Eventstreams: Bump image, use service-utils	TChin	operations/deployment-charts	master	+6 -3

Customize query in gerrit

Related Changes in GitLab:

	Title	Reference	Author	Source Branch	Dest Branch
	Add libssl-dev to docker image	repos/data-engineering/eventstreams!16	tchin	libssl-dev-test	master
	Initial migration to service-utils	repos/data-engineering/eventstreams!6	tchin	service-utils-migration	master

Customize query in GitLab

Related Objects
Search...

Status	Assigned	Task
Resolved	tchin	T360924 Replace service runner with a simplified library to better support metrics and debugging: service-utils
Resolved	tchin	T361769 Migrate and re-deploy eventstreams using service-utils
Resolved	tchin	T386204 Update eventstreams Grafana Dashboards to use histogram for router metrics
Resolved	tchin	T386709 Update eventstreams Grafana Dashboards to fix GC metrics

Event Timeline

Ahoelzl created this task.Apr 4 2024, 12:25 AM

Restricted Application added a subscriber: • Aklapper. · View Herald TranscriptApr 4 2024, 12:25 AM

Ahoelzl set the point value for this task to 8.Apr 4 2024, 12:25 AM

Ottomata removed Ottomata as the assignee of this task.Jun 12 2024, 1:47 PM

Ottomata added a parent task: T360924: Replace service runner with a simplified library to better support metrics and debugging: service-utils.

Ottomata subscribed.

• lbowmaker edited projects, added: Data-Engineering (Q1 2024 July 1st - September 30th); removed: Data-Engineering (Q4 2024 April 1st - June 30th).Jul 12 2024, 2:38 PM

Ottomata assigned this task to tchin.Jul 18 2024, 4:34 PM

Ottomata moved this task from Next Up to In progress on the Data-Engineering (Q1 2024 July 1st - September 30th) board.

tchin renamed this task from Migrate and re-deploy eventstreams using new service runner to Migrate and re-deploy eventstreams using service-utils.Jul 18 2024, 4:39 PM

Snwachukwu moved this task from In progress to In Review on the Data-Engineering (Q1 2024 July 1st - September 30th) board.Jul 29 2024, 3:11 PM

Jdforrester-WMF subscribed.Aug 21 2024, 1:59 PM

Eventstreams is currently deployed in the beta cluster successfully with service-utils
It turns out that KafkaSSE uses bunyan, so logging is a bit weird since now it uses 2 formats
Beta logstash does not seem to capture logging from stdout, so it only shows the logs from KafkaSSE. However, the eventstreams logs do exist when looking inside the docker container

@tchin can you detail the remaining work to complete this task and get eventstreams deployed in production? Thanks.

Just saw that service-utils passed security review, so I think the only step right now is just to release service-utils under @wikimedia/service-utils and deploy the new eventstreams to production