Subtask of the TEC3:O3:O3.1:Q4 Goal to migrate cpjobqueue to use the deployment pipeline
|Open||None||T198901 Migrate production services to kubernetes using the pipeline|
|Resolved||hnowlan||T220399 Migrate cpjobqueue to kubernetes|
|Resolved||holger.knust||T244387 Change-Prop consumer group must respect service name|
|Resolved||holger.knust||T245803 Make changeprop chart depend on Kafka-dev for minikube|
- Mentioned In
- rLPRI249979c079cf: changeprop-jobqueue: add stubs for secrets
T247838: Changeprop to k8s
rDEPLOYCHARTS151901e3efd0: changeprop: Correctly align the prometheus-statsd.conf call
rDEPLOYCHARTS03deee23cd22: changeprop: Package 0.9.5
rDEPLOYCHARTS8cc23fb85b00: Package changeprop charts and update index
rDEPLOYCHARTSadc3111ef230: Migrate changeprop & cpjobqueue to kubernetes
cpjobqueue is now running fully on Kubernetes. The instance running on scb has no rules enabled. All that remains to be done is to remove the puppet configuration and delete the deploy repo once that's done.
@hnowlan @akosiaris https://gerrit.wikimedia.org/r/603534 deleted 2 admin groups , changeprop-admin and cpjobqueue-admin. But these groups were still applied on kafka-main hosts in Hiera. This broke puppet on all kafka-main hosts as reported by @elukey
I merged 2 follow-ups that removed both groups from the kafka-main hosts and that unbroke puppet but it also actively removed the shell users eevans, ppchelko, mobrovac,... So now only root users can get on kafka-main hosts. I don't know why they were originally added yet. Just making sure that is expected and they actually don't need kafka hosts anymore. Do we need to look closer and ask them?
I would like to keep shell access to kafka hosts. Technically, we are still running both change-prop and cpjobqueue, just in k8s, and we still need the shell access for the same reasons as we did before, when services were running on scb.
Should it just be something like "kafka-users" maybe?
Sounds good. However, thinking more about it, mobrovac has left the foundation, @Eevans probably doesn't need kafka access. I only use my privileges to run kafkacat on the hosts, so I could probably live without access if there's another hosts in production where kafkacat is installed. I can run my queries form stat1007 for example.
Thank you. I can live with that, I have access to a number of places. Sorry I didn't think of a workaround like that. In case this will not be sufficient, I'll re-request access to kafka hosts.