On one hand most of the READMEs of build service related repos are written with minikube in mind,
while on the other hand we have lima-kilo, which is an attempt to simplify the process of setting up toolforge on local machines using kind.
We should make up our mind on which kubernetes bootstrapper to use for all toolforge things. This will help us avoid having more than one k8s cluster running toolforge things and having to switch clusters to work on specific parts of toolforge
Do you have any issues with any of them?
Did you try using vagrant too? (https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/83)
If vagrant works for everyone might be easier to keep kind (as it works well there).
The history is that build service started using minikube, then jobs-* started using kind instead and then lima-kilo kept using kind.
kind does not work for me directly, but it works through vagrant. Minikube, as it runs in a VM, works for me too.
Oh, if you do test that lima-kilo patch, please +1 or add a comment with your issues in the MR
I think the issue is more of having a unified way to do toolforge stuffs (atleast I think that's what lima-kilo was trying to do). Currently that idea is being defeated by the fact we still work on different parts of toolforge by switching between two different local clusters (kind and minikube)
I think we should either move away from minikube (read rewrite any minikube specific readme's to kind, verify that that all build services stuffs work on kind and fix them if they don't), or switch to using minikube on toolforge-jobs and lima-kilo
Sounds good to me, if nobody has anything against kind (I don not with the fixes in the other task), I'd say it's simpler going the kind way xd
With the lima-kilo patch in the other task you should be able to run builds* things inside lima-kilo, if that works for you you can drop the other environment (unless you want to keep it, that's ok too).
I think we're increasingly using lima-kilo, where kind has been working fine. If I'm wrong or you feel a strong preference for minikube, please chime in. Otherwise I will close this task as resolved a week from now.