Some aspects I see for each option:

Option 1

Pros:

• We keep using python of which we might have more experience with
• A lot of the current code can be reused

Cons:

• No single-binary packaging by default (there's some options around that we can explore: https://github.com/pyinstaller/pyinstaller)

Option 2

Pros:

• Single binary out of the box
• We know some golang
• Total control over what we want

Cons:

• Current code can't be reused

Option 3

Pros:

• Probably the least maintenance effort (autogenerated)

Cons:

• Needs to have a unified API definition, something that we might want anyhow (allows generating many language libraries and clis)
• Current code can't be reused

Agree with @dcaro's points, and also:

Option 1

Pros:

• Reuse of code would likely speed up the migration
• Allows us to do a gradual migration, merging one CLI at a time
• New contributors may be more familiar with Python than Go

Cons:

• Python is generally slower than Go (although that may not matter much in this specific case as the real speed bottleneck probably is network latency)

Option 2

Pros:

• Go is generally faster, although see above

Cons:

• Go's ecosystem is not yet as rich as Python's
• Go can be quite verbose for some tasks for which Python has higher-level abstractions; longer development time

Option 3

Same as for Option 2, and also:
Pros:

• Autogeneration minimizes the risk of human error
• Open API specifications can be easily converted into comprehensive API documentation

Cons:

• Autogenerated code may not take full advantage of language-specific features or optimizations, potentially leading to less idiomatic or less optimized Go code.
• Introduces a dependecy on the tooling around Open API and code generation

Allows us to do a gradual migration, merging one CLI at a time

That can be done with the option 2 also (the current modular design allows for easy plucking of each cli), and probably option 3 (depends on how we generate the clis, we already started looking into this when generating the builds cli from the api spec at the beginning, though we dropped that effort in favor of python).

Autogenerated code may not take full advantage of language-specific features or optimizations, potentially leading to less idiomatic or less optimized Go code.

Note that in the case of the autogenerated cli, we might not want any contributions (as we might not have to code anything there), case in which being less idiomatic becomes a non-issue (that imo is a big plus). Also as you mention in the option 1, performance is not a bottleneck, so language-specific optimizations are not that relevant either.

I don't have a strong preference, but I vote for Option 1 as I would prefer to tackle the different objectives separately:

1. consolidating the CLI
2. migrating to Go (can be done later if we like)
3. creating an OpenAPI definition (can be done later if we like)

Trying to achieve 2 or 3 things together seems (slightly) more risky and time-consuming, so my preference is to focus on objective 1, using Option 1.

I vote for option 3, as it's the one that will require less effort duplication, given that the api definiton is something that we want to do anyhow.
It achieves the cli consolidation without having to change any cli code, or rewrite any code, by generating it from the API definition.
And it achieves the binary cli by generating it in golang/java/whatever binary compiled lang we want.

As opposed to having to put effort to consolidate the cli, then more effort rewriting it in a compiled lang to distribute a binary, and then more effort again creating the API definition + cli generation.

What would "autogenerated CLI" mean? I remaing sceptical that you can reliably do something more than generate an OOP wrapper around the API methods, which would do nothing about parsing and wiring up parameters, dealing with output formatting, handling changes and deprecations in functionality, etc.

For option 3, once the openapi specification is complete, I presume you could also generate a python client? It seems the implied goal is to end up with go client binary, but the openapi option seems distinct from option 1 or 2?

In other words, I'm reading this request as follows. The toolforge cli is being consolidated. Choices are: Do it in python or do it in golang. And further, use openapi to do it (or not ). Is this accurate?

Any thoughts on complexity for each option? A casual reading suggests they may be in order of increasing complexity or close.

What would "autogenerated CLI" mean? I remaing sceptical that you can reliably do something more than generate an OOP wrapper around the API methods, which would do nothing about parsing and wiring up parameters, dealing with output formatting, handling changes and deprecations in functionality, etc.

There is always a few things that will need some manual coding, but the idea is to leave those at a the minimum possible and use the autogenerated code in as many places as we can. The 'basic/poc' generated code we used for the bulids-service already wrote the cli subcommands, parser, help strings and parameter validation itself, if that's what you are skeptical about.

In the worst case (that I highly doubt will be), generating the libraries out of the API is already a huge improvement.

For option 3, once the openapi specification is complete, I presume you could also generate a python client?

Yes, you can generate libraries + client on many languages.

It seems the implied goal is to end up with go client binary, but the openapi option seems distinct from option 1 or 2?
In other words, I'm reading this request as follows. The toolforge cli is being consolidated. Choices are: Do it in python or do it in golang. And further, use openapi to do it (or not ). Is this accurate?

We mention golang here as one of the goals discussed in the previous decision task was to be able to "easily install the client for users", and having a single self-contained binary to download as generated by golang is a big plus there, versus having to download a debian package + dependencies (that is what option 1 will have). See T346153#9243305

Any thoughts on complexity for each option? A casual reading suggests they may be in order of increasing complexity or close.

I would say that the easiest might be option 1 (depending on how we migrate them), as in just needs gluing the existing code together, though might need some refactors.
The more complex would be option 2, as it requires a manual rewrite of every cli.
And in between would be option 3, that needs a bit of effort to get the API definition, and a bit of effort and exploration to get the generated client usable.

Note that none of these options mentions that the webservice cli needs some modifications to be consolidated with the rest, and it deals with both k8s and grid (and it's quite complex), so I'd migrate the k8s side of it to an API before doing so and use that only on the cli. That pre-effort is needed and shared with all of the options.

In my opinion I think we should go with Option 1 in the short term and Option 3 in the long term. Option 2 is totally out of the question in my opinion because it offers no real benefit over Option 1 (the one advantage it has, speed, is not really a bottleneck for the type of application we are building so doesn't mean much, yet we have to worry about the time it will take to complete).
Option 1 in the short term because we want to start enjoying the benefits of the merge as soon as possible and Option 3 will likely take a while before we can come up with a complete openapi spec.
Option 3 in the long term because it helps us standardize the way we develop stuffs for toolforge. I can see the spec created for Option 3 being used in the future for something like toolforge UI

In T348749#9272678, @dcaro wrote:

I vote for option 3, as it's the one that will require less effort duplication, given that the api definiton is something that we want to do anyhow.
It achieves the cli consolidation without having to change any cli code, or rewrite any code, by generating it from the API definition.
And it achieves the binary cli by generating it in golang/java/whatever binary compiled lang we want.

As opposed to having to put effort to consolidate the cli, then more effort rewriting it in a compiled lang to distribute a binary, and then more effort again creating the API definition + cli generation.

+1 to all of this.

If Option 3 is the end goal, passing through Option 1 will just be time consuming without adding enough additional benefits, in my opinion.

In T348749#9273891, @Raymond_Ndibe wrote:

Option 1 in the short term because we want to start enjoying the benefits of the merge as soon as possible and Option 3 will likely take a while before we can come up with a complete openapi spec.

Maybe I'm misunderstanding how this works, but I think we don't have to create the spec all at once? Even if this is the case, I think it would be worth the extra wait/effort. As @dcaro mentioned, a unified API definition is probably something that we might want anyhow.

tl;dr I vote for option 3

Examples of golang medawiki-related cli:

https://gitlab.wikimedia.org/repos/releng/cli + automated documentation https://www.mediawiki.org/wiki/Cli/ref/mw

https://github.com/CanastaWiki/Canasta-CLI/

As there is no clear consensus, a decision meeting will be scheduled as described here: https://www.mediawiki.org/wiki/Wikimedia_Cloud_Services_team/Decision_Making