How to change/update configuration

Yeah, seems like a big one. In practice it seems like copying and editing a sample values.yaml like we have now leads to a lot of situations where the user's copy is out of sync with expectations about structure or individual values. Obviously this'll diminish if we don't make changes to the expected format, but maybe a layer of indirection here is worth thinking about...

A couple of other thoughts:

• If we did a Python CLI maybe we could write a little Flask app or something that'd optionally expose a local web interface.
• ...but maybe that's reinventing the wheel. Is there some existing thing we should just be using?

We had a meeting to discuss our next steps.

We talked about whether we would like to use Go or Python to create a CLI with a web interface. Pros and Cons were as follows:
Go
(pros)
Kubernetes, minikube, etc written in Go so could integrate nicely
Platform independent

(cons)
Not many Wikimedia developers use Go so could limit contribution

Python
(pros)
Used more by Wikimedia developers
Could use something like pywebview to easily create a web interface

(cons)
not necessarily platform independent

We discussed how to optimally support having multiple releases and change/update configuration more easily.
Skaffold was mentioned as a solution, but we are going to investigate alternatives and see whether we should use an existing solution which could fill our present and future needs, or whether they are to heavy and we should create our own.

Here is a sample skaffold configuration with the build step commented out.

apiVersion: skaffold/v1beta7
kind: Config
#build:
#  artifacts:
#  - image: mediawiki
#    context: ../core
#    push: false
#    sync:
#      '**/*/': .
#  - image: restbase
#    context: ../restbase
#    push: false
#  - image: parsoid
#    context: ../parsoid
#    push: false
deploy:
  helm:
    releases:
    - name: "my-release"
      chartPath: .
      skipBuildDependencies: true
      valuesFiles:
        - values.yaml

pywebview - kinda like electron but much simpler and built on python.

pywebview is a lightweight cross-platform wrapper around a webview component that allows to display HTML content in its own native GUI window. It gives you power of web technologies in your desktop application, hiding the fact that GUI is browser based. You can use pywebview either with a lightweight web framework like Flask or Bottle or on its own with a two way bridge between Python and DOM.

pywebview uses native GUI for creating a web component window: WinForms on Windows, Cocoa on macOS and QT or GTK on Linux. If you choose to freeze your application, pywebview does not bundle a heavy GUI toolkit or web renderer with it keeping the executable size small. pywebview is compatible with both Python 2 and 3.

gitkube

Gitkube is a tool for building and deploying docker images on Kubernetes using git push.

After a simple initial setup, users can simply keep git push-ing their repos to build and deploy to Kubernetes automatically.

wails.app

kinda like pywebview but for go

My two cents, I wanted to suggest consideration of PHP and JavaScript as languages to be used for creation of the CLI. In particular Symfony Console for PHP or Commander.js for JS. The main advantages IMHO are that one of these languages would be more friendly to contributions from existing MediaWiki developers. A secondary benefit, again very much IMHO :), is that installing the language and applications in these languages is more straightforward than Python. I haven't evaluated the client libraries for Kubernetes but I imagine they will be usablle.

Also, if the CLI is in PHP, we could put the entire application in a phar so it's just a single file to download for users who don't want to hack on it. (A binary built with Go has this advantage too.)

In T224939#5256605, @kostajh wrote:

My two cents, I wanted to suggest consideration of PHP and JavaScript as languages to be used for creation of the CLI.

PHP is among the contenders and I even suggested basing something on the existing MediaWiki maintenance script framework. Although it's a bit clunky I think it could possibly be cleaned up / modernized slightly to form a basis for what we want to do.

Also, if the CLI is in PHP, we could put the entire application in a phar so it's just a single file to download for users who don't want to hack on it. (A binary built with Go has this advantage too.)

This one is pretty compelling and it had us leaning slightly towards GO, especially because supporting Windows is a goal for this tool. Another goal is to leave open the possibility for having a web-based interface on top of a cli tool. Using something like pywebview or wails.app would be the ideal, IMO, however, PHP's built-in web server or a node.js web app could also meet the same need.

I'm biased in favor of some non-JS solution partly because I'm not personally great at JS, but mostly because it feels like there's a lot of installation overhead there for something that needs to run easily on a client machine as part of a bootstrapping process. That said, I concur that we should give due consideration to languages already very well understood in our developer community.

I've used PHP for CLI interfaces before and not found it too burdensome, really. As long as there's some scaffolding for arg parsing etc.

I think I'd break my understanding of the candidates down something like:

Probable ease of install, best to worst -

1. Go (we distribute static binaries)
2. Python (present or installable on most systems?)
3. PHP (easy on GNU/Linux, probably harder on Windows/Mac?)
4. JavaScript

Existing knowledge in community, most to least -

1. PHP
2. JS / Python (I'm guessing one or the other of these is actually higher?)
3. Go

I'd say the order of expertise is probably PHP/JS > Python > Go. But really it doesn't matter as much – we're not expecting most people to need to patch the installer/configurator, are we? Just edit config files? (Also node is likely going to be present on more machines than PHP or Python for CLI execution soon, if not already, especially on e.g. Windows or Macs.)

In T224939#5257358, @brennen wrote:

...

2. Python (present or installable on most systems?)

Noteworthy apple announcement: https://developer.apple.com/documentation/macos_release_notes/macos_catalina_10_15_beta_release_notes#3318248

Scripting language runtimes such as Python, Ruby, and Perl are included in macOS for compatibility with legacy software. Future versions of macOS won’t include scripting language runtimes by default, and might require you to install additional packages. If your software depends on scripting languages, it’s recommended that you bundle the runtime within the app. (49764202)

@thcipriani: at least on a mac with homebrew, it's as simple as brew install python

@thcipriani: at least on a mac with homebrew, it's as simple as brew install python

I dunno, for me that's just where the problems begin :) IMO without using virtualenv it's quite easy to end up with a broken python environment after you've done a few pip installs.

Another thing I was thinking about, though, is that the choice of the language may not matter a ton if there is a good hook system which will execute scripts pre/post commands. The hook system could default to calling bash scripts and from there developers could use the language of their choice.

In T224939#5271341, @kostajh wrote:

Another thing I was thinking about, though, is that the choice of the language may not matter a ton if there is a good hook system which will execute scripts pre/post commands. The hook system could default to calling bash scripts and from there developers could use the language of their choice.

This is my thinking as well. I think we can be fairly language agnostic.

Change 525227 had a related patch set uploaded (by 20after4; owner: 20after4):
[releng/local-charts@master] Skeleton go-based cli

https://gerrit.wikimedia.org/r/525227

Change 525563 had a related patch set uploaded (by 20after4; owner: 20after4):
[releng/local-charts@master] WIP adding the start command and managing minikube

https://gerrit.wikimedia.org/r/525563

Change 525227 merged by Thcipriani:
[releng/local-charts@master] local-charts: Skeleton go-based cli

https://gerrit.wikimedia.org/r/525227

Change 525563 merged by 20after4:
[releng/local-charts@master] local-charts: CLI for managing minikube, helm, etc

https://gerrit.wikimedia.org/r/525563