|mediawiki/services/function-orchestrator||master||+82 -30||Allow the orchestrator to support multiple executors.|
|Open||None||T299176 Phase Theta – Root task|
|Open||None||T282913 Implement agreed security and control measures in the Wikifunctions system|
|Open||cmassaro||T282923 Provide a mechanism by which sysadmins can easily disable/re-enable specific language evaluators from Wikifunctions|
Helm charts, playbooks needed (assuming we're doing different service endpoints for the evaluators in different languages).
James's estimated effort: 5 days? (if whoever picks this up thinks that will take considerably longer, please check whether we agree on the task)
But mostly final setting up of the charts which we need for prod launch anyway.
There are some things we still need to think through about the Helm charts and Kube configuration.
Here's a sketch of how to redesign the backend (see this document, Option 1, for more context).
The orchestrator now needs to know about multiple evaluators. We will need a mapping from programming languages to evaluator URIs which we consult before sending native code off for evaluation. The orchestrator will need to receive that mapping as some kind of configuration (can even be a series of env variables like the one we have now for the evaluator URI) and keep track of multiple Evaluator objects.
The orchestrator also needs to make decisions about which programming language a given function call needs. The logic in function-evaluator that maps programming language strings to executors will need to be moved to the orchestrator.
After this patch (or two), we can start putting two identical copies of the evaluator up in beta cluster: one for JS and one for Python. We'll need a patch (or several; I don't know much about how beta cluster is configured) to manage that.
After that, we can start differentiating the evaluators into executors for single programming languages.
First, we'll need to move the function-schemata submodule (or copy it) from the JS executor code to the service-layer code (service-template-node stuff) in function-evaluator.
After this series of patches, we'll have implemented what the design document for this work refers to as "option 1." It will then be possible to spin up/spin down executors for individual programming languages.
Adding a new programming language in future will then look like this:
- implement an executor (which may but need not use the current subprocess-based design);
- this includes service-layer concerns, a new Blubber configuration, AND everything from before: (de)serialization, code injection, etc.
- add a new Evaluator in function-orchestrator that is initialized from relevant config (perhaps just an env variable, e.g. CPP_EXECUTOR_URI);
- add the new executor service to production config in Kube, Helm, etc.