|Resolved||bd808||T140110 Packages to be installed in Toolforge Kubernetes Images (Tracking)|
|Open||Gopavasanth||T220228 GSoC19 Proposal : VideoCutTool to cut videos in Commons|
|Resolved||bd808||T195103 Add Support for Node.js 10 LTS to Toolforge|
- Mentioned In
- T223784: Announce availability of node 10, npm 5.8 and python 3.5 for Toolforge Kubernetes webservices
T169451: newer npm for nodejs Kubernetes instances
rODIT5b5c8147f88f: node10: new images
T212981: npm is missing from Stretch image
T213641: Design mechanism and process for upgrading Kubernetes container runtimes
- Mentioned Here
- T203239: Create Debian packages for Node.js 10 upgrade
We have a node.js docker image already, so I think this is just a matter of either updating or creating a new one.
Creating a new image should be less disruptive for people using the existing image.
Since you seems more familiar with docker than I am, would you mind sending a patch for the repo?
We could follow up with building the image and uploading to our registry
If 'this' is using upstream Docker images then yes. Currently we require various features in the base image (notably LDAP support in NSS and the webservice python package) in order to run a container on the Toolforge Kubernetes cluster.
Removing these requirements at some unspecified point in the future is possible, but will require discussion, planning, and implementation changes that are not likely to be considered seriously during the coming fiscal year. For better or worse, webservice has a history dating back to (and still supporting) grid engine deployment, NFS storage, and usage of per-tool user accounts as the runtime process owner. Changing all of these assumptions will be quite a bit of work and also require a migration plan for the existing tools. The resulting change from an end user perspective will be quite large.
The more reasonable near term solution is for us to decide how to evaluate external Debian package providers (for example https://deb.nodesource.com) and determine if they are trusted for use in our containers. Once we have that evaluation criteria we can use it to evaluate package providers who provide newer versions than the base Debian distribution and eventually use them to produce images with these newer language runtimes.
I've found NodeSource support to be very good for the official packages. They usually release new packages very quickly following security updates from upstream and I can't remember a regression that was due to their packaging. Since they're also referred to in the official documentation, I would consider them a trusted source.
That doesn't mean we have to enable the NodeSource APT source in our containers and install from it. I would suggest importing their distributions repository and then follow our usual Gerrit workflow to incorporate updates and upload new packages to apt.wikimedia.org. Would that be acceptable and not cause issues if production decides to stay in 6.x for while? I worry that, if version pinning is not widespread for the nodejs package, people would be getting the newer node packages by accident. It might be useful to incorporate the major number in the package name (besides having the version) to avoid conflicts. Is it in line with Wikimedia packaging guidelines?
I've also found that the migration from 0.10 to 6.x was very traumatic but that subsequent updates to 8.x and even 10.x required very little work from the Node.js apps I was supporting at the time. Which makes developers more eager to use the latest and greatest if things don't break as they used to. I would suggest we add versioned Docker images like node8 and node10 (and refrain from using dev versions to avoid additional support burden).
If we follow this approach and drop support as soon as a major version becomes EOL, the node (6.x) image would have another 6-8 months of life, the to-be-created node8 package another 12-14 months and node10 would be supported up to April 2012 (see the infamous LTS page).
The webservice command would also need to be modified to accept node8/10. I'm sure I'm missing some important details here, please let me know
Per T203239, prod and CI have gone from Node 6 to Node 10. Maintaining a separate package for Toolforge at Node 8 is possible, but would require extra cost/maintenance from Cloud Services and/or a volunteer (with their permission).
Might be easier to go to Node 10 directly. Updated the task line as such, given this seems uncontroversial to me, but feel free to undo.
@Krinkle do you have any advice about what warnings if any would need to be given to tool maintainers using the current nodejs v6.11.0 Docker image if we switched the image out for one running the new nodejs 10 package that has been imported for Stretch? I'm just wondering if we will need to do something like we did for PHP here when we created a new image flavor so that people could opt-in rather than forcing the change on the next webservice restart.
I believe that generally, removals have been relatively rare in Node.js and most tools will probably be fine as-is.
Would there be a way to opt-out/downgrade if we go with the force-upgrade route? (E.g. for upto 3 months or something like that). If that's possible, then I think going forceful would be better. But if it's either opt-in/out or default, then we may want to go opt-in for month or two first and only then force it.
The choice is basically:
- add new 'toollabs-nodejs10-base' and 'toollabs-nodejs10-web' images and update webservice to know how to launch them
- change the existing 'toollabs-nodejs-base' Docker image to be based on Debian Stretch with the packages from our local "component/node10" repo
Neither is technically difficult. The question is basically if we should be providing more and more runtime variations or if we should be forcing upgrades to the existing runtimes. The introduction of the php72 containers presents a precedent for the nodejs10 option. At least in theory the creation of that runtime came with a promise that the older php containers would be removed at some point in the future. We have not yet followed through on that promise. This is the part that gives me pause in adding new options. Deprecations take time to communicate and some ability to monitor voluntary transition before ultimately forcing change on the users. Ultimately however, this is probably the path we really need to pursue. I need to think about how to redesign the webservice command to make doing these kinds of rolling upgrades in runtime environments easier to manage for both the maintainers of the runtimes and the end users.
$ ssh login.tools.wmflabs.org $ become bd808-test $ webservice --backend=kubernetes node10 shell Defaulting container name to interactive. Use 'kubectl describe pod/interactive -n bd808-test' to see all of the containers in this pod. If you don't see a command prompt, try pressing enter. $ nodejs --version v10.4.0 $ logout Session ended, resume using 'kubectl attach interactive -c interactive -i -t' command when the pod is running Pod stopped. Session cannot be resumed.