Currently we have 9 3 tools running tomcat webservices. In the long term we'd like to not be responsible for maintaining tomcat - need to find an alternative that works for all these tools. Ideally they'd use an embedded HTTP server like Jetty (for new users) and for older ones we find a equivalent that can work.
Running tomcat job list as of 2021-09-26T18:25Z based on https://sge-status.toolforge.org/ and the "tomcat-..." naming convention:
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Drop setup-tomcat | labs/toollabs | master | +1 -25 | |
| Drop Tomcat support | operations/software/tools-webservice | master | +0 -156 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | taavi | T141396 Provide a migration path for tools running tomcat | |||
| Resolved | Benjavalero | T210827 Help to run Java8 web-app |
For folks who just have a war file that they want to run, https://www.eclipse.org/jetty/documentation/9.4.x/runner.html might be a lighter and easier to script replacement for tomcat.
In my case, my Java tool replacer is prepared to be packaged as a WAR to be run in Tomcat, or as an SpringBoot standalone JAR.
I have been running the tool as a WAR so far. On the other side, when running the tool as a JAR in the lab server, I am not able to access it from the outside (https://tools.wmflabs.org/replacer/), I guess because of the port used.
Any hint about the port to use so the tool can be accessed in a regular browser ?
For a job grid webservice, a PORT environment variable will be exported by the webservice-runner process that is spawned to start your webservice.
For a Kubernetes webservice, the expected port is always 8000.
I have tried the port 8000 with no success.
I have started the Kubernetes JDK8 shell:
webservice --backend=kubernetes jdk8 shell
Then I have started the standalone JAR with an embedded Tomcat server:
java -jar replacer.jar
In the log displayed, I can read the following message:
Tomcat started on port(s): 8000 (http)
But when I try to access the tool as usual in https://tools.wmflabs.org/replacer/ :
The URI you have requested, /replacer/, is not currently serviced.
The shell action does not set the attributes on the Kubernetes pod which are required for our ingress solution to connect the frontend proxy to the pod. It took a bit of digging for me to figure out how the jdk8 backend is expected to work. Eventually I determined that it is setup to use our "generic" startup mechanism.
This process should work to start your tool as a Kubernetes webservice:
#!/bin/sh cd $HOME exec java -jar $HOME/replacer.jar
$ webservice --backend=kubernetes jdk8 start $HOME/replacer.shIt would be a bit nicer if you could give the whole java -jar replacer.jar command directly to webservice, but the options parsing the webservice script is doing right now has problems with any "extra" args that start with '-'.
The proposed solution has worked for me, so I will stop working with Tomcat. This will allow me also to finally migrate to Java 8.
For the record, besides changing the port, I have had to also change the context path in the Spring Boot configuration file:
server: port: 8000 servlet.context-path: /replacer/
Thanks a lot!
Is there anything left to do here? @bd808 is there some magic not documented anywhere to see if we have any tomcat tools remaining?
Running tomcat job list as of 2021-05-17T15:22Z based on https://sge-status.toolforge.org/ and the "tomcat-..." naming convention:
Tool pbbot has been moved to k8s. I'm a bit surprised that it stays well within default memory limits (512 Mi), while this would not be possible in sge (needs 5-6 Gi).
Glad to hear that!
Part of the problem with JVM based tools on the grid engine is that grid jobs are not contained in a cgroup or any other operating system level constraint. This in turn means that the JVM thinks that it has all the cpu cores and ram of the underlying grid exec node to itself, and so it asks the kernel for resources based on that assumption. When running on Kubernetes, the JVM process is started within a Kubernetes managed container (currently Docker) which means that when the JVM interrogates it's environment it sees the limits set for that container and responds appropriately by requesting resources that "fit" inside the container's limits.
Thank you for the ping, @Majavah.
geolink has been converted to PHP as part of another tool and can be deleted (T291801).
So looks like the Buster migration broke the Tomcat backend, as the Tomcat install is tied to a specific Java version and re-creating the Tomcat installs has been broken due to T329433. Since no-one noticed this I'd like to go ahead and just remove the Tomcat backend.
Change 888295 had a related patch set uploaded (by Majavah; author: Majavah):
[labs/toollabs@master] drop setup-tomcat
Change 888296 had a related patch set uploaded (by Majavah; author: Majavah):
[operations/software/tools-webservice@master] Drop Tomcat support
Change 888296 merged by jenkins-bot:
[operations/software/tools-webservice@master] Drop Tomcat support
This will be fully complete after we make new toolforge-webservice and misctools releases.