Title of session: Toolforge & Cloud VPS demos
Session description: Demo session about new and lesser-known existing features in Toolforge and Cloud VPS, including the latest Build Service features, object storage in Cloud VPS, recent improvements to the Toolforge jobs framework, and more. If there's anything specific you'd like to se demo'd, comment in the task.
Username for contact: @Slst2020
Session duration (25 or 50 min): ?
Session type (presentation, workshop, discussion, etc.): presentation/demo
Language of session (English, Arabic, etc.): English
Prerequisites (some Python, etc.): This session is mainly targeted at users who have one or more projects deployed in Toolforge or Cloud VPS, but anyone is welcome
Any other details to share?:
Interested? Add your username below:

Notes from session:

Toolforge & Cloud VPS demos

Date and time: May 5, 2024 11:30am

Relevant links

• Phabricator task: https://phabricator.wikimedia.org/T362062
• Session slides:

Links

• Example apps:
  • Django: https://gitlab.wikimedia.org/toolforge-repos/django-test-buildpack-app
  • FastAPI: https://gitlab.wikimedia.org/toolforge-repos/fastapi-blueprint-buildpack-nlp https://gitlab.wikimedia.org/toolforge-repos/sample-python-buildpack-asgi
  • Flask + NodeJS: https://gitlab.wikimedia.org/toolforge-repos/nodejs-flask-buildpack-sample
  • Ruby on Rails: https://gitlab.wikimedia.org/toolforge-repos/sample-ruby-rails-buildpack-app
• https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service

Presenter

@[[phab:p/Slst2020/|Slst2020]]

Participants

• Tarrow
• Brennen
• Jay Prakash
• Gopa Vasanth
• NicoleLBee
• Damilare
• @Wfan

+13 participants

Notes

• Build Service
  • Based on Heroku buildpacks. See: buildpacks.io
  • Builds a container image without having to have Dockerfiles
  • example app https://gitlab.wikimedia.org/toolforge-repos/django-test-buildpack-app
  • Configure

use a Procfile to tell buildpack how to run your image

• • • "web" is a magic keyword for web service
    • even without a Procfile you can often get away with it
    • requirements.txt with dependencies
    • runtime.txt: optional, specifies runtime version
  • you might need to configure your app to use some toolforge service (like an sql db). In order to pass in credentials for this you may well want use env variables
  • Set Environment variables
    • <code>become django-test</code>
    • <code>toolforge envvars</code> - create, delete, list, etc.
      • TOOL_ vars are available without defining a file
  • Build
    • toolforge build start [repo path]
    • Path needs to be accessible to public
    • Q: What if we make a local change to file?
    • A: Needs to be in the repo. Not possible to do from a local checkout. This also helps force tools to be open source.
    • Q: Can we build from a branch?
    • A: Yes - <code>toolforge build --ref</code> lets you point to another branch
    • Q: Is it possible to build on push?
    • A: Not yet, working on that. A lot of things to define first.
  • Build Quota
  • Run a one-off job (e.g. db migration)
    • <code>toolforge jobs run --wait --image [tool]/[image] --command [name] somejob
    • If something goes wrong, it'll tell you, then you can look at the logs. Can be complicated because the pod will get reaped after a while - we're trying to improve this.
  • Deploy
    • <code>toolforge webservice buildservice start --mount none</code>
    • <code>--mount none</code> is because we need to be explicit that we don't need NFS
    • <code>toolforge webservice buildservice restart</code>
  • Q: Ports?
  • A: Buildpacks know what defaults are for various application frameworks etc.
  • Q: How do you configure toolforge to contact health check endpoints?
  • A: Health check demo...
  • Health check
    • <code>toolforge webservice --help</code> can see the healthcheck cmd
    • Basic TCP probes are defined per webservice
• Jobs Framework
  • <code>toolforge jobs list</code> will show name type and status of jobs
  • Dump
    • <code>toolforge jobs dump > jobs.yaml</code> - dump running jobs into a YAML file named jobs with all the info that's needed to redeploy them.
    • Can be loaded with <code>toolforge jobs load jobs.yaml</code>
    • Very helpful if trying to maintain someone else's tool and you don't know what commands they used to generate jobs. Gives you the bare minimum that the framework needs to run the exact same thing.
• Q: Is this the desired path for deploying tools going forwards?
• A: Yes.
  • Health check (only continuous jobs)

Links

• Example apps:
  • Django: https://gitlab.wikimedia.org/toolforge-repos/django-test-buildpack-app
  • FastAPI: https://gitlab.wikimedia.org/toolforge-repos/fastapi-blueprint-buildpack-nlp https://gitlab.wikimedia.org/toolforge-repos/sample-python-buildpack-asgi
  • Flask + NodeJS: https://gitlab.wikimedia.org/toolforge-repos/nodejs-flask-buildpack-sample
  • Ruby on Rails: https://gitlab.wikimedia.org/toolforge-repos/sample-ruby-rails-buildpack-app
• https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service

Questions

Q: Are there other keys for the procfile, other than web and migrate?
A: The only important key is "web", other keys can be manually defined

Photos

Social