You can reuse https://wikitech.wikimedia.org/wiki/Help:Toolforge/Building_container_images/My_first_Buildpack_Python_tool I think

@komla can you specify what do you need reviews of?

@dcaro it's in draft on my page so just someone to take a look/review. I will make it Python specific(as opposed to the PHP one above).

I've added the link now.

In T437899#12318856, @komla wrote:

@dcaro it's in draft on my page so just someone to take a look/review. I will make it Python specific(as opposed to the PHP one above).

I've added the link now.

Is it ready for review? The code in the repo has some working files (backups, etc.), and the wiki page seems unfinished (code is not formatted, no details on what's going on, etc.).

Btw. why not use the already existing tutorials?
(we will need to update them anyhow, and they are actual tools, like a user will have, and they can be reached through the http url like a regular tool, ex. https://sample-python-buildpack-app.toolforge.org/, then deployed periodically as a tool would, etc.)

Some early feedback:

We will deploy a Python application called recent-changes. The application runs every five minutes as a scheduled job.

We should talk about tools, continuous jobs, scheduled jobs, etc. avoid using non-toolforge terms like application (or you'll have to define what it means too). In this case, I think the equivalent would be a component.

With Push-to-Deploy, the application's source code and its Toolforge deployment configuration live together in a Git repository.

It's not 100% true, the config can live anywhere that's public. We can rephrase like "in this tutorial we will put the tool configuration and the source code in the same git repository..." instead

Open toolforge.yaml and change the repository URL so that it points to your publicly accessible repository.

The code shown is not valid toolforge config, it seems to have lost the format

The important parts of the configuration are:

Maybe use snippets of code here, it's a bit confusing for example the recent-changes entry.

toolforge components config create path/to/toolforge.yaml

When creating the configuration, instead of this you can use the one-liner:

curl 'https://gitlab.wikimedia.org/toolforge-repos/sample-php-buildpack-app/-/raw/main/toolforge.yaml?ref_type=heads' | toolforge components config create

Adapting the url to the repo of the user, that will avoid the user from having to download the config file or clone the repo in the bastion at all.

Push-to-Deploy will build the application from your Git repository and deploy the scheduled component.

It's the components-api, not push-to-deploy, push-to-deploy is the process of pushing a git commit, and then the deployment getting created, you are literally creating the deployment by hand here.

Congratulations! You have deployed a scheduled Toolforge job using Push-to-Deploy.

That's not correct, the tool was deployed manually, not using push-to-deploy (git push -> gitlab ci creates deployment)

Make a change and redeploy

With push to deploy, you don't need to redeploy manually, it's done for you by gitlab ci

Fyi. The section of the help that explains how to setup the push-to-deploy is https://wikitech.wikimedia.org/wiki/Help:Toolforge/Deploy_your_tool#Triggering_a_deployment_from_your_CI_runner

@dcaro thanks. Yeah, I will just use some of the existing repos for subsequent ones.

I converted the My first PHP tool tutorial to a Python version for this one using a clean repo(no working files etc).

The recent-changes is a codebase I used while experimenting with jobs.

A couple comments, the tool is not running correctly (https://sample-python-ptd-app.toolforge.org gives error). There's python code that's malformed (will not work if copy-pasted). There's no push to deploy configuration at all in the sample repo either (no .gitlab-ci.yml).

I recommend following the tutorial yourself when building the tool, and making sure that:

• Every command can be copy-pasted without errors
• Every file can be copy-pasted without errors
• The tool can be setup and deployed and works
• The tool actually sets up push-to-deploy and automatic deployment works correctly (in this it might be better to expand in the tutorial how to set it up minimally, instead of just redirecting to the long detailed docs)
  • You can see the setup in the php one: https://gitlab.wikimedia.org/toolforge-repos/sample-php-buildpack-app/-/blob/main/.gitlab-ci.yml?ref_type=heads
  • I also recommend setting up a scheduled pipeline that does the autodeployment once a day, so we know if the example we have breaks at any point (not part of the tutorial, but helpful, see https://gitlab.wikimedia.org/toolforge-repos/sample-php-buildpack-app/-/pipeline_schedules)

Checking ....

It's working now: https://sample-python-ptd-app.toolforge.org

I didn't update the repo url in toolforge.yaml file

Now getting no healthy upstream

From the logs:

tools.sample-python-ptd-app@tools-bastion-15:~$ toolforge jobs logs webservice

...

2026-09-15T08:19:16.399894Z [webservice-55f4987668-kdjj5] [job] [2026-09-15 08:19:16 +0000] [13] [ERROR] Exception in worker process
2026-09-15T08:19:16.399929Z [webservice-55f4987668-kdjj5] [job] Traceback (most recent call last):
2026-09-15T08:19:16.399938Z [webservice-55f4987668-kdjj5] [job]   File "/layers/heroku_python/venv/lib/python3.14/site-packages/gunicorn/arbiter.py", line 753, in spawn_worker
2026-09-15T08:19:16.399950Z [webservice-55f4987668-kdjj5] [job]     worker.init_process()
2026-09-15T08:19:16.399957Z [webservice-55f4987668-kdjj5] [job]     ~~~~~~~~~~~~~~~~~~~^^
2026-09-15T08:19:16.399966Z [webservice-55f4987668-kdjj5] [job]   File "/layers/heroku_python/venv/lib/python3.14/site-packages/gunicorn/workers/base.py", line 137, in init_process
2026-09-15T08:19:16.399974Z [webservice-55f4987668-kdjj5] [job]     self.load_wsgi()
2026-09-15T08:19:16.399982Z [webservice-55f4987668-kdjj5] [job]     ~~~~~~~~~~~~~~^^
2026-09-15T08:19:16.399990Z [webservice-55f4987668-kdjj5] [job]   File "/layers/heroku_python/venv/lib/python3.14/site-packages/gunicorn/workers/base.py", line 149, in load_wsgi
2026-09-15T08:19:16.399997Z [webservice-55f4987668-kdjj5] [job]     self.wsgi = self.app.wsgi()
2026-09-15T08:19:16.400004Z [webservice-55f4987668-kdjj5] [job]                 ~~~~~~~~~~~~~^^
2026-09-15T08:19:16.400012Z [webservice-55f4987668-kdjj5] [job]   File "/layers/heroku_python/venv/lib/python3.14/site-packages/gunicorn/app/base.py", line 66, in wsgi
2026-09-15T08:19:16.400019Z [webservice-55f4987668-kdjj5] [job]     self.callable = self.load()
2026-09-15T08:19:16.400027Z [webservice-55f4987668-kdjj5] [job]                     ~~~~~~~~~^^
2026-09-15T08:19:16.400035Z [webservice-55f4987668-kdjj5] [job]   File "/layers/heroku_python/venv/lib/python3.14/site-packages/gunicorn/app/wsgiapp.py", line 57, in load
2026-09-15T08:19:16.400042Z [webservice-55f4987668-kdjj5] [job]     return self.load_wsgiapp()
2026-09-15T08:19:16.400050Z [webservice-55f4987668-kdjj5] [job]            ~~~~~~~~~~~~~~~~~^^
2026-09-15T08:19:16.400057Z [webservice-55f4987668-kdjj5] [job]   File "/layers/heroku_python/venv/lib/python3.14/site-packages/gunicorn/app/wsgiapp.py", line 47, in load_wsgiapp
2026-09-15T08:19:16.400065Z [webservice-55f4987668-kdjj5] [job]     return util.import_app(self.app_uri)
2026-09-15T08:19:16.400072Z [webservice-55f4987668-kdjj5] [job]            ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
2026-09-15T08:19:16.400080Z [webservice-55f4987668-kdjj5] [job]   File "/layers/heroku_python/venv/lib/python3.14/site-packages/gunicorn/util.py", line 420, in import_app
2026-09-15T08:19:16.400087Z [webservice-55f4987668-kdjj5] [job]     mod = importlib.import_module(module)
2026-09-15T08:19:16.400095Z [webservice-55f4987668-kdjj5] [job]   File "/layers/heroku_python/python/lib/python3.14/importlib/__init__.py", line 88, in import_module
2026-09-15T08:19:16.400103Z [webservice-55f4987668-kdjj5] [job]     return _bootstrap._gcd_import(name[level:], package, level)
2026-09-15T08:19:16.400110Z [webservice-55f4987668-kdjj5] [job]            ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-09-15T08:19:16.400118Z [webservice-55f4987668-kdjj5] [job]   File "<frozen importlib._bootstrap>", line 1406, in _gcd_import
2026-09-15T08:19:16.400125Z [webservice-55f4987668-kdjj5] [job]   File "<frozen importlib._bootstrap>", line 1371, in _find_and_load
2026-09-15T08:19:16.400133Z [webservice-55f4987668-kdjj5] [job]   File "<frozen importlib._bootstrap>", line 1342, in _find_and_load_unlocked
2026-09-15T08:19:16.400140Z [webservice-55f4987668-kdjj5] [job]   File "<frozen importlib._bootstrap>", line 938, in _load_unlocked
2026-09-15T08:19:16.400148Z [webservice-55f4987668-kdjj5] [job]   File "<frozen importlib._bootstrap_external>", line 755, in exec_module
2026-09-15T08:19:16.400155Z [webservice-55f4987668-kdjj5] [job]   File "<frozen importlib._bootstrap_external>", line 893, in get_code
2026-09-15T08:19:16.400163Z [webservice-55f4987668-kdjj5] [job]   File "<frozen importlib._bootstrap_external>", line 823, in source_to_code
2026-09-15T08:19:16.400170Z [webservice-55f4987668-kdjj5] [job]   File "<frozen importlib._bootstrap>", line 491, in _call_with_frames_removed
2026-09-15T08:19:16.400178Z [webservice-55f4987668-kdjj5] [job]   File "/workspace/app.py", line 5
2026-09-15T08:19:16.400185Z [webservice-55f4987668-kdjj5] [job]     app = Flask(**name**)
2026-09-15T08:19:16.400193Z [webservice-55f4987668-kdjj5] [job]                         ^
2026-09-15T08:19:16.400200Z [webservice-55f4987668-kdjj5] [job] SyntaxError: invalid syntax
2026-09-15T08:19:16.400234Z [webservice-55f4987668-kdjj5] [job] invalid syntax (app.py, line 5)
2026-09-15T08:19:16.400242Z [webservice-55f4987668-kdjj5] [job] [2026-09-15 08:19:16 +0000] [13] [INFO] Worker exiting (pid: 13)
2026-09-15T08:19:16.430303Z [webservice-55f4987668-kdjj5] [job] [2026-09-15 08:19:16 +0000] [1] [ERROR] Control server error: [Errno 13] Permission denied: '/.gunicorn'
2026-09-15T08:19:16.438686Z [webservice-55f4987668-kdjj5] [job] [2026-09-15 08:19:16 +0000] [1] [ERROR] Worker (pid:13) exited with code 3.
2026-09-15T08:19:16.438785Z [webservice-55f4987668-kdjj5] [job] [2026-09-15 08:19:16 +0000] [1] [ERROR] Shutting down: Master
2026-09-15T08:19:16.438799Z [webservice-55f4987668-kdjj5] [job] [2026-09-15 08:19:16 +0000] [1] [ERROR] Reason: Worker failed to boot.

Fixed.

I've noticed that webservices take a bit more time to build(ie show status:successful) than jobs.

Is this expected or is it from my end?

Published here: https://wikitech.wikimedia.org/wiki/Help:Toolforge/My_first_Python_tool

In T437899#12321201, @komla wrote:

Fixed.

I've noticed that webservices take a bit more time to build(ie show status:successful) than jobs.

Is this expected or is it from my end?

Can you elaborate? What do you mean by webservices? (jobs with publish, like the one you are using in the tutorial, or the ones created with toolforge webservice ...)

The build process has happened already by the time they try to run both jobs and webservices, the steps are:

• Build the image (if needed, same process for toolforge jobs or toolforge webservice)
• Start the job or webservice that uses that built image

Yes, jobs with publish like this one.

Perhaps, I'm just perceiving it even though it may not be the case.

I'll test a bit more.

@komla the code snippets here https://wikitech.wikimedia.org/wiki/Help:Toolforge/My_first_Python_tool are still malformed, did you move the tutorial somewhere else?

Do you mean they are like this

Instead of this:

I'm using the <syntaxhighlight lang="python">

I will use the code sample template then.

This is how that link show for me:

I see. Interesting!

Let me use the code sample template instead of the syntaxhighlight tag.

You just have to do the same you just did with the other code:
https://wikitech.wikimedia.org/w/index.php?title=Help%3AToolforge%2FMy_first_Python_tool&diff=2457349&oldid=2457324

I've changed that specific code block.

Please confirm before I proceed with the others.

In T437899#12324072, @komla wrote:

I've changed that specific code block.

Please confirm before I proceed with the others.

Nope (did not work), you have to manually format it, like you did in https://wikitech.wikimedia.org/w/index.php?title=Help%3AToolforge%2FMy_first_Python_tool&diff=2457349&oldid=2457324

That's weird. That previous change are indentation changes.

This is what I'm seeing now:

Does the code block still show as black background?

I've added the indentation to the 'html' in the return statement

In T437899#12324099, @komla wrote:

Does the code block still show as black background?

The background is because I chose dark theme, that's not an issue, the indentation is

You can test it out if it works yourself by copy-pasting the code when creating the tool.

Yeah, it does. No syntax errors.

komla@EDOR-MACHINE:~/src/wmf/___tests$ python3 -m py_compile app.py
komla@EDOR-MACHINE:~/src/wmf/___tests$

https://wikitech.wikimedia.org/wiki/Help:Toolforge/My_first_Python_tool still shows wrong for me:

I don't have much time right now, so will take me a bit to re-review, feel free to ask someone else to give it a look if they have time.

Thanks @dcaro

All this while, I was focused on the longer app.py code block not this shorter version.

It's formatted now.

Looks better, a few other notes:

I would not say 'components service configuration', just tool configuration.

The command is missing half of it:

curl 'https://gitlab.wikimedia.org/toolforge-repos/sample-python-ptd-app/-/raw/main/toolforge.yaml?ref_type=heads' | toolforge components config create

Okay, cool.

I will add the brief set up directly in the tutorial.

Thanks, why is it collapsed?

This is using the old process:

The new process is just an include, see https://wikitech.wikimedia.org/wiki/Help:Toolforge/Deploy_your_tool#Triggering_a_deployment_from_your_CI_runner

There's also a typo here (Name -> Names)

@dcaro, I just feel it is tidier when collapsed because the text on the page is getting long.

The new process is just an include, see https://wikitech.wikimedia.org/wiki/Help:Toolforge/Deploy_your_tool#Triggering_a_deployment_from_your_CI_runner

I see.
Let me try that and update.

Done.

I've fixed the typo and used the new include directive for .gitlab-ci.yml

In T437899#12356141, @komla wrote:

@dcaro, I just feel it is tidier when collapsed because the text on the page is getting long.

Up to you of course, I think this depends on what you want to communicate. As I see it if we are focusing on the push to deploy, I would lean towards collapsing other sections (ex. the big blocks of code when creating the app) and leaving the push to deploy bits on the top level for visibility.

Good idea!

I will do that shortly.