Hmmm... I think that there might be some issue when generating the models from the toolforge openapi spec on components-api, as it generated this:

class JobsHttpHealthCheck(BaseModel):
    type: Literal["path"]
    path: str

Found something using restish to try to patch the same way components did:

msg: "Input tag 'path' found using 'health_check_type' | 'type' does not match any of the expected tags: <HealthCheckType.SCRIPT: 'script'>, <HealthCheckType.HTTP: 'http'>"

The jobs api does not really give any extra info:

│ INFO:     127.0.0.1:42416 - "PATCH /v1/tool/tf-test2/jobs/ HTTP/1.0" 422 Unprocessable Entity                                                                                                                                               │

I'm able to reproduce in lima-kilo with:

In T405262#11799849, @aputhin wrote:

Changing to medium prio after refinement. We need a bit more clarity on impact of the use cases for this feature. @dcaro to list the use cases we already identified.

In T422515#11803289, @fgiunchedi wrote:

The other aspect to consider, which was the culprit in this case, is OS_PROJECT_ID vs OS_PROJECT_NAME usage (+OS_PROJECT_DOMAIN_NAME, which is default AFAICT).

I ran a quick audit (P90335) on cookbook invocations on cloudcumin1001 and it seems to me that in the vast majority of cases what we mean is OS_PROJECT_NAME and not OS_PROJECT_ID for wmcs-openstack invocations.

Made a note in https://wikitech.wikimedia.org/wiki/Help:Toolforge/Building_container_images#User_upgrade_notes_for_the_upgrade_started_on_2026-03-31, thanks!

this :)

dcaro@acme$ yq -p toml -o toml '
  .processes |= map(
    .command = ["bash"] + [.args[-1] + " \"$@\"", "--"]
    | .args = []
  )
'  /tmp/metadata.toml

Oop, I think this should do the trick:

dcaro@acme$ yq -p toml -o toml '
  .processes |= map(
    .command = ["bash"] + .args + [" \"$@\"", "--"]
    | .args = []
  )
'  /tmp/metadata.toml 
[[buildpacks]]
id = "heroku/dotnet"
version = "1.0.3"
api = "0.10"
homepage = "https://github.com/heroku/buildpacks-dotnet"
[[buildpacks]]
id = "heroku/procfile"
version = "4.2.1"
api = "0.10"
homepage = "https://github.com/heroku/buildpacks-procfile"
[[processes]]
type = "web"
command = ["bash", "-c", "./autoreport.sh", " \"$@\"", "--"]
direct = true
buildpack-id = "heroku/procfile"
[[processes]]
type = "another"
command = ["bash", "-c", "./another.sh", " \"$@\"", "--"]
direct = true
buildpack-id = "heroku/procfile"

It seems to work, but still trying to find the equivalent to jq '.processes |= map(.command = [.command] + (.args | .[-1] += " \"$@\"") + ["--"] | .args = [])', not there though:

dcaro@acme$ yq -p toml -o toml '
  .processes |= map(
    .command = ["bash"] + (.args[0:-1] + [(.args[-1] + " $@")])
    | .args = []
  )
'  /tmp/metadata.toml 
[[buildpacks]]
id = "heroku/dotnet"
version = "1.0.3"
api = "0.10"
homepage = "https://github.com/heroku/buildpacks-dotnet"
[[buildpacks]]
id = "heroku/procfile"
version = "4.2.1"
api = "0.10"
homepage = "https://github.com/heroku/buildpacks-procfile"
[[processes]]
type = "web"
command = ["bash", "type", "web", "command", "bash", "args", ["-c", "./autoreport.sh"], "direct", true, "buildpack-id", "./autoreport.sh $@"]
direct = true
buildpack-id = "heroku/procfile"
[[processes]]
type = "another"
command = ["bash", "type", "another", "command", "anotherbash", "args", ["-c", "./another.sh"], "direct", true, "buildpack-id", "./another.sh $@"]  <-- it's adding the whole element, not just args
direct = true
buildpack-id = "heroku/procfile"

In T422224#11798221, @dcaro wrote:

In T422224#11797303, @Hawkeye7 wrote:

@dcaro Testing failed. This is not sufficient.

My bots take command line arguments. So for example, with the Procfile
web: bin/publish/Liftwing

I run Liftwing with the command "web BYD"

I get
usage: Liftwing <article>

The command line arguments have not been passed through!!

That is a current limitation yep (T356016: [builds-builder,jobs-api,upstream] Calling nontrivial Procfile commands with arguments results in confusing error (“no such file or directory”)), if you want to pass arguments you'll have to wrap them in shell scripts, looking

Note that you'll be the biggest user ;)

In T422462#11800709, @Addshore wrote:

is a relatively big amount of data

I expect I can verify with a subset.
I can also likely strip out some of the longer / larger parts of the data and not load those
Though I also expect there will be overhead when I load it.
How much disk is there to play with?

In T400917#11800585, @taavi wrote:

https://wikitech.wikimedia.org/wiki/Help:Toolforge/Running_jobs#Job_logs needs updating for these changes.

This is fixed also

This is fixed now, you can pass args to any procfile entry command, for example, with a procfile like:

## Procfile
worker: celery --app sample_complex_app_backend.tasks.celery_app worker --loglevel INFO

+1 from me, is a relatively big amount of data, so if you have a way to verify with less data if elastic works for you it would be appreciated (also, keep into account the number of replicas for your indices, if you don't need HA maybe 1 is enough).

I'll close this, but feel free to reopen if you still have issues.

Sent a patch upstream (https://github.com/heroku/buildpacks-procfile/issues/310), but will probably need to hack-fix it for it to work now.

In T422224#11797303, @Hawkeye7 wrote:

@dcaro Testing failed. This is not sufficient.

My bots take command line arguments. So for example, with the Procfile
web: bin/publish/Liftwing

I run Liftwing with the command "web BYD"

I get
usage: Liftwing <article>

The command line arguments have not been passed through!!

I suspect this has gotten worse with the latest update, the upstream move to using command = ["bash", "-c"] makes passing custom args impossible, looking

I have been trying to trigger this from my own tool, running toolforge components config create config.yaml in a loop for 1000 times, and was unable to get it to fail.

I'll close this for now, but feel free to reopen if you test the MR/change and still have the issue.

In T422384#11793745, @dcaro wrote:

Got a fix for this, waiting for reviews, should be merged soon-ish: https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-builder/-/merge_requests/86

Got a fix for this, waiting for reviews, should be merged soon-ish: https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-builder/-/merge_requests/86

@Hawkeye7 as @bd808 mentions, the new path for the procfile is bin/publish/AutoReport, or the full path /workspace/bin/publish/AutoReport.

In T387141#11791590, @bd808 wrote:

T422384: [builds-builder] incompatibility of fagiani/apt and builder stack "24" is reporting the bug I filed as T394466: [build-service] remove legacy fagiani/apt 0.2.5 builder from `--use-latest-versions` stack which was merged into this task but not actually fixed before the Buildpack environment version upgrade happened.

In T421929#11787825, @Soda wrote:

Got it, will make a note to not use the -f from now on (until -f is fixed)

In T422384#11792774, @magnusmanske wrote:

I have added the project.toml file to both my repo root and the tool root. Same thing.

That would be the future-proof way to go yep (url changed https://wikitech.wikimedia.org/wiki/Help:Toolforge/Building_container_images#heroku/deb-packages_(Ubuntu_2024.04_stack,_current_default). I'll give it a quick look, but if it gives too much trouble I might just disable the old apt buildpack.

I had to manually upgrade cffi by unpinning the deps to python 3.9, so it would get a new enough cffi.

The --use-deprecated-versions flag is available, and will start failing on the 27-04-2026, so all tools should have migrated by then.