Page MenuHomePhabricator

[builds-api] return image digest
Closed, ResolvedPublicFeature

Description

Feature summary (what you would like to be able to do and where):

Currently the builds-api returns the image that was published e.g.

{
  "build": {
    "repository": "https://github.com/cluebotng/external-grafana-alloy.git",
    "ref": "refs/tags/v0.1.7",
    "use_latest_versions": true
  },
  "run": {
    "command": "run-alloy",
    "cpu": "0.5",
    "emails": null,
    "filelog": null,
    "filelog_stderr": null,
    "filelog_stdout": null,
    "memory": "0.5Gi",
    "mount": null,
    "port": 8118,
    "replicas": null,
    "health_check_script": null,
    "health_check_http": "/health"
  },
  "component_type": "continuous"
}

I would like to know exactly which image was published so it can be retrieved via tag and/or digest (see https://github.com/opencontainers/image-spec/blob/main/descriptor.md#digests).

Use case(s) (list the steps that you performed to discover that problem, and describe the actual underlying problem which you want to solve. Do not describe only a solution):

With the tag always set to latest and no digest provided it is impossible to assert that a retrieved image (job execution, component configuration, local test) was produced from a build.

There is also the case that multiple builds can overwrite the same image, which when used from scripts/jobs-cli/components-api ends is all sorts of potential conflicts and confusion.

There is also currently workarounds such as the restart logic in components-api to handle a changed image is not changed as far as e.g. jobs-api is concerned, because the image configuration is identical (x-ref T403321).

Benefits (why should this be implemented?):

  • Enhanced security (specific, known, output can be re-used and checked for e.g. sbom contents)
  • Reduce consumer logic (no workarounds to infer a build changing, it changed or didn't)
  • Easy mapping of which images are in-use (e.g. jobs-api -> builds-api -> harbor <> kubernetes)

Details

Related Changes in GitLab:
TitleReferenceAuthorSource BranchDest Branch
images - expand name parsing to support digestsrepos/cloud/toolforge/jobs-api!229damianfeature/expand-image-name-parsingmain
Use the image name as provided by builds-apirepos/cloud/toolforge/components-api!143damianfeature/use-image-uri-from-build-apimain
Customize query in GitLab

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

This is already parsed out of the report generated by pack and shoved into the logs by ./deployment/chart/scripts/results.sh

Tekton supports results, so that logic can probably be expanded to emit into e.g. results.message.path, which can then be exposed back to builds-api.

fgiunchedi triaged this task as Medium priority.Sep 1 2025, 2:31 PM

I'll try and have a look at getting this done once T403321 is merged.

https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/134 - use the value returned from builds-api, rather than constructing it locally.

https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-api/-/merge_requests/144 - fetch the results emitted from results.sh and include them in the API response

The results are already exposed to the TaskRun, so the above finds the TaskRun for the given PipelineRun and yanks the values out of the object.

This means the existing builds will pick up this logic as soon as it goes live and there is no need for backwards compatibility.

For the consumer 192.168.5.15/tool-tf-test/cronjob:latest and 192.168.5.15/tool-tf-test/cronjob:latest@sha256:5f6c57dce00bf19d7763590bf004ba6319ec8caad27ece47d68f56d3b8ecae0f are functionality the same (actually the same image), with image_name being returned standalone already.

Components-api change is not tied to the builds-api behaviour change, so can get merged separately (destination_image is already returned, so moving to consume it is a noop).

group_203_bot_f4d95069bb2675e4ce1fff090c1c1620 opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1009

builds-api: bump to 0.0.202-20251023122338-895f8556

Looks good to me - thanks for the extended effort on getting this deployed @dcaro

tools.cluebotng-monitoring@tools-bastion-15:~$ toolforge build show
Build ID: cluebotng-monitoring-buildpacks-pipelinerun-b7fqr
Start Time: 2025-09-16T14:29:01Z
End Time: 2025-09-16T14:31:24Z
Status: ok
Message: Tasks Completed: 1 (Failed: 0, Cancelled 0), Skipped: 0
Parameters:
    Source URL: https://github.com/cluebotng/monitoring.git
    Ref: refs/tags/v2.6.5
    Envvars: N/A
Use latest versions: True
Destination Image: tools-harbor.wmcloud.org/tool-cluebotng-monitoring/prometheus:latest@sha256:d281c16aa8bdb1487638a8acb28901125ae16da88134ded1e208619aae90436a
DamianZaremba claimed this task.
DamianZaremba reassigned this task from DamianZaremba to dcaro.