The Loki API requires a timeframe to request logs for. The current implementation hardcodes that timeframe as the last 1 hour, but that should be customizable by user, either as a relative time (last X hours?) or as absolute start/end times.
Description
Event Timeline
From my usage of logs it would be useful (on the basis of a time window being explicitly required):
- --since N accepting something like [15m, 2h, 1d] (with a reasonable default, the current 1hour seems fine to keep)
- --since accepting an explicit date time e.g. `--since "2025-08-05 10:00:00" (format should be easy to type, with the timezone always UTC the same as the nodes)
- --until in the same context as 2.
Additional thoughts:
- It would be nice if the API returned a warning if the time window falls outside of the configured retention time (if this is queryable via the loki api).
- I think it would be reasonable for the API to deal with explicit dates, requiring since with an optional until and the CLI handles the conversion of human friendly strings [15m, 2h, 1d] to an absolute date.
- In the context of a one-off job, the logical --since time is the job start time (this is not exposed via the jobs api directly, but can be parsed out of the long status). It would be reasonable to leave this up to the consumer for implementation.
This issue blocks the migration of my tools away from file logs. One hour is very low.
Extremely ugly workaround in case you need to query logs older than 1h, and you're a Toolforge admin:
root@tools-k8s-control-7:~# kubectl exec -it pod/jobs-api-5fc546b54d-nljwc -n jobs-api -- bash root@jobs-api-5fc546b54d-nljwc:/app# apt update && apt install curl root@jobs-api-5fc546b54d-nljwc:/app# curl http://loki-tools-read.loki.svc.tools.local:3100/loki/api/v1/query_range -H "X-Scope-OrgID: tool-admin" --data-urlencode 'query={job="updatetools"}' --data-urlencode 'since=10h' --data-urlencode 'limit=1000' --data-urlencode 'direction=forward' | jq
raymond-ndibe opened https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-cli/-/merge_requests/145
[cli] support start and end params in logs
raymond-ndibe opened https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/267
support start and end params in logs endpoint
raymond-ndibe opened https://gitlab.wikimedia.org/repos/cloud/toolforge/logs-api/-/merge_requests/12
support start and end params in logs endpoints
raymond-ndibe opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1140
logs-api,logs: add logs tests
raymond-ndibe opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1141
logs-api,logs: test start, end params for logs-api and jobs-api log endpoint
dcaro closed https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/267
support start and end params in logs endpoint
raymond-ndibe merged https://gitlab.wikimedia.org/repos/cloud/toolforge/logs-api/-/merge_requests/12
support since and until params in logs endpoints
group_203_bot_f4d95069bb2675e4ce1fff090c1c1620 opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1187
logs-api: bump to 0.0.16-20260330220252-cd8acfa2
raymond-ndibe merged https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1187
logs-api: bump to 0.0.16-20260330220252-cd8acfa2
raymond-ndibe merged https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-cli/-/merge_requests/145
[cli] support since and until params in logs
raymond-ndibe closed https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1141
logs: test since, until params for jobs-api logs
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Running_jobs#Job_logs needs updating for these changes.