[builds-builder] Cache .m2 folder (local maven repository) between builds
Open, LowPublic
Actions

Assigned To

None

Authored By

	Don-vip
	Nov 1 2023, 8:45 PM

Description

Affected tool: spacemedia

I tried to do

After a successful first build (spacemedia-buildpacks-pipelinerun-56fz5) that took 7 minutes, I tried to start a new one to check that the next builds are faster.
As per documentation the .m2 folder (local maven repository) should be cached between builds, so the new build shouldn't spend ~6 minutes to download all dependencies again.

And this happened

The new build (spacemedia-buildpacks-pipelinerun-b4p75) redownloaded all dependencies and spent 7 minutes again:

Build ID: spacemedia-buildpacks-pipelinerun-b4p75
Start Time: 2023-11-01T20:14:54Z
End Time: 2023-11-01T20:21:42Z
Status: ok
Message: Tasks Completed: 1 (Failed: 0, Cancelled 0), Skipped: 0
Parameters:
    Source URL: https://gitlab.wikimedia.org/toolforge-repos/spacemedia
    Ref: N/A
Destination Image: tools-harbor.wmcloud.org/tool-spacemedia/tool-spacemedia:latest

But I was expecting

Previously downloaded Maven artifacts should not be downloaded again, and the build time should have been around 1 minute max.

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		None	T350307 [builds-builder] Cache .m2 folder (local maven repository) between builds
		Open		None	T350689 [builds-builder] Explore adding caching support

Event Timeline

Don-vip created this task.Nov 1 2023, 8:45 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 1 2023, 8:45 PM

Don-vip updated the task description. (Show Details)Nov 1 2023, 8:46 PM

As per documentation the .m2 folder (local maven repository) should be cached between builds, so the new build shouldn't spend ~6 minutes to download all dependencies again.

Caching builds is a feature specific to Heroku as a platform, not something that Heroku buildpacks come with as a default. Agree it would be a nice feature to have on Toolforge though.

Slst2020 renamed this task from [buildservice] Bug - .m2 folder (local maven repository) is not cached between builds to [buildservice] Cache .m2 folder (local maven repository) between builds.Nov 7 2023, 2:42 PM

Slst2020 added a parent task: T350689: [builds-builder] Explore adding caching support.

Slst2020 changed the subtype of this task from "Bug Report" to "Task".

There is "some" support for caching in tekton too (https://github.com/tektoncd/catalog/blob/main/task/buildpacks-phases/0.2/buildpacks-phases.yaml#L23), but we'll have to figure out how to do it, as it also depends on us having persistent volumes on k8s (that we do not yet) or finding some other solution (caching proxy/etc.).

taavi edited projects, added Toolforge Build Service; removed Toolforge Build Service (Beta release).Dec 12 2023, 7:16 PM

dcaro removed a parent task: T350689: [builds-builder] Explore adding caching support.Mar 5 2024, 10:30 AM

dcaro added a subtask: T350689: [builds-builder] Explore adding caching support.

dcaro triaged this task as Low priority.Mar 5 2024, 10:40 AM

dcaro edited projects, added Toolforge; removed Toolforge Build Service.

dcaro moved this task from Backlog to Ready to be worked on on the Toolforge board.Mar 12 2024, 11:05 AM

dcaro renamed this task from [buildservice] Cache .m2 folder (local maven repository) between builds to [builds-builder] Cache .m2 folder (local maven repository) between builds.Apr 3 2024, 1:28 PM

Don-vip awarded a token.Sun, Jun 2, 12:43 PM

[builds-builder] Cache .m2 folder (local maven repository) between buildsOpen, LowPublicActions