Toolforge Aptfile still not producing working copy of `ffmpeg`
Closed, ResolvedPublicBUG REPORT
Actions

Assigned To

Authored By

	derenrich
	Aug 28 2024, 10:37 PM

Description

This is a recreation of the closed ticket https://phabricator.wikimedia.org/T365633 It is still failing though with a slightly different error.

Steps to replicate the issue (include links if applicable):

Create a buildpack with an Aptfile that contains ffmpeg
Try to execute ffmpeg
Results in linker failures e.g.

An example can be found at https://gitlab.wikimedia.org/repos/future-audiences/video-answer-tool/-/tree/sprinthackular2024d?ref_type=heads

I have no name!@video-answer-tool-5647866cfb-zsrtj:/layers$ ./fagiani_apt/apt/usr/bin/ffmpeg
./fagiani_apt/apt/usr/bin/ffmpeg: error while loading shared libraries: libavdevice.so.58: cannot open shared object file: No such file or directory

What happens?:

I have no name!@video-answer-tool-5647866cfb-zsrtj:/layers$ ./fagiani_apt/apt/usr/bin/ffmpeg
./fagiani_apt/apt/usr/bin/ffmpeg: error while loading shared libraries: libavdevice.so.58: cannot open shared object file: No such file or directory
I have no name!@video-answer-tool-5647866cfb-zsrtj:/layers$ export LD_LIBRARY_PATH=/layers/fagiani_apt/apt/usr/lib/x86_64-linux-gnu/:/layers/fagiani_apt/apt/usr/lib/x86_64-linux-gnu/oss4-libsalsa/:/layers/fagiani_apt/apt/usr/lib/x86_64-linux-gnu/pulseaudio/:/layers/fagiani_apt/apt/usr/lib/x86_64-linux-gnu/blas/:/layers/fagiani_apt/apt/usr/lib/x86_64-linux-gnu/lapack/:/layers/fagiani_apt/apt/usr/lib/x86_64-linux-gnu/samba/:/layers/fagiani_apt/apt/lib/
I have no name!@video-answer-tool-5647866cfb-zsrtj:/layers$ ./fagiani_apt/apt/usr/bin/ffmpeg
./fagiani_apt/apt/usr/bin/ffmpeg: symbol lookup error: /layers/fagiani_apt/apt/usr/lib/x86_64-linux-gnu/libflite.so.1: undefined symbol: snd_pcm_set_params, version ALSA_0.9

What should have happened instead?:

ffmpeg should work

@dcaro reports it fixed so maybe I'm doing something wrong.

Related Objects

Mentioned In: T389158: Request creation of futureaudiences VPS project
Mentioned Here: T365633: Toolforge Aptfile not producing working copy of `ffmpeg`

Event Timeline

derenrich created this task.Aug 28 2024, 10:37 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 28 2024, 10:37 PM

I think that's because there's no Procfile there.
The ffmpeg fix is bound to the buildpack that parses the Procfile right now (I'll see if I can change it to use the Aptfile one). If you add a Procfile with web: yarn start it should work until I change it.

dcaro claimed this task.Aug 29 2024, 8:43 AM

dcaro triaged this task as High priority.

dcaro edited projects, added Toolforge (Toolforge iteration 14); removed Toolforge.

dcaro moved this task from Next Up to In Progress on the Toolforge (Toolforge iteration 14) board.

dcaro opened https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-builder/-/merge_requests/60

fix_imagemagick: bind to the apt buildpack

That's not the *only* issue though, looking, I suspect that the other packages you have in the Aptfile might be colliding with ffmpeg dependencies (testing)

dcaro merged https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-builder/-/merge_requests/60

fix_imagemagick: bind to the apt buildpack

project_1317_bot_df3177307bed93c3f34e421e26c86e38 opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/493

builds-builder: bump to 0.0.116-20240829144318-0543cb47

In T373565#10102358, @dcaro wrote:

That's not the *only* issue though, looking, I suspect that the other packages you have in the Aptfile might be colliding with ffmpeg dependencies (testing)

Found the issue, so there's two things:

The imagemagick fix is only happening when the Procfile is added (not Aptfile), there's a fix out for review for that
There was a bug on the apt buildpack that was not setting the library path correctly (missing : between two specific entries) - this is fixed already

I'll update when the fix is rolled out (today)

dcaro merged https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/493

builds-builder: bump to 0.0.116-20240829144318-0543cb47

Deployed, can you try building again?

i just build again with the code at

i'm getting the error (when using kubectl to get into the pod)

I have no name!@video-answer-tool-756dbb867c-6dbpp:/layers$ fagiani_apt/apt/usr/bin/ffmpeg
fagiani_apt/apt/usr/bin/ffmpeg: error while loading shared libraries: libavdevice.so.58: cannot open shared object file: No such file or directory

However if I manually set the LD_LIBRARY_PATH then it works

I have no name!@video-answer-tool-756dbb867c-6dbpp:/layers$ export export LD_LIBRARY_PATH=/layers/fagiani_apt/apt/usr/lib/x86_64-linux-gnu/:/layers/fagiani_apt/apt/usr/lib/x86_64-linux-gnu/oss4-libsalsa/:/layers/fagiani_apt/apt/usr/lib/x86_64-linux-gnu/pulseaudio/:/layers/fagiani_apt/apt/usr/lib/x86_64-linux-gnu/blas/:/layers/fagiani_apt/apt/usr/lib/x86_64-linux-gnu/lapack/:/layers/fagiani_apt/apt/usr/lib/x86_64-linux-gnu/samba/:/layers/fagiani_apt/apt/lib/
I have no name!@video-answer-tool-756dbb867c-6dbpp:/layers$ ./fagiani_apt/apt/usr/bin/ffmpeg
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers

I'm presuming that when running under normal conditions that gets set for me so I don't need to worry. If so I'd called this fixed. Thanks!

In T373565#10103917, @derenrich wrote:

i just build again with the code at

i'm getting the error (when using kubectl to get into the pod)

I have no name!@video-answer-tool-756dbb867c-6dbpp:/layers$ fagiani_apt/apt/usr/bin/ffmpeg
fagiani_apt/apt/usr/bin/ffmpeg: error while loading shared libraries: libavdevice.so.58: cannot open shared object file: No such file or directory

However if I manually set the LD_LIBRARY_PATH then it works

I have no name!@video-answer-tool-756dbb867c-6dbpp:/layers$ export export LD_LIBRARY_PATH=/layers/fagiani_apt/apt/usr/lib/x86_64-linux-gnu/:/layers/fagiani_apt/apt/usr/lib/x86_64-linux-gnu/oss4-libsalsa/:/layers/fagiani_apt/apt/usr/lib/x86_64-linux-gnu/pulseaudio/:/layers/fagiani_apt/apt/usr/lib/x86_64-linux-gnu/blas/:/layers/fagiani_apt/apt/usr/lib/x86_64-linux-gnu/lapack/:/layers/fagiani_apt/apt/usr/lib/x86_64-linux-gnu/samba/:/layers/fagiani_apt/apt/lib/
I have no name!@video-answer-tool-756dbb867c-6dbpp:/layers$ ./fagiani_apt/apt/usr/bin/ffmpeg
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers

I'm presuming that when running under normal conditions that gets set for me so I don't need to worry. If so I'd called this fixed. Thanks!

Yep, there's a binary, called launcher that sets the environment up inside buildpack images (you can run launcher bash from within the container to get a shell with the proper environment). It's appended to the command you use when running toolforge jobs (you might have seen it).

Let me know if you see any other weird things!

Maintenance_bot removed a project: Patch-For-Review.Aug 30 2024, 2:30 PM

dcaro moved this task from In Progress to Done on the Toolforge (Toolforge iteration 14) board.Sep 3 2024, 8:07 AM

dcaro mentioned this in T389158: Request creation of futureaudiences VPS project.Mar 18 2025, 5:12 PM

Toolforge Aptfile still not producing working copy of `ffmpeg`Closed, ResolvedPublicBUG REPORTActions

Description

Related Objects

Event Timeline

Toolforge Aptfile still not producing working copy of `ffmpeg`
Closed, ResolvedPublicBUG REPORT
Actions