Page MenuHomePhabricator

Install pygments in Shellbox container with pip, not a Debian package
Closed, ResolvedPublicFeature

Description

Currently we install pygments for SyntaxHighlight in the Shellbox container using a Debian package that I manually updated. Unfortunately this means that a root/SRE has to upload any new packaging to apt.wikimedia.org. People have gotten used to getting new pygments versions regularly, which is good though.

I think we're better off just installing it in a virtualenv with pip and adding it to the $PATH. That would allow any deployer to upgrade pygments.

From a security perspective, we should build it from the sdist (pinned to the hash) and not use upstream wheels. That also makes it easier to review the source to make sure it's not obviously malicious (but also, we're running this inside Shellbox).

Tagging ServiceOps new for a +1 on the idea, I can probably handle the implementation.

Event Timeline

From a security perspective, we should build it from the sdist (pinned to the hash) and not use upstream wheels.

As far as I know we do not go to this level of control/review for any other containerized Python service. Pinning to known good versions in a requirements.txt or other Python config management system is the more commonly applied standard as far as I am aware. This is at least how Striker, Toolhub, and developer-portal install their python package dependencies during container creation.

@Legoktm, technically speaking, are we talking about pip install --no-binary pygments ? Not sure this is supported in blubber, at least my cursory reading of https://gitlab.wikimedia.org/repos/releng/blubber/-/blob/main/config/python.go says no, so it might require some upstream changes there.

Change 906133 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[mediawiki/libs/Shellbox@master] syntaxhighlight: install pygments from PyPi

https://gerrit.wikimedia.org/r/906133

bd808 changed the task status from Open to In Progress.Apr 10 2023, 9:15 PM
bd808 claimed this task.
bd808 triaged this task as Medium priority.
bd808 changed the subtype of this task from "Task" to "Feature Request".

Copying some of my comments from the gerrit patch here for wider visibility:

This implementation is less paranoid than Legoktm's initial suggestion of building the package ourselves in the container from an sdist pinned to a hash. Instead this would use PyPi distributed wheels installed via pip. This generally matches the build process of other production Python+blubber containers such as those used by Striker, Toolhub, and Developer portal.

The create_pygmentize_bundle script in SyntaxHighlight_GeSHi appears to functionally do the same work of downloading the built wheel from PyPi before embedding the wheel in a zip bundle for distribution with the extension.

Could we get a +1 from someone in serviceops on the general approach of installing via pip instead of a Debian package? We're still figuring out some of the deployment details but otherwise pretty close to moving forward with this.

Change 908312 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[mediawiki/libs/Shellbox@master] syntaxhighlight: add /srv/app/pygmentize script

https://gerrit.wikimedia.org/r/908312

Could we get a +1 from someone in serviceops on the general approach of installing via pip instead of a Debian package? We're still figuring out some of the deployment details but otherwise pretty close to moving forward with this.

Both @akosiaris and @Joe are out of office this week. Calendar stalking makes me think that we will see one or both of them respond sometime during the week of April 17th. I'm excited to get the new code out to users, but they have been waiting for wikitext support in the SyntaxHighlight extension since at least 2011 so a few more days/weeks should not be too disappointing.

Hi :) Any progress here? Or is there any information when it will be possible to implement new schema?

Could we get a +1 from someone in serviceops on the general approach of installing via pip instead of a Debian package? We're still figuring out some of the deployment details but otherwise pretty close to moving forward with this.

+1ed :-)

Change 908312 merged by jenkins-bot:

[mediawiki/libs/Shellbox@master] syntaxhighlight: add /srv/app/pygmentize script

https://gerrit.wikimedia.org/r/908312

Change 912854 had a related patch set uploaded (by Legoktm; author: Legoktm):

[operations/deployment-charts@master] shellbox-syntaxhighlight: Bump version

https://gerrit.wikimedia.org/r/912854

Change 912854 merged by jenkins-bot:

[operations/deployment-charts@master] shellbox-syntaxhighlight: Bump version

https://gerrit.wikimedia.org/r/912854

Change 912895 had a related patch set uploaded (by Legoktm; author: Legoktm):

[operations/mediawiki-config@master] Point SyntaxHighlight at /srv/app/pygmentize

https://gerrit.wikimedia.org/r/912895

The new /srv/app/pygmentize entrypoint is now in place, but that was also the first Shellbox deployment in about 4 months and it pulled in a few other changes, so I think it would be best to let it sit for a day to reduce the chance of a rollback, and then flip over to the new entrypoint and deploy the new pip-installed version.

Change 912895 merged by jenkins-bot:

[operations/mediawiki-config@master] Point SyntaxHighlight at /srv/app/pygmentize

https://gerrit.wikimedia.org/r/912895

Mentioned in SAL (#wikimedia-operations) [2023-05-01T20:05:34Z] <taavi@deploy1002> Started scap: Backport for [[gerrit:912895|Point SyntaxHighlight at /srv/app/pygmentize (T320848)]]

Mentioned in SAL (#wikimedia-operations) [2023-05-01T20:06:53Z] <taavi@deploy1002> legoktm and taavi: Backport for [[gerrit:912895|Point SyntaxHighlight at /srv/app/pygmentize (T320848)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-05-01T20:13:47Z] <taavi@deploy1002> Finished scap: Backport for [[gerrit:912895|Point SyntaxHighlight at /srv/app/pygmentize (T320848)]] (duration: 08m 12s)

Production wikis are now using the /srv/app/pygmentize entrypoint, but are still ultimately calling Pygments 2.11.2. The remaining step will be to build and deploy the container from https://gerrit.wikimedia.org/r/c/mediawiki/libs/Shellbox/+/906133/

Change 906133 merged by jenkins-bot:

[mediawiki/libs/Shellbox@master] syntaxhighlight: install Pygments 2.15.1 from PyPI

https://gerrit.wikimedia.org/r/906133

Change 914014 had a related patch set uploaded (by Legoktm; author: Legoktm):

[operations/deployment-charts@master] shellbox: Bump to 2023-05-01-213815

https://gerrit.wikimedia.org/r/914014

Change 914014 merged by jenkins-bot:

[operations/deployment-charts@master] shellbox: Bump to 2023-05-01-213815

https://gerrit.wikimedia.org/r/914014