⚓ T249949 Stop using integration/composer and then archive the repo

Subject	Repo	Branch	Lines +/-
Archive integration/composer	integration/config	master	+4 -21
Archive repository	integration/composer	master	+2 -131 K
Upgrade composer to 1.10.22, per CVE-2021-29472	integration/composer	master	+4 K -2 K
delete contint::composer	operations/puppet	production	+0 -22
releases::mediawiki: remove PHP packages	operations/puppet	production	+0 -43

Status	Assigned	Task
Resolved	hashar	T249949 Stop using integration/composer and then archive the repo
Resolved	Reedy	T249464 Replace integration/composer with a composer.phar in CI usage
Resolved	Legoktm	T249525 Use packaged composer
Resolved	taavi	T287900 Switch Toolforge installation of "composer" to use the Debian package
Resolved	aborrero	T277653 Toolforge: add Debian Buster to the grid and eliminate Debian Stretch
Resolved	aborrero	T277866 cloud-init: figure out how to change /etc/hosts from cloud-init/vendordata
Resolved	aborrero	T278232 Toolforge: figure out how to work with the new domain in the grid
Resolved	aborrero	T278748 Toolforge: introduce support for selecting grid queue release
Resolved	aborrero	T282972 "--release is not implemented for --backend=kubernetes" with latest tools-webservice from Git
Resolved	taavi	T288961 /usr/local/bin/webservice:109: DeprecationWarning: dist() and linux_distribution() functions are deprecated in Python 3.5
Resolved	aborrero	T300501 Toolforge grid: uwsgi in buster fails to load python3 venvs
Resolved	taavi	T280037 Toolforge: set up monitoring tooling for stretch deprecation
Duplicate	None	T280252 Toolforge Buster bastion no longer tab completes become command
Resolved	taavi	T284767 Toolforge: migrate cron servers to Debian Buster
Declined	None	T298089 Sort out Mono repositories for the buster grid
Resolved	aborrero	T298948 Toolforge grid deployment/management automation
Declined	None	T300032 spicerack: introduce GridEngine controller
Resolved	aborrero	T301665 Toolforge jobs framework: create documentation on wikitech
Resolved	taavi	T309525 Toolforge: Create a cookbook to decomission a SGE node
Resolved	taavi	T309732 New SGE nodes can't talk to the grid engine master
Resolved	• nskaggs	T309821 Buster webservice grid went BOOM!
Declined	taavi	T309902 Tiny swap on many grid nodes
Resolved	Legoktm	T293055 Switch extdist to Bullseye and composer Debian package

Jdforrester-WMF created this task.Apr 10 2020, 7:39 PM

Restricted Application added subscribers: Liuxinyu970226, Aklapper. · View Herald TranscriptApr 10 2020, 7:39 PM

Jdforrester-WMF added a subtask: T249464: Replace integration/composer with a composer.phar in CI usage.Apr 10 2020, 7:39 PM

Jdforrester-WMF closed subtask T249464: Replace integration/composer with a composer.phar in CI usage as Resolved.Apr 10 2020, 9:06 PM

modules/profile/manifests/releases/mediawiki.pp:    class { '::contint::composer': }

That is for the releases Jenkins which we should overhaul and migrate to a different system, namely using Docker / VM ( https://releases-jenkins.wikimedia.org/ ).

modules/role/manifests/ci/slave/labs.pp:    include contint::composer

I guess we can drop that one and purge the repository from all the Jenkins agents.

modules/phragile/manifests/init.pp that is solely on WMCS.
modules/profile/manifests/toolforge/grid/exec_environ.pp

Maybe those can be ported to wget the composer.phar, verify the checksum and put it under /usr/local/bin ? An alternative is to craft a dummy Debian package to ship the .phar which might be reasonably easy to do. Composer 1.10-1 is packaged in Debian, should be possible to backport it to all the distribution on which we need it.

Jdforrester-WMF moved this task from Untriaged to Backlog on the Continuous-Integration-Infrastructure board.Apr 17 2020, 4:15 PM

For the CI containers we went to use upstream composer.phar.

For deployment of composer on WMCS instances via puppet, it seems to me a Debian package will be a better fit. So marking as being blocked by T249525: Use packaged composer.

hashar added a subtask: T249525: Use packaged composer.May 5 2020, 2:12 PM

hashar changed the status of subtask T249525: Use packaged composer from Stalled to Open.May 5 2020, 2:16 PM

hashar triaged this task as Medium priority.May 5 2020, 3:21 PM

Legoktm changed the status of subtask T249525: Use packaged composer from Open to Stalled.Jun 10 2020, 1:31 AM

Change 607858 had a related patch set uploaded (by Hashar; owner: Dzahn):
[operations/puppet@production] releases::mediawiki: remove PHP packages

https://gerrit.wikimedia.org/r/607858

gerritbot added a project: Patch-For-Review.Jun 26 2020, 7:54 AM

hashar updated the task description. (Show Details)Jun 26 2020, 8:48 AM

Change 607858 merged by Dzahn:
[operations/puppet@production] releases::mediawiki: remove PHP packages

https://gerrit.wikimedia.org/r/c/operations/puppet/ /607858

I have cleaned the releases* hosts.

Maintenance_bot removed a project: Patch-For-Review.Jul 1 2020, 8:20 PM

hashar updated the task description. (Show Details)Nov 30 2020, 2:30 PM

Change 675213 had a related patch set uploaded (by Hashar; author: Dzahn):
[operations/puppet@production] delete contint::composer

https://gerrit.wikimedia.org/r/675213

gerritbot added a project: Patch-For-Review.Mar 27 2021, 7:49 AM

Change 675213 merged by Dzahn:
[operations/puppet@production] delete contint::composer

https://gerrit.wikimedia.org/r/675213

Maintenance_bot removed a project: Patch-For-Review.Mar 30 2021, 6:12 PM

Jdforrester-WMF added a project: Cloud-Services.Apr 30 2021, 1:58 AM

Jdforrester-WMF updated the task description. (Show Details)

Jdforrester-WMF added a subscriber: Legoktm.

@Legoktm, is there a nice way to stop extdist still using this? Same for WMCS folk and the Toolforge usage.

I can manually re-build the repo to 1.10.22 but that won't re-generate the instances' local clones of it, I presume?

Change 683764 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/composer@master] Upgrade composer to 1.10.22, per CVE-2021-29472

https://gerrit.wikimedia.org/r/683764

gerritbot added a project: Patch-For-Review.Apr 30 2021, 2:03 AM

We can switch to the packaged version of composer as part of T277249: Migrate extdist.wmflabs.org to Debian Buster.

Legoktm mentioned this in T277249: Migrate extdist.wmflabs.org to Debian Buster.Apr 30 2021, 2:05 AM

Change 683764 abandoned by Hashar:

[integration/composer@master] Upgrade composer to 1.10.22, per CVE-2021-29472

Reason:

> Specifically crafted URL values allow code to be executed in the HgDriver if hg/Mercurial is installed on the system

That is really for packagist and we don't have mercurial anywhere.

I'd like to avoid upgrading composer cause I have absolutely no idea what kind of side effect it will have for us. So unless an update is really required, I do not want to update composer :)

https://gerrit.wikimedia.org/r/683764

Maintenance_bot removed a project: Patch-For-Review.Jul 29 2021, 3:10 PM

Legoktm mentioned this in T287900: Switch Toolforge installation of "composer" to use the Debian package.Aug 2 2021, 6:03 PM

Legoktm added a subtask: T287900: Switch Toolforge installation of "composer" to use the Debian package.

taavi changed the status of subtask T287900: Switch Toolforge installation of "composer" to use the Debian package from Open to In Progress.Sep 26 2021, 4:05 PM

Mentioned in SAL (#wikimedia-releng) [2021-10-11T15:35:53Z] <hashar> gerrit: marked integration/composer read-only. We are no more using it and the few use cases left are tracked at https://phabricator.wikimedia.org/T249949

hashar edited projects, added Toolforge; removed Cloud-Services.Oct 11 2021, 3:36 PM

hashar changed the status of subtask T249525: Use packaged composer from Stalled to Open.

Toolforge is in progress via T287900 (thank you).

I filed T293055 for extdist which would require new VM based on Bullseye to replace the Stretch ones.

Legoktm closed subtask T249525: Use packaged composer as Resolved.Feb 1 2022, 7:51 AM

Legoktm closed subtask T293055: Switch extdist to Bullseye and composer Debian package as Resolved.Oct 15 2022, 3:43 AM

hashar reopened subtask T293055: Switch extdist to Bullseye and composer Debian package as Open.Nov 3 2022, 2:57 PM

Marking this tracking task stalled. It is pending on Toolforge Docker images (T287900) which are based on Jessie, Stretch, Buster and Bullseye. The first three still require integration/composer.git rather than a Debian package in order to get a recent composer version.

hashar updated the task description. (Show Details)Sep 26 2023, 7:07 AM

taavi closed subtask T287900: Switch Toolforge installation of "composer" to use the Debian package as Resolved.Feb 8 2024, 3:32 PM

hashar changed the task status from Stalled to Open.Feb 8 2024, 3:36 PM

hashar updated the task description. (Show Details)Mar 29 2024, 8:37 AM

hashar added a project: Projects-Cleanup.Mar 29 2024, 8:40 AM

Change #1015518 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] Archive integration/composer

https://gerrit.wikimedia.org/r/1015518

gerritbot added a project: Patch-For-Review.Mar 29 2024, 10:16 AM

Change #1015519 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/composer@master] Archive repository

https://gerrit.wikimedia.org/r/1015519