Page MenuHomePhabricator
Create Task
Maniphest T106548

[Task] create mirror for our composer dependencies
Closed, DeclinedPublic
Actions

Description

Create a mirror for for our composer dependencies. This would be used by CI and possibly also for deployment and build purposes. While the most pressing issue is Github rate limiting us, it has the advantage that our infrastructure would be independent from Github and Packagist (except for pulling new versions when they are added in a Github maintained repositories).

Satis ( https://getcomposer.org/doc/articles/handling-private-packages-with-satis.md ) is the software that can do this.

"When GitHub or BitBucket repositories are mirrored on your local satis" - https://getcomposer.org/doc/articles/handling-private-packages-with-satis.md#downloads seems to imply it supports both mirroring of git repositories and creation of archives.

Related Objects
Search...

Event Timeline

JanZerebecki created this task.Jul 22 2015, 4:58 PM
JanZerebecki raised the priority of this task from to Needs Triage.
JanZerebecki updated the task description. (Show Details)
JanZerebecki added projects: Wikidata, Continuous-Integration-Infrastructure.
JanZerebecki added subscribers: Addshore, JeroenDeDauw, JanZerebecki, Aklapper.
JanZerebecki updated the task description. (Show Details)Jul 22 2015, 6:33 PM
JanZerebecki set Security to None.
Legoktm subscribed.Jul 22 2015, 6:48 PM

Last time I played with satis, it was just a packagist.org mirror, not a mirror of the actual archives. https://toranproxy.com/ looks more like what we need...

Legoktm added a project: Composer.Jul 22 2015, 6:48 PM
JanZerebecki mentioned this in T88453: make building wikidata for deployment independent from external packagist.org.Jul 23 2015, 7:49 PM
hashar subscribed.Jul 27 2015, 2:51 PM

I played a bit with Satis ages ago. IIRC you can list the packages you want to install on the mirror and end up with a local/private composer server.

Feel free to set one up on labs so we can play with it? Can then be moved to production.

Another possibility would be to set up a proxy that would cache packages and/or make sure composer is used with --prefer-dist since the tarball ends up being cached locally and would be hold by the proxy.

Lydia_Pintscher moved this task from incoming to needs discussion or investigation on the Wikidata board.Aug 17 2015, 3:17 PM
JanZerebecki renamed this task from create mirror for for our composer dependencies to create mirror for our composer dependencies.Aug 17 2015, 3:34 PM
daniel triaged this task as Low priority.Sep 10 2015, 3:35 PM
daniel subscribed.

Seems nice to have - until github or packagist go down for a couple of days, then this becomes critical...

JanZerebecki renamed this task from create mirror for our composer dependencies to [Task] create mirror for our composer dependencies.Sep 10 2015, 7:25 PM
JanZerebecki added a subtask: T107840: [Task] try out satis.
hashar added a comment.Sep 10 2015, 7:59 PM

Maybe with Diffusion we have tarballs available for all repos. Potentially we could then have composer to be pointed to it. I am not sure how much loads it would cause there though.

hashar mentioned this in T106519: [Bug] github.com is 403ing downloads from Wikimedia CI during composer update.Oct 11 2016, 9:08 AM
hashar closed this task as Declined.Nov 18 2016, 3:11 PM

There is no resource/lead to move to provide our own repository of packages. We are relying on Github and https://packagist.org/ . MediaWiki extensions are probably usually downloaded on the Extension Distributor.

Maybe one day that idea will surface again, meanwhile there is little need to keep this task open.

thiemowmde closed subtask T107840: [Task] try out satis as Declined.Dec 7 2017, 4:09 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL