To complete the build phase of the new container release pipeline, we'll need a version of Docker that supports multi-stage builds (>= 17.05). Ideally this would be provided in the form of a WMF sanctioned package, but the upgrade to those packages will likely move slow due to interdependencies between Ops k8s work and Toolforge. We may have to make due with the upstream package for now.
Description
Description
Details
Details
Customize query in gerrit
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | None | T170453 FY2017/18 Program 6: Streamlined Service delivery | |||
| Invalid | None | T170480 FY2017/18 Program 6 - Outcome 2: Developers are able to develop and test their applications through a unified pipeline towards production deployment. | |||
| Invalid | None | T170481 FY2017/18 Program 6 - Outcome 2 - Objective 2: Set up a continuous integration and deployment pipeline | |||
| Resolved | None | T157469 Mathoid CI Container Build | |||
| Resolved | dduvall | T175297 Define new Jenkins pipeline for container build phase | |||
| Resolved | hashar | T176267 Upgrade docker on integration-slave-docker-* | |||
| Resolved | thcipriani | T175293 Provision Docker >= 17.05 on contint1001 |
Event Timeline
Comment Actions
In apt.wikimedia.org we have:
| docker.io | 1.6.2~dfsg1-1~bpo8+1 | http://mirrors.wikimedia.org/debian/ jessie-backports/main amd64 Packages |
| docker-engine | 1.12.6-0~debian-jessie | http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/thirdparty amd64 Packages |
docker.io comes from the Debian project.
docker-engine is the package imported from upstream (hence the thirdparty component)
Potentially the required upstream package could be added to a new component (eg: docker17) or one has to figure out how to upgrade Docker on other pieces of the infra relying on it.
Do you need the docker daemon to be running on contint1001?
Comment Actions
Change 377492 had a related patch set uploaded (by Thcipriani; owner: Thcipriani):
[operations/puppet@production] CI: install docker-ce from download.docker.com
Comment Actions
Either the patch I have attached here or adding a new component in apt.wikimedia.org works for me. This is for a production box, so adding a 3rd party repo doesn't seem like standard protocol. The apt repo on carbon is a bit of a black box to me, so I don't know what's needed to add a new package (docker17 or similar) there. @Joe or @akosiaris do you have preferences/guidance here?
Do you need the docker daemon to be running on contint1001?
Yes, that's the plan.
Comment Actions
After some discussions with @Muehlenhoff, I think we can import that under the component thirdparty/ci as is and enable that component just on contint1001. I 'll have a look (I don't expect it to be difficult or causing any issues) and implement it.
Comment Actions
Change 377492 abandoned by Thcipriani:
CI: install docker-ce from download.docker.com
Reason:
new plan to add package to new component
Comment Actions
Change 379182 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/puppet@production] Add thirdparty/ci component to jessie and stretch
Comment Actions
Change 379183 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/puppet@production] Enable thirdparty/ci on role::ci::slave
Comment Actions
Change 379182 merged by Alexandros Kosiaris:
[operations/puppet@production] Add thirdparty/ci component to jessie and stretch
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-21T10:26:33Z] <akosiaris> upload docker-ce_17.06.2~ce-0~debian_amd64.deb to apt.wikimedia.org jessie-wikimedia/thirdparty/ci T175293
Comment Actions
Change 379183 merged by Alexandros Kosiaris:
[operations/puppet@production] Enable thirdparty/ci on role::ci::slave
Comment Actions
Change 379510 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/puppet@production] Install docker-ce on role::ci::slave hosts
Comment Actions
Change 379510 merged by Alexandros Kosiaris:
[operations/puppet@production] Install docker-ce on role::ci::slave hosts
Comment Actions
contint1001:~$ apt-cache policy docker-ce
docker-ce:
Installed: 17.06.2~ce-0~debian
Candidate: 17.06.2~ce-0~debian
Version table:
*** 17.06.2~ce-0~debian 0
1001 http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/thirdparty/ci amd64 Packages
100 /var/lib/dpkg/statuscontint1001:~$ docker --version Docker version 17.06.2-ce, build cec0b72
\O/
We would need to rethink the disk partition slightly, but that can be done later / in another task.