Page MenuHomePhabricator

Provision Docker >= 17.05 on contint1001
Closed, ResolvedPublic

Description

To complete the build phase of the new container release pipeline, we'll need a version of Docker that supports multi-stage builds (>= 17.05). Ideally this would be provided in the form of a WMF sanctioned package, but the upgrade to those packages will likely move slow due to interdependencies between Ops k8s work and Toolforge. We may have to make due with the upstream package for now.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 7 2017, 5:54 PM
thcipriani triaged this task as Medium priority.
thcipriani moved this task from Backlog to CI on the Release Pipeline board.
thcipriani moved this task from Backlog to In-progress on the Release-Engineering-Team (Kanban) board.

In apt.wikimedia.org we have:

docker.io1.6.2~dfsg1-1~bpo8+1http://mirrors.wikimedia.org/debian/ jessie-backports/main amd64 Packages
docker-engine1.12.6-0~debian-jessiehttp://apt.wikimedia.org/wikimedia/ jessie-wikimedia/thirdparty amd64 Packages

docker.io comes from the Debian project.
docker-engine is the package imported from upstream (hence the thirdparty component)

Potentially the required upstream package could be added to a new component (eg: docker17) or one has to figure out how to upgrade Docker on other pieces of the infra relying on it.

Do you need the docker daemon to be running on contint1001?

Change 377492 had a related patch set uploaded (by Thcipriani; owner: Thcipriani):
[operations/puppet@production] CI: install docker-ce from download.docker.com

https://gerrit.wikimedia.org/r/377492

Potentially the required upstream package could be added to a new component (eg: docker17) or one has to figure out how to upgrade Docker on other pieces of the infra relying on it.

Either the patch I have attached here or adding a new component in apt.wikimedia.org works for me. This is for a production box, so adding a 3rd party repo doesn't seem like standard protocol. The apt repo on carbon is a bit of a black box to me, so I don't know what's needed to add a new package (docker17 or similar) there. @Joe or @akosiaris do you have preferences/guidance here?

Do you need the docker daemon to be running on contint1001?

Yes, that's the plan.

akosiaris added a subscriber: Muehlenhoff.EditedSep 18 2017, 5:00 PM

Potentially the required upstream package could be added to a new component (eg: docker17) or one has to figure out how to upgrade Docker on other pieces of the infra relying on it.

Either the patch I have attached here or adding a new component in apt.wikimedia.org works for me. This is for a production box, so adding a 3rd party repo doesn't seem like standard protocol. The apt repo on carbon is a bit of a black box to me, so I don't know what's needed to add a new package (docker17 or similar) there. @Joe or @akosiaris do you have preferences/guidance here?

After some discussions with @Muehlenhoff, I think we can import that under the component thirdparty/ci as is and enable that component just on contint1001. I 'll have a look (I don't expect it to be difficult or causing any issues) and implement it.

Change 377492 abandoned by Thcipriani:
CI: install docker-ce from download.docker.com

Reason:
new plan to add package to new component

https://gerrit.wikimedia.org/r/377492

Change 379182 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/puppet@production] Add thirdparty/ci component to jessie and stretch

https://gerrit.wikimedia.org/r/379182

Change 379183 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/puppet@production] Enable thirdparty/ci on role::ci::slave

https://gerrit.wikimedia.org/r/379183

Change 379182 merged by Alexandros Kosiaris:
[operations/puppet@production] Add thirdparty/ci component to jessie and stretch

https://gerrit.wikimedia.org/r/379182

Mentioned in SAL (#wikimedia-operations) [2017-09-21T10:26:33Z] <akosiaris> upload docker-ce_17.06.2~ce-0~debian_amd64.deb to apt.wikimedia.org jessie-wikimedia/thirdparty/ci T175293

Change 379183 merged by Alexandros Kosiaris:
[operations/puppet@production] Enable thirdparty/ci on role::ci::slave

https://gerrit.wikimedia.org/r/379183

Change 379510 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/puppet@production] Install docker-ce on role::ci::slave hosts

https://gerrit.wikimedia.org/r/379510

Change 379510 merged by Alexandros Kosiaris:
[operations/puppet@production] Install docker-ce on role::ci::slave hosts

https://gerrit.wikimedia.org/r/379510

akosiaris closed this task as Resolved.Sep 21 2017, 2:28 PM

And done. Resolving

contint1001:~$ apt-cache policy docker-ce
docker-ce:
  Installed: 17.06.2~ce-0~debian
  Candidate: 17.06.2~ce-0~debian
  Version table:
 *** 17.06.2~ce-0~debian 0
       1001 http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/thirdparty/ci amd64 Packages
        100 /var/lib/dpkg/status
contint1001:~$ docker --version
Docker version 17.06.2-ce, build cec0b72

\O/

We would need to rethink the disk partition slightly, but that can be done later / in another task.