Page MenuHomePhabricator
Create Task
Maniphest T126246

Provision openjdk-7-jre-headless on Nodepool slaves
Closed, ResolvedPublic
Actions

Description

The Nodepool image lacks a Java installation. Jenkins provision it by itself after it managed to ssh to the CI slave.

We would need to have the package openjdk-7-jre-headless installed, that will in turn let other jobs use java.

Details

SubjectRepoBranchLines +/-
dib: provision openJDKintegration/configmaster+6 -0
dib: add non-free, contribs componentsintegration/configmaster+2 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

hashar created this task.Feb 8 2016, 5:51 PM
hashar claimed this task.
hashar raised the priority of this task from to High.
hashar updated the task description. (Show Details)
hashar added projects: Continuous-Integration-Infrastructure, Continuous-Integration-Scaling.
hashar added subscribers: gerritbot, JanZerebecki, hashar and 5 others.
• Johsthao closed this task as a duplicate of T126250: <spam>.Feb 8 2016, 6:24 PM
matmarex reopened this task as Open.Feb 8 2016, 6:32 PM
hashar added a comment.Feb 8 2016, 10:02 PM

The apt configuration is broken on Nodepool instances, most probably because of duplicate definitions:

The disk image builder provides /etc/apt/sources.list with:

deb http://mirrors.wikimedia.org/debian/ jessie main
deb http://mirrors.wikimedia.org/debian/ jessie-backports main

Puppet ::apt class ships /etc/apt/sources.list.d/debian-backports.list :

deb http://mirrors.wikimedia.org/debian/ jessie-backports main contrib non-free
deb-src http://mirrors.wikimedia.org/debian/ jessie-backports main contrib non-free

Thus jessie-backports is duplicated with conflicting components:

- main
+ main contrib non-free

The list of components is hardcoded both in diskimage-builder and in our apt puppet class :-/

• mobrovac added a comment.Feb 8 2016, 10:14 PM

AFAIK, duplicates are considered only as warnings and do not influence the update/upgrade path.

hashar renamed this task from Provision openjdk-8-jre-headless on Nodepool slaves to Provision openjdk-7-jre-headless on Nodepool slaves.Feb 12 2016, 11:13 AM
hashar updated the task description. (Show Details)
hashar set Security to None.
hashar added a subtask: T126613: Rebuild integration-dev (instance to build images).

The instance I used to build image is gone (T126613) :-(

hashar moved this task from Backlog to In-progress on the Continuous-Integration-Scaling board.Feb 12 2016, 10:34 PM
hashar added a comment.Feb 12 2016, 10:39 PM

The exact message is:

$ apt-cache policy openjdk-7-jre-headless
W: Duplicate sources.list entry http://mirrors.wikimedia.org/debian/ jessie-backports/main amd64 Packages (/var/lib/apt/lists/mirrors.wikimedia.org_debian_dists_jessie-backports_main_binary-amd64_Packages)
W: You may want to run apt-get update to correct these problems
N: Unable to locate package openjdk-7-jre-headless
hashar mentioned this in T119143: Migrate javascript npm CI jobs to Nodepool.Feb 12 2016, 10:51 PM
hashar added a parent task: T126230: Provision pkg-config on Nodepool instances.
hashar mentioned this in T126230: Provision pkg-config on Nodepool instances.Feb 12 2016, 10:54 PM
gerritbot added a comment.Feb 12 2016, 10:57 PM

Change 270433 had a related patch set uploaded (by Hashar):
dib: add non-free, contribs components

https://gerrit.wikimedia.org/r/270433

gerritbot added a project: Patch-For-Review.Feb 12 2016, 10:57 PM
hashar removed a project: Continuous-Integration-Infrastructure.Feb 12 2016, 11:00 PM
hashar added a comment.Feb 12 2016, 11:31 PM

Better:

$ virt-cat -a image-jessie-20160212T225848Z.qcow2 /etc/apt/sources.list
deb http://mirrors.wikimedia.org/debian/ jessie main non-free contrib
deb http://mirrors.wikimedia.org/debian/ jessie-backports main non-free contrib
$ virt-cat -a image-jessie-20160212T225848Z.qcow2 /etc/apt/sources.list.d/debian-backports.list
deb http://mirrors.wikimedia.org/debian/ jessie-backports main contrib non-free
deb-src http://mirrors.wikimedia.org/debian/ jessie-backports main contrib non-free
$ virt-cat -a image-jessie-20160212T225848Z.qcow2 /etc/apt/sources.list.d/wikimedia.list
deb http://apt.wikimedia.org/wikimedia jessie-wikimedia main backports thirdparty
deb-src http://apt.wikimedia.org/wikimedia jessie-wikimedia main backports thirdparty

Gotta fix the order of components just in case. will do in https://gerrit.wikimedia.org/r/270433

• mobrovac added a comment.Feb 12 2016, 11:33 PM

Note that deb http://mirrors.wikimedia.org/debian/ jessie-backports main contrib non-free appears twice, once in /etc/apt/sources.list and another in /etc/apt/sources.list.d/debian-backports.list

hashar added a comment.Feb 12 2016, 11:50 PM

Hooked in the image I build via:

$ virt-rescue -a image-jessie-20160212T225848Z.qcow2 
<linux boot>
$ mount /dev/sda1 /sysroot/
$ chroot /sysroot/

From there:

apt-cache policy
Package files:
 100 /var/lib/dpkg/status
     release a=now
1001 http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/thirdparty amd64 Packages
     release v=8,o=Wikimedia,a=jessie-wikimedia,n=jessie-wikimedia,l=Wikimedia,c=thirdparty
     origin apt.wikimedia.org
1001 http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/backports amd64 Packages
     release v=8,o=Wikimedia,a=jessie-wikimedia,n=jessie-wikimedia,l=Wikimedia,c=backports
     origin apt.wikimedia.org
1001 http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/main amd64 Packages
     release v=8,o=Wikimedia,a=jessie-wikimedia,n=jessie-wikimedia,l=Wikimedia,c=main
     origin apt.wikimedia.org
 100 http://mirrors.wikimedia.org/debian/ jessie-backports/contrib amd64 Packages
     release o=Debian Backports,a=jessie-backports,n=jessie-backports,l=Debian Backports,c=contrib
     origin mirrors.wikimedia.org
 100 http://mirrors.wikimedia.org/debian/ jessie-backports/non-free amd64 Packages
     release o=Debian Backports,a=jessie-backports,n=jessie-backports,l=Debian Backports,c=non-free
     origin mirrors.wikimedia.org
 100 http://mirrors.wikimedia.org/debian/ jessie-backports/main amd64 Packages
     release o=Debian Backports,a=jessie-backports,n=jessie-backports,l=Debian Backports,c=main
     origin mirrors.wikimedia.org
 500 http://mirrors.wikimedia.org/debian/ jessie/contrib amd64 Packages
     release v=8.3,o=Debian,a=stable,n=jessie,l=Debian,c=contrib
     origin mirrors.wikimedia.org
 500 http://mirrors.wikimedia.org/debian/ jessie/non-free amd64 Packages
     release v=8.3,o=Debian,a=stable,n=jessie,l=Debian,c=non-free
     origin mirrors.wikimedia.org
 500 http://mirrors.wikimedia.org/debian/ jessie/main amd64 Packages
     release v=8.3,o=Debian,a=stable,n=jessie,l=Debian,c=main
     origin mirrors.wikimedia.org
Pinned packages:
W: Duplicate sources.list entry http://mirrors.wikimedia.org/debian/ jessie-backports/main amd64 Packages (/var/lib/apt/lists/mirrors.wikimedia.org_debian_dists_jessie-backports_main_binary-amd64_Packages)
W: Duplicate sources.list entry http://mirrors.wikimedia.org/debian/ jessie-backports/contrib amd64 Packages (/var/lib/apt/lists/mirrors.wikimedia.org_debian_dists_jessie-backports_contrib_binary-amd64_Packages)
W: Duplicate sources.list entry http://mirrors.wikimedia.org/debian/ jessie-backports/non-free amd64 Packages (/var/lib/apt/lists/mirrors.wikimedia.org_debian_dists_jessie-backports_non-free_binary-amd64_Packages)
W: You may want to run apt-get update to correct these problems

Still dupes but:

apt-cache madison openjdk-7-jre-headless pkg-config

openjdk-7-jre-headless7u91-2.6.3-1~deb8u1http://mirrors.wikimedia.org/debian/ jessie/main amd64 Packages
pkg-config0.28-1http://mirrors.wikimedia.org/debian/ jessie/main amd64 Packages
gerritbot added a comment.Feb 15 2016, 10:11 AM

Change 270703 had a related patch set uploaded (by Hashar):
dib: provision openJDK

https://gerrit.wikimedia.org/r/270703

hashar added a project: OKR-Work.Feb 15 2016, 10:18 AM
Stashbot subscribed.Feb 15 2016, 10:57 AM

Mentioned in SAL [2016-02-15T10:57:49Z] <hashar> bumped Nodepool disk image to fix apt configuration / include jre and package-config. Refreshing snapshot image. Poke T126246 T126230 T119143

hashar added a comment.Feb 15 2016, 11:11 AM

Jenkins manage to find java now:

[02/15/16 11:08:48] [SSH] Checking java version of java
[02/15/16 11:08:48] [SSH] java -version returned 1.7.0_91.
hashar removed a subtask: T126613: Rebuild integration-dev (instance to build images).Feb 15 2016, 11:12 AM

No more blocked by T126613: Rebuild integration-dev (instance to build images). I have build the image on my local machine.

gerritbot added a comment.Feb 15 2016, 1:44 PM

Change 270703 merged by jenkins-bot:
dib: provision openJDK

https://gerrit.wikimedia.org/r/270703

hashar closed this task as Resolved.Feb 15 2016, 4:29 PM
hashar mentioned this in T120963: Nodepool instances have duplicate entry for jessie-backports/main.Feb 15 2016, 10:33 PM
hashar added a comment.Feb 15 2016, 10:49 PM

Will follow up / clean the configuration for the APT repository via T120963

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits