Attempt to provide a Trusty image for Nodepool
Closed, ResolvedPublic

Description

From T125821: Provide a HHVM package for jessie-wikimedia matching version of trusty-wikimedia we now have a hhvm package for Jessie. There are concerned about ICU lib version being different between Trusty and Jessie and @akosiaris suggested to run the tests on the same OS as what is used on production.

This task is about creating a disk image using Trusty and see what happens.

Restricted Application added a subscriber: TerraCodes. · View Herald TranscriptApr 20 2016, 7:37 PM

Change 284900 had a related patch set uploaded (by Hashar):
dib: glue for Ubuntu Trusty imaging

https://gerrit.wikimedia.org/r/284900

I have tried to build an image and puppet apply fails with invalid option: --test

Preparing puppet links...
removed directory: '/etc/puppet/files'
'/etc/puppet/files' -> '/puppet/files'
'/etc/puppet/hieradata' -> '/puppet/hieradata'
removed directory: '/etc/puppet/manifests'
'/etc/puppet/manifests' -> '/puppet/manifests'
removed directory: '/etc/puppet/modules'
'/etc/puppet/modules' -> '/puppet/modules'
removed directory: '/etc/puppet/templates'
'/etc/puppet/templates' -> '/puppet/templates'
Applying puppet manifests ...
Applying puppet manifest /tmp/in_target.d/wikimedia-puppet/bootstrap.pp
Error: Could not parse application options: invalid option: --test
Puppet run failed. Exit code 1
Starting debug shell. Exit to resume building.
At stage after-error

The command being:

puppet apply --test --verbose --modulepath=$PUPPET_DIR/modules

Jessie has puppet 3.7.2 and the apply command comes with:

* --test:
  Enable the most common options used for testing. These are 'verbose',
  'detailed-exitcodes' and 'show_diff'.

Trusty has puppet 3.4.3 which does not have --test.

Later fails with:

  • base::service_unit attempting to start the Xvfb service. That can not work in a chroot. Setting FACTER_initsystem=upstart might fix it, that is how we solved the issue for systemd T129320
  • various dev dependencies that have no equivalent in Trusty. Harnessing them with os_version('debian >= jessie')

Upstart/ missing packages solved in PS8 https://gerrit.wikimedia.org/r/#/c/284900/7..8/dib/puppet/ciimage.pp,cm

Now fails doing the final dist-upgrade:

Performing apt-get dist-upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages will be DOWNGRADED:
  libpcre3
0 upgraded, 0 newly installed, 1 downgraded, 0 to remove and 0 not upgraded.
Need to get 275 kB of archives.
After this operation, 127 kB of additional disk space will be used.
E: There are problems and -y was used without --force-yes
# apt-cache policy libpcre3
libpcre3:
  Installed: 1:8.31-2ubuntu2.3
  Candidate: 1:8.31-2ubuntu2.1+wm1
  Version table:
 *** 1:8.31-2ubuntu2.3 0
        500 http://mirrors.wikimedia.org/ubuntu/ trusty-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     1:8.31-2ubuntu2.2 0
        500 http://mirrors.wikimedia.org/ubuntu/ trusty-security/main amd64 Packages
     1:8.31-2ubuntu2.1+wm1 0
       1001 http://apt.wikimedia.org/wikimedia/ trusty-wikimedia/main amd64 Packages
     1:8.31-2ubuntu2 0
        500 http://mirrors.wikimedia.org/ubuntu/ trusty/main amd64 Packages

Debootstrap got us 1:8.31-2ubuntu2.3 from trusty-updates.

trusty-wikimedia is pinned with an higher priority '1001' and comes with 1:8.31-2ubuntu2.1+wm1.

hashar added a subtask: Restricted Task.Apr 25 2016, 11:53 AM

Change 285173 had a related patch set uploaded (by Hashar):
dib: allow dist-upgrade to downgrade package

https://gerrit.wikimedia.org/r/285173

Filled T133528 about libpcre3. Meanwhile I have hacked dist-upgrade to allow downgrade with APT::Get::force-yes "true" with https://gerrit.wikimedia.org/r/285173

Forged image-trusty-20160425T124552Z.qcow2. Will boot that on labs.

Mentioned in SAL [2016-04-25T13:15:00Z] <hashar> openstack image create --file /home/hashar/image-trusty-20160425T124552Z.qcow2 ci-trusty-wikimedia --disk-format qcow2 --property show=true # T133203

Change 285178 had a related patch set uploaded (by Hashar):
nodepool: pool in 2 TRUSTY instances

https://gerrit.wikimedia.org/r/285178

Change 284900 merged by jenkins-bot:
dib: glue for Ubuntu Trusty imaging

https://gerrit.wikimedia.org/r/284900

Change 285173 merged by jenkins-bot:
dib: allow dist-upgrade to downgrade package

https://gerrit.wikimedia.org/r/285173

Mentioned in SAL [2016-04-25T13:34:19Z] <hashar> Nodepool is attempting to create a Trusty snapshot with name ci-trusty-wikimedia-1461591203 | T133203

The image more or less boot on labs but fails to acquire network connection (ping does not work) :-( There are now specific informations in the Horizon log.

hashar added a comment.EditedApr 25 2016, 1:58 PM

Some boot trouble I had with Jessie back in July 2015 is documented via T105152, specially T105152#1440683 gives some debug steps.

The console log is:

1​[H[J[1;1Htext is deprecated. Use set gfxpayload=vga=normal before linux command
2​instead.
3​text is deprecated. Use set gfxpayload=vga=normal before linux command
4​instead.
5​[ 0.000000] Initializing cgroup subsys cpuset
6​[ 0.000000] Initializing cgroup subsys cpu
7​[ 0.000000] Initializing cgroup subsys cpuacct
8​[ 0.000000] Linux version 3.13.0-85-generic (buildd@lgw01-32) (gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) ) #129-Ubuntu SMP Thu Mar 17 20:50:15 UTC 2016 (Ubuntu 3.13.0-85.129-generic 3.13.11-ckt36)
9​[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-3.13.0-85-generic root=LABEL=cloudimg-rootfs ro nofb nomodeset vga=normal console=tty0 console=ttyS0,115200 no_timer_check nofb nomodeset vga=normal
10​[ 0.000000] KERNEL supported cpus:
11​[ 0.000000] Intel GenuineIntel
12​[ 0.000000] AMD AuthenticAMD
13​[ 0.000000] Centaur CentaurHauls
14​[ 0.000000] e820: BIOS-provided physical RAM map:
15​[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
16​[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
17​[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
18​[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bfffdfff] usable
19​[ 0.000000] BIOS-e820: [mem 0x00000000bfffe000-0x00000000bfffffff] reserved
20​[ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
21​[ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
22​[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000013fffffff] usable
23​[ 0.000000] NX (Execute Disable) protection: active
24​[ 0.000000] SMBIOS 2.4 present.
25​[ 0.000000] Hypervisor detected: KVM
26​[ 0.000000] No AGP bridge found
27​[ 0.000000] e820: last_pfn = 0x140000 max_arch_pfn = 0x400000000
28​[ 0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
29​[ 0.000000] e820: last_pfn = 0xbfffe max_arch_pfn = 0x400000000
30​[ 0.000000] found SMP MP-table at [mem 0x000f0b40-0x000f0b4f] mapped at [ffff8800000f0b40]
31​[ 0.000000] Scanning 1 areas for low memory corruption
32​[ 0.000000] Using GB pages for direct mapping
33​[ 0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff]
34​[ 0.000000] init_memory_mapping: [mem 0x13fe00000-0x13fffffff]
35​[ 0.000000] init_memory_mapping: [mem 0x13c000000-0x13fdfffff]
36​[ 0.000000] init_memory_mapping: [mem 0x100000000-0x13bffffff]
37​[ 0.000000] init_memory_mapping: [mem 0x00100000-0xbfffdfff]
38​[ 0.000000] RAMDISK: [mem 0x35bb2000-0x36dd0fff]
39​[ 0.000000] ACPI: RSDP 00000000000f0950 000014 (v00 BOCHS )
40​[ 0.000000] ACPI: RSDT 00000000bffffbf2 000034 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001)
41​[ 0.000000] ACPI: FACP 00000000bffff1c0 000074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001)
42​[ 0.000000] ACPI: DSDT 00000000bfffe040 001180 (v01 BOCHS BXPCDSDT 00000001 BXPC 00000001)
43​[ 0.000000] ACPI: FACS 00000000bfffe000 000040
44​[ 0.000000] ACPI: SSDT 00000000bffff234 00093E (v01 BOCHS BXPCSSDT 00000001 BXPC 00000001)
45​[ 0.000000] ACPI: APIC 00000000bffffb72 000080 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001)
46​[ 0.000000] ACPI: RSDT 00000000bffffbf2 000034 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001)
47​[ 0.000000] No NUMA configuration found
48​[ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000013fffffff]
49​[ 0.000000] Initmem setup node 0 [mem 0x00000000-0x13fffffff]
50​[ 0.000000] NODE_DATA [mem 0x13fff9000-0x13fffdfff]
51​[ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
52​[ 0.000000] kvm-clock: cpu 0, msr 1:3fff5001, boot clock
53​[ 0.000000] Zone ranges:
54​[ 0.000000] DMA [mem 0x00001000-0x00ffffff]
55​[ 0.000000] DMA32 [mem 0x01000000-0xffffffff]
56​[ 0.000000] Normal [mem 0x100000000-0x13fffffff]
57​[ 0.000000] Movable zone start for each node
58​[ 0.000000] Early memory node ranges
59​[ 0.000000] node 0: [mem 0x00001000-0x0009efff]
60​[ 0.000000] node 0: [mem 0x00100000-0xbfffdfff]
61​[ 0.000000] node 0: [mem 0x100000000-0x13fffffff]
62​[ 0.000000] ACPI: PM-Timer IO Port: 0xb008
63​[ 0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
64​[ 0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
65​[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
66​[ 0.000000] ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
67​[ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
68​[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
69​[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
70​[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
71​[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
72​[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
73​[ 0.000000] Using ACPI (MADT) for SMP configuration information
74​[ 0.000000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
75​[ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
76​[ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff]
77​[ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff]
78​[ 0.000000] PM: Registered nosave memory: [mem 0xbfffe000-0xbfffffff]
79​[ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff]
80​[ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff]
81​[ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff]
82​[ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff]
83​[ 0.000000] e820: [mem 0xc0000000-0xfeffbfff] available for PCI devices
84​[ 0.000000] Booting paravirtualized kernel on KVM
85​[ 0.000000] setup_percpu: NR_CPUS:256 nr_cpumask_bits:256 nr_cpu_ids:2 nr_node_ids:1
86​[ 0.000000] PERCPU: Embedded 27 pages/cpu @ffff88013fc00000 s81536 r8192 d20864 u1048576
87​[ 0.000000] kvm-clock: cpu 0, msr 1:3fff5001, primary cpu clock
88​[ 0.000000] KVM setup async PF for cpu 0
89​[ 0.000000] kvm-stealtime: cpu 0, msr 13fc0d000
90​[ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 1032071
91​[ 0.000000] Policy zone: Normal
92​[ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-3.13.0-85-generic root=LABEL=cloudimg-rootfs ro nofb nomodeset vga=normal console=tty0 console=ttyS0,115200 no_timer_check nofb nomodeset vga=normal
93​[ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
94​[ 0.000000] xsave: enabled xstate_bv 0x7, cntxt size 0x340
95​[ 0.000000] Checking aperture...
96​[ 0.000000] No AGP bridge found
97​[ 0.000000] Memory: 4026876K/4193904K available (7426K kernel code, 1146K rwdata, 3416K rodata, 1336K init, 1448K bss, 167028K reserved)
98​[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
99​[ 0.000000] Hierarchical RCU implementation.
100​[ 0.000000] RCU dyntick-idle grace-period acceleration is enabled.
101​[ 0.000000] RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=2.
102​[ 0.000000] Offload RCU callbacks from all CPUs
103​[ 0.000000] Offload RCU callbacks from CPUs: 0-1.
104​[ 0.000000] NR_IRQS:16640 nr_irqs:512 16
105​[ 0.000000] Console: colour VGA+ 80x25
106​[ 0.000000] console [tty0] enabled
107​[ 0.000000] console [ttyS0] enabled
108​[ 0.000000] allocated 16777216 bytes of page_cgroup
109​[ 0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups
110​[ 0.000000] tsc: Detected 2693.248 MHz processor
111​[ 0.008000] Calibrating delay loop (skipped) preset value.. 5386.49 BogoMIPS (lpj=10772992)
112​[ 0.008000] pid_max: default: 32768 minimum: 301
113​[ 0.008000] Security Framework initialized
114​[ 0.008028] AppArmor: AppArmor initialized
115​[ 0.009454] Yama: becoming mindful.
116​[ 0.010939] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
117​[ 0.014226] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
118​[ 0.016411] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes)
119​[ 0.017598] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes)
120​[ 0.018945] Initializing cgroup subsys memory
121​[ 0.020010] Initializing cgroup subsys devices
122​[ 0.020830] Initializing cgroup subsys freezer
123​[ 0.022038] Initializing cgroup subsys blkio
124​[ 0.022836] Initializing cgroup subsys perf_event
125​[ 0.023689] Initializing cgroup subsys hugetlb
126​[ 0.024942] mce: CPU supports 10 MCE banks
127​[ 0.025771] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
128​[ 0.025771] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0
129​[ 0.025771] tlb_flushall_shift: 5
130​[ 0.028332] Freeing SMP alternatives memory: 32K (ffffffff81e6e000 - ffffffff81e76000)
131​[ 0.034606] ACPI: Core revision 20131115
132​[ 0.036020] ACPI: All ACPI Tables successfully acquired
133​[ 0.037241] ftrace: allocating 28612 entries in 112 pages
134​[ 0.048346] Enabling x2apic
135​[ 0.049040] Enabled x2apic
136​[ 0.049890] Switched APIC routing to physical x2apic.
137​[ 0.052668] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
138​[ 0.054333] smpboot: CPU0: Intel Xeon E312xx (Sandy Bridge) (fam: 06, model: 2a, stepping: 01)
139​[ 0.056032] Performance Events: unsupported p6 CPU model 42 no PMU driver, software events only.
140​[ 0.059614] NMI watchdog: disabled (cpu0): hardware events not enabled
141​[ 0.060114] x86: Booting SMP configuration:
142​[ 0.061053] .... node #0, CPUs: #1[ 0.059596] kvm-clock: cpu 1, msr 1:3fff5041, secondary cpu clock
143
144​[ 0.076075] x86: Booted up 1 node, 2 CPUs
145​[ 0.076057] KVM setup async PF for cpu 1
146​[ 0.076057] kvm-stealtime: cpu 1, msr 13fd0d000
147​[ 0.078681] smpboot: Total of 2 processors activated (10772.99 BogoMIPS)
148​[ 0.080494] devtmpfs: initialized
149​[ 0.082832] EVM: security.selinux
150​[ 0.084009] EVM: security.SMACK64
151​[ 0.084669] EVM: security.ima
152​[ 0.085281] EVM: security.capability
153​[ 0.086532] pinctrl core: initialized pinctrl subsystem
154​[ 0.088086] regulator-dummy: no parameters
155​[ 0.088926] RTC time: 13:48:59, date: 04/25/16
156​[ 0.090053] NET: Registered protocol family 16
157​[ 0.091629] cpuidle: using governor ladder
158​[ 0.092005] cpuidle: using governor menu
159​[ 0.092894] ACPI: bus type PCI registered
160​[ 0.093624] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
161​[ 0.095773] PCI: Using configuration type 1 for base access
162​[ 0.097364] bio: create slab <bio-0> at 0
163​[ 0.097364] ACPI: Added _OSI(Module Device)
164​[ 0.097364] ACPI: Added _OSI(Processor Device)
165​[ 0.100006] ACPI: Added _OSI(3.0 _SCP Extensions)
166​[ 0.100811] ACPI: Added _OSI(Processor Aggregator Device)
167​[ 0.103934] ACPI: Interpreter enabled
168​[ 0.104010] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S1_] (20131115/hwxface-580)
169​[ 0.105710] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S2_] (20131115/hwxface-580)
170​[ 0.107402] ACPI: (supports S0 S3 S4 S5)
171​[ 0.108004] ACPI: Using IOAPIC for interrupt routing
172​[ 0.112056] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
173​[ 0.114372] ACPI: No dock devices found.
174​[ 0.118853] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
175​[ 0.119912] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI]
176​[ 0.120015] acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM
177​[ 0.121687] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
178​[ 0.124139] acpiphp: Slot [3] registered
179​[ 0.124902] acpiphp: Slot [4] registered
180​[ 0.125662] acpiphp: Slot [5] registered
181​[ 0.126418] acpiphp: Slot [6] registered
182​[ 0.128034] acpiphp: Slot [7] registered
183​[ 0.128898] acpiphp: Slot [8] registered
184​[ 0.129660] acpiphp: Slot [9] registered
185​[ 0.130429] acpiphp: Slot [10] registered
186​[ 0.131195] acpiphp: Slot [11] registered
187​[ 0.131961] acpiphp: Slot [12] registered
188​[ 0.132023] acpiphp: Slot [13] registered
189​[ 0.132875] acpiphp: Slot [14] registered
190​[ 0.133652] acpiphp: Slot [15] registered
191​[ 0.134425] acpiphp: Slot [16] registered
192​[ 0.135201] acpiphp: Slot [17] registered
193​[ 0.135961] acpiphp: Slot [18] registered
194​[ 0.136024] acpiphp: Slot [19] registered
195​[ 0.137146] acpiphp: Slot [20] registered
196​[ 0.137977] acpiphp: Slot [21] registered
197​[ 0.138752] acpiphp: Slot [22] registered
198​[ 0.139533] acpiphp: Slot [23] registered
199​[ 0.140024] acpiphp: Slot [24] registered
200​[ 0.141148] acpiphp: Slot [25] registered
201​[ 0.141905] acpiphp: Slot [26] registered
202​[ 0.142673] acpiphp: Slot [27] registered
203​[ 0.143439] acpiphp: Slot [28] registered
204​[ 0.144023] acpiphp: Slot [29] registered
205​[ 0.144809] acpiphp: Slot [30] registered
206​[ 0.145587] acpiphp: Slot [31] registered
207​[ 0.146364] PCI host bridge to bus 0000:00
208​[ 0.147222] pci_bus 0000:00: root bus resource [bus 00-ff]
209​[ 0.148006] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7]
210​[ 0.149988] pci_bus 0000:00: root bus resource [io 0x0d00-0xadff]
211​[ 0.151013] pci_bus 0000:00: root bus resource [io 0xae0f-0xaeff]
212​[ 0.152005] pci_bus 0000:00: root bus resource [io 0xaf20-0xafdf]
213​[ 0.153041] pci_bus 0000:00: root bus resource [io 0xafe4-0xffff]
214​[ 0.154056] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
215​[ 0.155166] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff]
216​[ 0.168275] pci 0000:00:01.3: quirk: [io 0xb000-0xb03f] claimed by PIIX4 ACPI
217​[ 0.169628] pci 0000:00:01.3: quirk: [io 0xb100-0xb10f] claimed by PIIX4 SMB
218​[ 0.208294] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
219​[ 0.209854] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
220​[ 0.211520] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
221​[ 0.212812] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
222​[ 0.214230] ACPI: PCI Interrupt Link [LNKS] (IRQs *9)
223​[ 0.215723] ACPI: Enabled 16 GPEs in block 00 to 0F
224​[ 0.216195] vgaarb: setting as boot device: PCI:0000:00:02.0
225​[ 0.217834] vgaarb: device added: PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none
226​[ 0.220028] vgaarb: loaded
227​[ 0.221030] vgaarb: bridge control possible 0000:00:02.0
228​[ 0.223129] SCSI subsystem initialized
229​[ 0.224164] ACPI: bus type USB registered
230​[ 0.224839] usbcore: registered new interface driver usbfs
231​[ 0.227003] usbcore: registered new interface driver hub
232​[ 0.228040] usbcore: registered new device driver usb
233​[ 0.228961] PCI: Using ACPI for IRQ routing
234​[ 0.229074] NetLabel: Initializing
235​[ 0.229725] NetLabel: domain hash size = 128
236​[ 0.230482] NetLabel: protocols = UNLABELED CIPSOv4
237​[ 0.232020] NetLabel: unlabeled traffic allowed by default
238​[ 0.232983] Switched to clocksource kvm-clock
239​[ 0.239860] AppArmor: AppArmor Filesystem Enabled
240​[ 0.240825] pnp: PnP ACPI init
241​[ 0.241469] ACPI: bus type PNP registered
242​[ 0.242825] pnp: PnP ACPI: found 7 devices
243​[ 0.243901] ACPI: bus type PNP unregistered
244​[ 0.254996] NET: Registered protocol family 2
245​[ 0.256375] TCP established hash table entries: 32768 (order: 6, 262144 bytes)
246​[ 0.257755] TCP bind hash table entries: 32768 (order: 7, 524288 bytes)
247​[ 0.260370] TCP: Hash tables configured (established 32768 bind 32768)
248​[ 0.261461] TCP: reno registered
249​[ 0.262105] UDP hash table entries: 2048 (order: 4, 65536 bytes)
250​[ 0.263250] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes)
251​[ 0.265021] NET: Registered protocol family 1
252​[ 0.265825] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
253​[ 0.266814] pci 0000:00:01.0: PIIX3: Enabling Passive Release
254​[ 0.268163] pci 0000:00:01.0: Activating ISA DMA hang workarounds
255​[ 0.269470] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
256​[ 0.271157] Trying to unpack rootfs image as initramfs...
257​[ 0.588954] Freeing initrd memory: 18556K (ffff880035bb2000 - ffff880036dd1000)
258​[ 0.590400] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
259​[ 0.591459] software IO TLB [mem 0xbbffe000-0xbfffe000] (64MB) mapped at [ffff8800bbffe000-ffff8800bfffdfff]
260​[ 0.593453] microcode: CPU0 sig=0x206a1, pf=0x1, revision=0x1
261​[ 0.594501] microcode: CPU1 sig=0x206a1, pf=0x1, revision=0x1
262​[ 0.596931] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
263​[ 0.598403] Scanning for low memory corruption every 60 seconds
264​[ 0.600165] Initialise system trusted keyring
265​[ 0.600988] audit: initializing netlink socket (disabled)
266​[ 0.601906] type=2000 audit(1461592141.019:1): initialized
267​[ 0.628496] HugeTLB registered 2 MB page size, pre-allocated 0 pages
268​[ 0.630649] zbud: loaded
269​[ 0.632322] VFS: Disk quotas dquot_6.5.2
270​[ 0.633077] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
271​[ 0.634616] fuse init (API version 7.22)
272​[ 0.635745] msgmni has been set to 7901
273​[ 0.636523] Key type big_key registered
274​[ 0.637977] Key type asymmetric registered
275​[ 0.638857] Asymmetric key parser 'x509' registered
276​[ 0.640182] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
277​[ 0.641658] io scheduler noop registered
278​[ 0.642433] io scheduler deadline registered (default)
279​[ 0.645361] io scheduler cfq registered
280​[ 0.646176] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
281​[ 0.647126] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
282​[ 0.648589] ipmi message handler version 39.2
283​[ 0.649438] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
284​[ 0.650757] ACPI: Power Button [PWRF]
285​[ 0.651620] GHES: HEST is not enabled!
286​[ 0.653298] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 10
287​[ 0.657111] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
288​[ 0.659103] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
289​[ 0.689213] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
290​[ 0.721466] 00:05: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A
291​[ 0.723766] Linux agpgart interface v0.103
292​[ 0.725567] brd: module loaded
293​[ 0.726699] loop: module loaded
294​[ 0.727707] blk-mq: CPU -> queue map
295​[ 0.728400] CPU 0 -> Queue 0
296​[ 0.729006] CPU 1 -> Queue 0
297​[ 0.735171] vda: vda1
298​[ 0.739194] scsi0 : ata_piix
299​[ 0.740680] scsi1 : ata_piix
300​[ 0.741701] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc0a0 irq 14
301​[ 0.742896] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc0a8 irq 15
302​[ 0.744834] libphy: Fixed MDIO Bus: probed
303​[ 0.745943] tun: Universal TUN/TAP device driver, 1.6
304​[ 0.746886] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
305​[ 0.750647] PPP generic driver version 2.4.2
306​[ 0.753025] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
307​[ 0.754321] ehci-pci: EHCI PCI platform driver
308​[ 0.755707] ehci-platform: EHCI generic platform driver
309​[ 0.757640] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
310​[ 0.758688] ohci-pci: OHCI PCI platform driver
311​[ 0.759829] ohci-platform: OHCI generic platform driver
312​[ 0.760826] uhci_hcd: USB Universal Host Controller Interface driver
313​[ 0.762963] uhci_hcd 0000:00:01.2: UHCI Host Controller
314​[ 0.764266] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1
315​[ 0.765723] uhci_hcd 0000:00:01.2: irq 11, io base 0x0000c040
316​[ 0.766806] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001
317​[ 0.768666] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
318​[ 0.770660] usb usb1: Product: UHCI Host Controller
319​[ 0.771825] usb usb1: Manufacturer: Linux 3.13.0-85-generic uhci_hcd
320​[ 0.773166] usb usb1: SerialNumber: 0000:00:01.2
321​[ 0.774462] hub 1-0:1.0: USB hub found
322​[ 0.776575] hub 1-0:1.0: 2 ports detected
323​[ 0.777487] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
324​[ 0.779968] serio: i8042 KBD port at 0x60,0x64 irq 1
325​[ 0.780841] serio: i8042 AUX port at 0x60,0x64 irq 12
326​[ 0.781930] mousedev: PS/2 mouse device common for all mice
327​[ 0.783189] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1
328​[ 0.785748] rtc_cmos 00:00: RTC can wake from S4
329​[ 0.787107] rtc_cmos 00:00: rtc core: registered rtc_cmos as rtc0
330​[ 0.789322] rtc_cmos 00:00: alarms up to one day, 114 bytes nvram
331​[ 0.790470] device-mapper: uevent: version 1.0.3
332​[ 0.792068] device-mapper: ioctl: 4.27.0-ioctl (2013-10-30) initialised: dm-devel@redhat.com
333​[ 0.793694] ledtrig-cpu: registered to indicate activity on CPUs
334​[ 0.794936] TCP: cubic registered
335​[ 0.802542] NET: Registered protocol family 10
336​[ 0.804709] NET: Registered protocol family 17
337​[ 0.805564] Key type dns_resolver registered
338​[ 0.806560] Loading compiled-in X.509 certificates
339​[ 0.808245] Loaded X.509 cert 'Magrathea: Glacier signing key: c633e9bfa6ca49d53d2eb5256a35877d04f164f8'
340​[ 0.809906] registered taskstats version 1
341​[ 0.815654] Key type trusted registered
342​[ 0.820876] Key type encrypted registered
343​[ 0.821659] AppArmor: AppArmor sha1 policy hashing enabled
344​[ 0.822613] IMA: No TPM chip found, activating TPM-bypass!
345​[ 0.826554] regulator-dummy: disabling
346​[ 0.827407] Magic number: 12:33:838
347​[ 0.828335] rtc_cmos 00:00: setting system clock to 2016-04-25 13:49:00 UTC (1461592140)
348​[ 0.829927] BIOS EDD facility v0.16 2004-Jun-25, 0 devices found
349​[ 0.830952] EDD information not available.
350​[ 0.901751] Freeing unused kernel memory: 1336K (ffffffff81d20000 - ffffffff81e6e000)
351​[ 0.903377] Write protecting the kernel read-only data: 12288k
352​[ 0.904958] Freeing unused kernel memory: 756K (ffff880001743000 - ffff880001800000)
353​[ 0.908943] Freeing unused kernel memory: 680K (ffff880001b56000 - ffff880001c00000)
354​Loading, please wait...
355​[ 0.923799] systemd-udevd[105]: starting version 204
356​Begin: Loading essential drivers ... done.
357​Begin: Running /scripts/init-premount ... done.
358​Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
359​[ 0.996651] FDC 0 is a S82078B
360​Begin: Running /scripts/local-premount ... done.
361​[ 1.064795] EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts: (null)
362​Begin: Running /scripts/local-bottom ... [ 1.088322] usb 1-1: new full-speed USB device number 2 using uhci_hcd
363​[ 1.134232] vda: vda1
364​GROWROOT: CHANGED: partition=1 start=2048 old: size=5773312 end=5775360 new: size=83881984,end=83884032
365​[ 1.178099] EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts: (null)
366​done.
367​done.
368​Begin: Running /scripts/init-bottom ... done.
369​[ 1.257598] usb 1-1: New USB device found, idVendor=0627, idProduct=0001
370​[ 1.260421] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=5
371​[ 1.262152] usb 1-1: Product: QEMU USB Tablet
372​[ 1.263405] usb 1-1: Manufacturer: QEMU
373​[ 1.264740] usb 1-1: SerialNumber: 42
374​[ 1.525155] random: init urandom read with 15 bits of entropy available
375​[ 1.609398] init: plymouth-upstart-bridge main process (190) terminated with status 1
376​[ 1.612520] init: plymouth-upstart-bridge main process ended, respawning
377​[ 1.623438] init: plymouth-upstart-bridge main process (199) terminated with status 1
378​[ 1.625060] init: plymouth-upstart-bridge main process ended, respawning
379​[ 1.635221] init: plymouth-upstart-bridge main process (202) terminated with status 1
380​[ 1.637546] init: plymouth-upstart-bridge main process ended, respawning
381​[ 2.506085] intel_rapl: no valid rapl domains found in package 0
382​ * Stopping Send an event to indicate plymouth is up[74G[ OK ]
383​ * Starting Mount filesystems on boot[74G[ OK ]
384​ * Starting Populate and link to /run filesystem[74G[ OK ]
385​ * Stopping Populate and link to /run filesystem[74G[ OK ]
386​ * Stopping Track if upstart is running in a container[74G[ OK ]
387​ * Starting Signal sysvinit that virtual filesystems are mounted[74G[ OK ]
388​ * Starting Signal sysvinit that virtual filesystems are mounted[74G[ OK ]
389​ * Starting Bridge udev events into upstart[74G[ OK ]
390​ * Starting Signal sysvinit that remote filesystems are mounted[74G[ OK ]
391​ * Starting Signal sysvinit that the rootfs is mounted[74G[ OK ]
392​ * Starting device node and kernel event manager[74G[ OK ]
393​ * Starting load modules from /etc/modules[74G[ OK ]
394​ * Starting cold plug devices[74G[ OK ]
395​ * Starting log initial device creation[74G[ OK ]
396​ * Stopping load modules from /etc/modules[74G[ OK ]
397​ * Starting Clean /tmp directory[74G[ OK ]
398​ * Stopping Clean /tmp directory[74G[ OK ]
399​ * Starting Signal sysvinit that local filesystems are mounted[74G[ OK ]
400​ * Starting Flush boot log to disk[74G[ OK ]
401​ * Starting flush early job output to logs[74G[ OK ]
402​ * Stopping Mount filesystems on boot[74G[ OK ]
403​ * Stopping Flush boot log to disk[74G[ OK ]
404​ * Stopping flush early job output to logs[74G[ OK ]
405​ * Starting Bridge file events into upstart[74G[ OK ]
406​ * Starting configure network device security[74G[ OK ]
407​ * Starting Mount network filesystems[74G[ OK ]
408​ * Stopping Mount network filesystems[74G[ OK ]
409​ * Starting Mount network filesystems[74G[ OK ]
410​ * Starting configure network device[74G[ OK ]
411​ * Stopping Mount network filesystems[74G[ OK ]
412​ * Starting configure network device security[74G[ OK ]
413​ * Starting Mount network filesystems[74G[ OK ]
414​ * Starting Failsafe Boot Delay[74G[ OK ]
415​ * Starting configure network device[74G[ OK ]
416​ * Stopping Mount network filesystems[74G[ OK ]
417​ * Starting Bridge socket events into upstart[74G[ OK ]
418​ * Stopping cold plug devices[74G[ * Starting save udev log and update rules[74G[ OK ]
419​ * Stopping save udev log and update rules[74G[ OK ]
420​ * Starting userspace bootsplash[74G[ OK ]
421​ * Starting Send an event to indicate plymouth is up[74G[ OK ]
422​ * Stopping Send an event to indicate plymouth is up[74G[ OK ]
423​ * Starting Mount network filesystems[74G[ OK ]
424​ * Stopping Failsafe Boot Delay[74G[ OK ]
425​ * Starting System V initialisation compatibility[74G[ OK ]
426​ * Stopping userspace bootsplash[74G[ OK ]
427​ * Stopping Mount network filesystems[74G[ OK ]
428​ * Starting Mount network filesystems[74G[ OK ]
429​ * Stopping System V initialisation compatibility[74G[ OK ]
430​ * Starting System V runlevel compatibility[74G[ OK ]
431​ * Starting configure virtual network devices[74G[ OK ]
432​ * Stopping Mount network filesystems[74G[ OK ]
433​ * Starting puppet queue [80G [74G[ OK ]
434​ * Zuul Server: /etc/default/zuul is not set to START_DAEMON=1: exiting [80G [74G[[31mfail[39;49m]
435​ * Zuul Merger: /etc/default/zuul-merger is not set to START_DAEMON=1: exiting [80G [74G[[31mfail[39;49m]
436​ * Starting puppet agent [80G [74G[ OK ]
437​ * Stopping System V runlevel compatibility[74G[ OK ]

Change 285184 had a related patch set uploaded (by Hashar):
dib: enable network on Trusty image

https://gerrit.wikimedia.org/r/285184

Change 285184 merged by jenkins-bot:
dib: enable network on Trusty image

https://gerrit.wikimedia.org/r/285184

Needed the simple-init element to be added. Now fails with:

  * Zuul Merger: /etc/default/zuul-merger is not set to START_DAEMON=1: exiting       [80G [74G[[31mfail[39;49m]
+ * Starting Bridge socket events into upstart[74G[ OK ]
+debconf: unable to initialize frontend: Dialog
+debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76.)
+debconf: falling back to frontend: Readline
+debconf: unable to initialize frontend: Readline
+debconf: (This frontend requires a controlling tty.)
+debconf: falling back to frontend: Teletype
+dpkg-query: package 'openssh-server' is not installed and no information is available
+Use dpkg --info (= dpkg-deb --info) to examine archive files,
+and dpkg --contents (= dpkg-deb --contents) to list their contents.
+/usr/sbin/dpkg-reconfigure: openssh-server is not installed
+ * Stopping Run glean to configure network interfaces[74G[ OK ]

dpkg-query: package 'openssh-server' is not installed and no information is available
`

Change 285186 had a related patch set uploaded (by Hashar):
dib: install openssh-server for Trusty

https://gerrit.wikimedia.org/r/285186

Booted a Trusty image:

 * Starting OpenSSH server[74G[ OK ]
ssh start/running, process 687

I managed to ssh to it from labnodepool1001.eqiad.wmnet as jenkins. \O/

Change 285186 merged by jenkins-bot:
dib: install openssh-server for Trusty

https://gerrit.wikimedia.org/r/285186

Change 285194 had a related patch set uploaded (by Hashar):
dib: always install cloud-init

https://gerrit.wikimedia.org/r/285194

cloud-init ends up lacking the EC2 datasource and instead has None, None. Seems to be due to the simple-init element.

Will try again tomorrow morning as time allow.

Have to replace elements ubuntu-minimal, simple-init with simply ubuntu and:

Cloud-init v. 0.7.5 finished at Mon, 25 Apr 2016 18:00:46 +0000. Datasource DataSourceEc2.  Up 18.65 seconds

Change 285218 had a related patch set uploaded (by Hashar):
dib: fix cloud-init on Trusty to use Ec2

https://gerrit.wikimedia.org/r/285218

Change 285218 merged by jenkins-bot:
dib: fix cloud-init on Trusty to use Ec2

https://gerrit.wikimedia.org/r/285218

Summary

Provisioning a Trusty image was more or less straightforward. A bunch of puppet classes have been stripped out since they are meant for Jessie and no jobs will make use of them in Trusty anyway.

I have confirmed Nodepool manage to boot and snapshot an instance out of the Trusty image. The one thing left is apply Nodepool setting to let it know about the Trusty image which is https://gerrit.wikimedia.org/r/#/c/285178/

Change 285194 abandoned by Hashar:
dib: always install cloud-init

Reason:
Replaced 'ubuntu-minimal' by 'ubuntu' and it seems to work fine.

https://gerrit.wikimedia.org/r/285194

Change 285178 merged by Ori.livneh:
nodepool: pool in 2 TRUSTY instances

https://gerrit.wikimedia.org/r/285178

Mentioned in SAL [2016-04-25T19:39:50Z] <hashar> Nodepool now booting 2 trusty instances for CI needs. T133203

hashar closed this task as Resolved.Apr 25 2016, 7:45 PM

Nodepool processed:

2016-04-25 19:41
INFO nodepool.image.build.wmflabs-eqiad.ci-trusty-wikimedia: ./setup_node.sh complete (hostname: ci-trusty-wikimedia-1461613142)
INFO nodepool.SnapshotImageUpdater: Image ci-trusty-wikimedia-1461613142 in wmflabs-eqiad is ready
INFO nodepool.NodePool: Need to launch 2 ci-trusty-wikimedia nodes for gallium.wikimedia.org on wmflabs-eqiad
INFO nodepool.NodeLauncher: Creating server with hostname ci-trusty-wikimedia-83124 in wmflabs-eqiad from image ci-trusty-wikimedia for node id: 83124
INFO nodepool.NodeLauncher: Creating server with hostname ci-trusty-wikimedia-83125 in wmflabs-eqiad from image ci-trusty-wikimedia for node id: 83125

INFO nodepool.NodeLauncher: Node id: 83124 added to jenkins
INFO nodepool.NodeLauncher: Node id: 83125 added to jenkins

They are available in Jenkins and ready to be consumed:

libpcre3 is in theory still a blocker but is worked around by forcing dist-upgrade to downgrade.

Mentioned in SAL [2016-04-25T19:46:19Z] <hashar> Nodepool now has a couple trusty instances intended to experiment with Zend 5.5 / HHVM migration . https://phabricator.wikimedia.org/T133203#2236625

MoritzMuehlenhoff closed subtask Restricted Task as Resolved.Apr 28 2016, 10:14 AM