Migrate operations/puppet CI job from Jessie to Stretch
Closed, ResolvedPublic

Description

@Joe mentioned it yesterday, the puppet master have been migrated to stretch, it makes sense to have the CI job to use Stretch as well.

A few patches have been made to prepare a CI container:

https://gerrit.wikimedia.org/r/#/c/integration/config/+/470419/
https://gerrit.wikimedia.org/r/#/c/integration/config/+/470574/
https://gerrit.wikimedia.org/r/#/c/integration/config/+/470834/

docker-registry.wikimedia.org/releng/operations-puppet:0.4.2 has Stretch.

hashar created this task.Oct 31 2018, 2:08 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 31 2018, 2:08 PM

Change 470839 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Non voting puppet job using Stretch

https://gerrit.wikimedia.org/r/470839

hashar claimed this task.Oct 31 2018, 2:10 PM

Change 470839 merged by jenkins-bot:
[integration/config@master] Non voting puppet job using Stretch

https://gerrit.wikimedia.org/r/470839

Deployed, the new job uses Stretch and is non voting until it is proven to be working properly :]

Mentioned in SAL (#wikimedia-operations) [2018-10-31T14:17:31Z] <hasharLunch> Adding a Stretch based CI job for operations/puppet (non voting job for now) | T208422

hashar added a subscriber: Bstorm.Nov 7 2018, 9:55 AM

I have retrieved the build status for both jobs using:

ssh contint1001.wikimedia.org grep operations-puppet-tests.*/console debug.log.2018-11-??

Parsed the outcome with:

#!/usr/bin/env python3

import re

with open('puppetjob') as f:
    lines = f.readlines()

prev = None
for line in lines:
    m = re.match('.*(?P<url>https://.*/console).*(?P<result>SUCCESS|FAILURE).*', line)
    if not m:
        print('Error parsing %s' % line)
        continue

    if prev is None:
        prev = m.groupdict()
        continue
    else:
        if prev['result'] != m.groupdict()['result']:
            print('MISMATCH')
            print('{result} {url}'.format_map(prev))
            print('{result} {url}'.format_map(m.groupdict()))
        prev = None

And I have a couple mismatches:

MISMATCH
SUCCESS https://integration.wikimedia.org/ci/job/operations-puppet-tests-jessie-docker/103/console
FAILURE https://integration.wikimedia.org/ci/job/operations-puppet-tests-stretch-docker-non-voting/103/console
MISMATCH
SUCCESS https://integration.wikimedia.org/ci/job/operations-puppet-tests-jessie-docker/105/console
FAILURE https://integration.wikimedia.org/ci/job/operations-puppet-tests-stretch-docker-non-voting/105/console

That is for https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/471020/3 and https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/471020/4

They fail because the patch introduce a tox virtualenv based on puppet 3 but the container does not have it. That is fixed by: T208873

There is no other mismatch so I will promote the stretch based job to be voting and drop the jessie based job.

Change 472117 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Promote puppet Stretch job and rm Jessie one

https://gerrit.wikimedia.org/r/472117

Change 472117 merged by jenkins-bot:
[integration/config@master] Promote puppet Stretch job and rm Jessie one

https://gerrit.wikimedia.org/r/472117

Mentioned in SAL (#wikimedia-operations) [2018-11-07T10:06:46Z] <hashar> CI: switched operations/puppet job to be based on Stretch ( T208422 ) and to add python3 ( T208873 )

hashar closed this task as Resolved.Nov 7 2018, 10:21 AM