Update zuul to latest pre 3.0 commit
Closed, DeclinedPublic

Description

We need this https://github.com/openstack-infra/zuul/commit/c7370284742f2ed5c136d0e5fcd08ca2d23c8fe1 patch in preparation for gerrit 2.14 update T156120

On contint1001, the list of Python modules installed as Debian packages versus the ones embedded in our Zuul package via dh_virtualenv:

$ colordiff -U0 --label jessie  <(pip freeze) --label=dh_virtualenv <(/usr/share/python/zuul/bin/pip freeze)
--- jessie
+++ dh_virtualenv
@@ -0,0 +1 @@
+APScheduler==3.0.6
@@ -2,0 +4 @@
+GitPython==2.0.8
@@ -25 +27,3 @@
-gear==0.5.5
+futures==3.0.5
+gear==0.7.0
+gitdb==0.6.4
@@ -28 +32 @@
-lockfile==0.8
+lockfile==0.12.2
@@ -32,0 +37 @@
+prettytable==0.7.2
@@ -38 +43 @@
-python-daemon==1.5.5
+python-daemon==2.0.6
@@ -50 +55,3 @@
-statsd==3.0.1
+smmap==0.9.0
+statsd==2.1.2
+tzlocal==1.2.2
@@ -54,0 +62 @@
+zuul==2.5.0-8-gcbc7f62-wmf4jessie1
There are a very large number of changes, so older changes are hidden. Show Older Changes
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 15 2017, 9:23 PM
Paladox triaged this task as Normal priority.Feb 15 2017, 9:35 PM

Changing to normal as we need it to have a higher priority then the gerrit 2.14 upgrade task. As this will need to be done before then :)

Paladox moved this task from Bugs & stuff to Local hacks on the Gerrit board.Mar 2 2017, 10:11 PM

Tested the patch and fixes gerrit 2.14 support. Without it, zuul broke with gerrit.

This https://github.com/openstack-infra/zuul/commit/01740bd14778cfc2632804522d4548fac57efff8 will require us to update our layout.yaml file but i doint think that will be a big deal. Replacing verified with Verified. (Notice capitals)

I think the 2.14 update will be done after this is complete, we only need to backport https://github.com/openstack-infra/zuul/commit/c7370284742f2ed5c136d0e5fcd08ca2d23c8fe1 then the rest can be done later.

https://github.com/openstack-infra/zuul/commit/c7370284742f2ed5c136d0e5fcd08ca2d23c8fe1 will unblock updating to 2.14. :).

This needs to be done so we can do T140366 and T178385

Change 386194 had a related patch set (by Paladox) published:
[integration/zuul@debian/jessie-wikimedia] Backport fix "Fix change number extraction on new enough Gerrit master"

https://gerrit.wikimedia.org/r/386194

Change 386194 abandoned by Paladox:
Backport fix "Fix change number extraction on new enough Gerrit master"

https://gerrit.wikimedia.org/r/386194

Change 356181 had a related patch set uploaded (by Paladox; owner: Paladox):
[integration/zuul@debian/jessie-wikimedia] Backport fix "Fix change number extraction on new enough Gerrit master"

https://gerrit.wikimedia.org/r/356181

hashar updated the task description. (Show Details)Nov 3 2017, 5:56 PM

@hashar I guess the diff you put in the description means we need to add it those does in the requirement text file?

hashar added a comment.Nov 3 2017, 6:27 PM

They are already albeit not pinned to a specific version. So whenever rebuilding the package, dh_virtualenv ends up including whatever latest version is available. That might break things.

So most probably we would want a patch in git that pin the dependencies in requirements.txt.

@hashar ah i see.

I can create that patch :).

Im using the deb built from my change anyways. So i can test this :).

@hashar i've pinned git here https://gerrit.wikimedia.org/r/#/c/388624/ :)

The rest of the deps should work as they work for me :).

@hashar without pinning the other deps, zuul works for me at least on jessie. We wont need to generate zuul for trusty and precise now :). Only for jessie :).

Change 388624 had a related patch set uploaded (by Paladox; owner: Paladox):
[integration/zuul@debian/jessie-wikimedia] Pin GitPython to >=0.3.3,<2.1.2

https://gerrit.wikimedia.org/r/388624

That is blocking the upgrade of Gerrit from 2.13 to 2.14 which is overdue now. Spoke with @demon about it during the weekly Release-Engineering-Team checkin, I would overhaul the Zuul package and polish it up first week of January.

hashar raised the priority of this task from Normal to High.Dec 18 2017, 5:38 PM

We will want to drop the precise branch too. :)

hashar added a comment.Jan 3 2018, 9:04 PM

I have created the branch patch-queue/debian/jessie-wikimedia which points to the same commit as patch-queue/debian/precise-wikimedia. I guess I rework and reorder all the patches, sending to upstream the ones that are worth potentially merging (I did send https://review.openstack.org/#/c/531012/ for the old T137525)

Change 403153 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/zuul@patch-queue/debian/jessie-wikimedia] WMF: pin requirements to what is deployed

https://gerrit.wikimedia.org/r/403153

Change 403153 merged by Hashar:
[integration/zuul@patch-queue/debian/jessie-wikimedia] WMF: pin requirements to what is deployed

https://gerrit.wikimedia.org/r/403153

Change 403157 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/zuul@debian/jessie-wikimedia] Refresh patches for Jessie

https://gerrit.wikimedia.org/r/403157

Mentioned in SAL (#wikimedia-releng) [2018-01-09T14:53:54Z] <hashar> Change integration/zuul.git HEAD from 'master' to 'patch-queue/debian/jessie-wikimedia' | T158243 T162191

Change 403168 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] debian-glue voting for zuul debian/* branches

https://gerrit.wikimedia.org/r/403168

Change 403168 merged by jenkins-bot:
[integration/config@master] debian-glue voting for zuul debian/* branches

https://gerrit.wikimedia.org/r/403168

Change 403172 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Add debian-glue to Zuul gate-and-submit

https://gerrit.wikimedia.org/r/403172

Change 403172 merged by jenkins-bot:
[integration/config@master] Add debian-glue to Zuul gate-and-submit

https://gerrit.wikimedia.org/r/403172

hashar added a comment.Jan 9 2018, 4:04 PM

Got something build which I am confident to try deploying https://people.wikimedia.org/~hashar/debs/zuul_2.5.0-8-gcbc7f62-wmf6/

Will give it a rough test and then just do it.

It is not using the latest upstream, but just polish up the debianization.

Mentioned in SAL (#wikimedia-operations) [2018-01-10T08:28:13Z] <hashar> contint1001: upgraded Zuul 2.5.0-8-gcbc7f62-wmf4jessie1 .. 2.5.0-8-gcbc7f62-wmf6 | T158243

The repackaged Zuul seems to work fine.

I would need the package on https://people.wikimedia.org/~hashar/debs/zuul_2.5.0-8-gcbc7f62-wmf6/ to be uploaded on apt.wikimedia.org under jessie-wikimedia/thirdparty (which would replace the wmf4 version 2.5.0-8-gcbc7f62-wmf4jessie1 ).

@hashar thanks :).

Should we cc moritzm for uploading the package?

greg added a subscriber: greg.Jan 10 2018, 5:18 PM

@hashar emailed the Ops list for assistance this EU morning.

Ok, thanks.

Dzahn added a subscriber: Dzahn.Jan 11 2018, 10:00 PM

I uploaded the files from https://people.wikimedia.org/~hashar/debs/zuul_2.5.0-8-gcbc7f62-wmf6/

 reprepro -C main include jessie-wikimedia zuul_2.5.0-8-gcbc7f62-wmf6_amd64.changes 
Exporting indices...
Deleting files no longer referenced...
root@install1002:/srv/wikimedia/incoming# reprepro ls zuul
zuul | 2.1.0-60-g1cc37f7-wmf4trusty1 | trusty-wikimedia | amd64, source
zuul |         2.5.0-8-gcbc7f62-wmf6 | jessie-wikimedia | amd64, source
zuul |  2.5.0-8-gcbc7f62-wmf4jessie1 | jessie-wikimedia | amd64, source

Change 403157 merged by jenkins-bot:
[integration/zuul@debian/jessie-wikimedia] Refresh patches for Jessie

https://gerrit.wikimedia.org/r/403157

Change 356181 merged by jenkins-bot:
[integration/zuul@debian/jessie-wikimedia] [DNM] Backport fix "Fix change number extraction on new enough Gerrit master"

https://gerrit.wikimedia.org/r/356181

Change 404263 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/zuul@debian/jessie-wikimedia] Merge tag '2.5.1' into debian/jessie-wikimedia

https://gerrit.wikimedia.org/r/404263

Change 404263 merged by jenkins-bot:
[integration/zuul@debian/jessie-wikimedia] Merge tag '2.5.1' into debian/jessie-wikimedia

https://gerrit.wikimedia.org/r/404263

Mentioned in SAL (#wikimedia-operations) [2018-01-15T10:51:11Z] <hashar> Upgrading zuul to 2.5.1 on contint1001 / contint2001 | T158243

Prods now using a zuul version that supports gerrit 2.14+ :).

Prods now using a zuul version that supports gerrit 2.14+ :).

Then can this task be closed?

We only updated to Zuul 2.5.1, but there’s 2.5.2 and 2.6.0. But this shouldn’t block gerrit’s 2.14 update now :)

greg lowered the priority of this task from High to Normal.Jan 17 2018, 5:21 PM

Lowering priority to Normal as we now unblocked the gerrit upgrade.

Change 388624 abandoned by Paladox:
Pin GitPython to >=0.3.3,<2.1.2

https://gerrit.wikimedia.org/r/388624

I doin't think we can upgrade to the latest zuul version now (we can upgrade to the 2.6 tagged version)

upstream support 3.x now only.

hashar lowered the priority of this task from Normal to Low.Feb 5 2018, 5:26 PM

status update

The main reason for this task was to overhaul the Debian packaging. That has been done.

I still want to update to the latest Zuul 2.5 patch (before 3.0 got merged in) so keeping this task opened.

Discussion regarding Zuul v3 will be in another task.

hashar renamed this task from Update zuul to upstream master to Update zuul to latest pre 3.0 commit.Feb 5 2018, 5:28 PM
hashar lowered the priority of this task from Low to Lowest.Feb 15 2018, 4:53 PM

Lack of upgrade caused T189859

hashar closed this task as Declined.Oct 17 2018, 7:22 AM

Will backport other commits as needed.