Page MenuHomePhabricator
Create Task
Maniphest T158243

Update zuul to latest pre 3.0 commit
Closed, DeclinedPublic
Actions

Description

We need this https://github.com/openstack-infra/zuul/commit/c7370284742f2ed5c136d0e5fcd08ca2d23c8fe1 patch in preparation for gerrit 2.14 update T156120

On contint1001, the list of Python modules installed as Debian packages versus the ones embedded in our Zuul package via dh_virtualenv:

$ colordiff -U0 --label jessie  <(pip freeze) --label=dh_virtualenv <(/usr/share/python/zuul/bin/pip freeze)
--- jessie
+++ dh_virtualenv
@@ -0,0 +1 @@
+APScheduler==3.0.6
@@ -2,0 +4 @@
+GitPython==2.0.8
@@ -25 +27,3 @@
-gear==0.5.5
+futures==3.0.5
+gear==0.7.0
+gitdb==0.6.4
@@ -28 +32 @@
-lockfile==0.8
+lockfile==0.12.2
@@ -32,0 +37 @@
+prettytable==0.7.2
@@ -38 +43 @@
-python-daemon==1.5.5
+python-daemon==2.0.6
@@ -50 +55,3 @@
-statsd==3.0.1
+smmap==0.9.0
+statsd==2.1.2
+tzlocal==1.2.2
@@ -54,0 +62 @@
+zuul==2.5.0-8-gcbc7f62-wmf4jessie1

Details

SubjectRepoBranchLines +/-
Pin GitPython to >=0.3.3,<2.1.2integration/zuuldebian/jessie-wikimedia+30 -0
Merge tag '2.5.1' into debian/jessie-wikimediaintegration/zuuldebian/jessie-wikimedia+31 -6
[DNM] Backport fix "Fix change number extraction on new enough Gerrit master"integration/zuuldebian/jessie-wikimedia+53 -0
Refresh patches for Jessieintegration/zuuldebian/jessie-wikimedia+106 -336
Add debian-glue to Zuul gate-and-submitintegration/configmaster+2 -0
debian-glue voting for zuul debian/* branchesintegration/configmaster+3 -1
WMF: pin requirements to what is deployedintegration/zuulpatch-queue/debian/jessie-wikimedia+9 -4
Backport fix "Fix change number extraction on new enough Gerrit master"integration/zuuldebian/jessie-wikimedia+0 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 15 2017, 9:23 PM
Paladox added a parent task: T156120: Update gerrit to 2.14.6.Feb 15 2017, 9:24 PM

This task blocks T156120

Paladox triaged this task as Medium priority.Feb 15 2017, 9:35 PM

Changing to normal as we need it to have a higher priority then the gerrit 2.14 upgrade task. As this will need to be done before then :)

Paladox moved this task from Bugs & stuff to WMF customizations on the Gerrit board.Mar 2 2017, 10:11 PM
Paladox added a comment.Apr 8 2017, 6:15 PM

@hashar apparently zuul can ddos gerrit. This is fixed in https://github.com/openstack-infra/zuul/commit/12d9aa67e68a833cfbb3b7d5ecddd8c9f5e3fc5f

Paladox added a comment.Apr 8 2017, 6:16 PM

Also see this https://github.com/openstack-infra/zuul/commit/773651ad7bf0fc6adba2357173ffb657d874478a fix for depends on.

greg moved this task from INBOX to Backlog on the Release-Engineering-Team board.May 20 2017, 12:01 PM
greg edited projects, added Release-Engineering-Team (Backlog); removed Release-Engineering-Team.
Paladox added a comment.May 30 2017, 1:50 PM

Tested the patch and fixes gerrit 2.14 support. Without it, zuul broke with gerrit.

Paladox removed a parent task: T156120: Update gerrit to 2.14.6.Jun 14 2017, 4:15 PM
Paladox added a subtask: T156120: Update gerrit to 2.14.6.
Paladox removed a subtask: T156120: Update gerrit to 2.14.6.Aug 31 2017, 9:12 AM
Paladox added a parent task: T156120: Update gerrit to 2.14.6.
Paladox added a comment.Sep 29 2017, 7:48 AM

This https://github.com/openstack-infra/zuul/commit/01740bd14778cfc2632804522d4548fac57efff8 will require us to update our layout.yaml file but i doint think that will be a big deal. Replacing verified with Verified. (Notice capitals)

Paladox added a comment.Oct 23 2017, 9:21 PM

I think the 2.14 update will be done after this is complete, we only need to backport https://github.com/openstack-infra/zuul/commit/c7370284742f2ed5c136d0e5fcd08ca2d23c8fe1 then the rest can be done later.

https://github.com/openstack-infra/zuul/commit/c7370284742f2ed5c136d0e5fcd08ca2d23c8fe1 will unblock updating to 2.14. :).

Paladox added a comment.Oct 24 2017, 2:16 PM

This needs to be done so we can do T140366 and T178385

gerritbot subscribed.Oct 24 2017, 2:30 PM

Change 386194 had a related patch set (by Paladox) published:
[integration/zuul@debian/jessie-wikimedia] Backport fix "Fix change number extraction on new enough Gerrit master"

https://gerrit.wikimedia.org/r/386194

gerritbot added a project: Patch-For-Review.Oct 24 2017, 2:30 PM
gerritbot added a comment.Oct 24 2017, 2:50 PM

Change 386194 abandoned by Paladox:
Backport fix "Fix change number extraction on new enough Gerrit master"

https://gerrit.wikimedia.org/r/386194

gerritbot added a comment.Oct 24 2017, 2:50 PM

Change 356181 had a related patch set uploaded (by Paladox; owner: Paladox):
[integration/zuul@debian/jessie-wikimedia] Backport fix "Fix change number extraction on new enough Gerrit master"

https://gerrit.wikimedia.org/r/356181

hashar updated the task description. (Show Details)Nov 3 2017, 5:56 PM
Paladox added a comment.Nov 3 2017, 6:12 PM

@hashar I guess the diff you put in the description means we need to add it those does in the requirement text file?

hashar added a comment.Nov 3 2017, 6:27 PM

They are already albeit not pinned to a specific version. So whenever rebuilding the package, dh_virtualenv ends up including whatever latest version is available. That might break things.

So most probably we would want a patch in git that pin the dependencies in requirements.txt.

Paladox added a comment.Nov 3 2017, 6:30 PM

@hashar ah i see.

I can create that patch :).

Im using the deb built from my change anyways. So i can test this :).

Paladox added a comment.Nov 3 2017, 10:53 PM

@hashar i've pinned git here https://gerrit.wikimedia.org/r/#/c/388624/ :)

The rest of the deps should work as they work for me :).

Paladox added a comment.Nov 6 2017, 10:35 PM

@hashar without pinning the other deps, zuul works for me at least on jessie. We wont need to generate zuul for trusty and precise now :). Only for jessie :).

gerritbot added a comment.Nov 6 2017, 10:35 PM

Change 388624 had a related patch set uploaded (by Paladox; owner: Paladox):
[integration/zuul@debian/jessie-wikimedia] Pin GitPython to >=0.3.3,<2.1.2

https://gerrit.wikimedia.org/r/388624

greg assigned this task to hashar.Dec 18 2017, 5:36 PM
greg edited projects, added Release-Engineering-Team (Kanban); removed Release-Engineering-Team (Backlog).
hashar added a comment.Dec 18 2017, 5:37 PM

That is blocking the upgrade of Gerrit from 2.13 to 2.14 which is overdue now. Spoke with @demon about it during the weekly Release-Engineering-Team checkin, I would overhaul the Zuul package and polish it up first week of January.

hashar raised the priority of this task from Medium to High.Dec 18 2017, 5:38 PM
Paladox added a comment.Jan 2 2018, 12:14 PM

We will want to drop the precise branch too. :)

hashar added a comment.Jan 3 2018, 9:04 PM

I have created the branch patch-queue/debian/jessie-wikimedia which points to the same commit as patch-queue/debian/precise-wikimedia. I guess I rework and reorder all the patches, sending to upstream the ones that are worth potentially merging (I did send https://review.openstack.org/#/c/531012/ for the old T137525)

Paladox added a parent task: T162787: Upgrade pbr for zuul.Jan 4 2018, 1:02 PM
Paladox removed a parent task: T162787: Upgrade pbr for zuul.
Paladox added a parent task: T162191: test-requirements.txt in ci-config still points to precise deb.
hashar moved this task from Backlog to In-progress on the Release-Engineering-Team (Kanban) board.Jan 8 2018, 6:00 PM
gerritbot added a comment.Jan 9 2018, 1:45 PM

Change 403153 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/zuul@patch-queue/debian/jessie-wikimedia] WMF: pin requirements to what is deployed

https://gerrit.wikimedia.org/r/403153

gerritbot added a comment.Jan 9 2018, 1:45 PM

Change 403153 merged by Hashar:
[integration/zuul@patch-queue/debian/jessie-wikimedia] WMF: pin requirements to what is deployed

https://gerrit.wikimedia.org/r/403153

gerritbot added a comment.Jan 9 2018, 2:05 PM

Change 403157 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/zuul@debian/jessie-wikimedia] Refresh patches for Jessie

https://gerrit.wikimedia.org/r/403157

Stashbot subscribed.Jan 9 2018, 2:53 PM

Mentioned in SAL (#wikimedia-releng) [2018-01-09T14:53:54Z] <hashar> Change integration/zuul.git HEAD from 'master' to 'patch-queue/debian/jessie-wikimedia' | T158243 T162191

Stashbot mentioned this in T162191: test-requirements.txt in ci-config still points to precise deb.Jan 9 2018, 2:53 PM
gerritbot added a comment.Jan 9 2018, 2:59 PM

Change 403168 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] debian-glue voting for zuul debian/* branches

https://gerrit.wikimedia.org/r/403168

gerritbot added a comment.Jan 9 2018, 3:03 PM

Change 403168 merged by jenkins-bot:
[integration/config@master] debian-glue voting for zuul debian/* branches

https://gerrit.wikimedia.org/r/403168

gerritbot added a comment.Jan 9 2018, 3:08 PM

Change 403172 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Add debian-glue to Zuul gate-and-submit

https://gerrit.wikimedia.org/r/403172

gerritbot added a comment.Jan 9 2018, 3:10 PM

Change 403172 merged by jenkins-bot:
[integration/config@master] Add debian-glue to Zuul gate-and-submit

https://gerrit.wikimedia.org/r/403172

hashar added a comment.Jan 9 2018, 4:04 PM

Got something build which I am confident to try deploying https://people.wikimedia.org/~hashar/debs/zuul_2.5.0-8-gcbc7f62-wmf6/

Will give it a rough test and then just do it.

It is not using the latest upstream, but just polish up the debianization.

Stashbot added a comment.Jan 10 2018, 8:28 AM

Mentioned in SAL (#wikimedia-operations) [2018-01-10T08:28:13Z] <hashar> contint1001: upgraded Zuul 2.5.0-8-gcbc7f62-wmf4jessie1 .. 2.5.0-8-gcbc7f62-wmf6 | T158243

hashar added a comment.Jan 10 2018, 8:40 AM

The repackaged Zuul seems to work fine.

I would need the package on https://people.wikimedia.org/~hashar/debs/zuul_2.5.0-8-gcbc7f62-wmf6/ to be uploaded on apt.wikimedia.org under jessie-wikimedia/thirdparty (which would replace the wmf4 version 2.5.0-8-gcbc7f62-wmf4jessie1 ).

Paladox added a comment.Jan 10 2018, 12:51 PM

@hashar thanks :).

Should we cc moritzm for uploading the package?

greg subscribed.Jan 10 2018, 5:18 PM

@hashar emailed the Ops list for assistance this EU morning.

Paladox added a comment.Jan 11 2018, 1:13 PM

Ok, thanks.

Dzahn subscribed.Jan 11 2018, 10:00 PM

I uploaded the files from https://people.wikimedia.org/~hashar/debs/zuul_2.5.0-8-gcbc7f62-wmf6/

 reprepro -C main include jessie-wikimedia zuul_2.5.0-8-gcbc7f62-wmf6_amd64.changes 
Exporting indices...
Deleting files no longer referenced...
root@install1002:/srv/wikimedia/incoming# reprepro ls zuul
zuul | 2.1.0-60-g1cc37f7-wmf4trusty1 | trusty-wikimedia | amd64, source
zuul |         2.5.0-8-gcbc7f62-wmf6 | jessie-wikimedia | amd64, source
zuul |  2.5.0-8-gcbc7f62-wmf4jessie1 | jessie-wikimedia | amd64, source
TerraCodes subscribed.Jan 14 2018, 11:41 AM
gerritbot added a comment.Jan 15 2018, 8:39 AM

Change 403157 merged by jenkins-bot:
[integration/zuul@debian/jessie-wikimedia] Refresh patches for Jessie

https://gerrit.wikimedia.org/r/403157

gerritbot added a comment.Jan 15 2018, 9:12 AM

Change 356181 merged by jenkins-bot:
[integration/zuul@debian/jessie-wikimedia] [DNM] Backport fix "Fix change number extraction on new enough Gerrit master"

https://gerrit.wikimedia.org/r/356181

gerritbot added a comment.Jan 15 2018, 9:43 AM

Change 404263 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/zuul@debian/jessie-wikimedia] Merge tag '2.5.1' into debian/jessie-wikimedia

https://gerrit.wikimedia.org/r/404263

gerritbot added a comment.Jan 15 2018, 10:36 AM

Change 404263 merged by jenkins-bot:
[integration/zuul@debian/jessie-wikimedia] Merge tag '2.5.1' into debian/jessie-wikimedia

https://gerrit.wikimedia.org/r/404263

Stashbot added a comment.Jan 15 2018, 10:51 AM

Mentioned in SAL (#wikimedia-operations) [2018-01-15T10:51:11Z] <hashar> Upgrading zuul to 2.5.1 on contint1001 / contint2001 | T158243

Paladox added a comment.Jan 15 2018, 4:33 PM

Prods now using a zuul version that supports gerrit 2.14+ :).

TerraCodes added a comment.Jan 16 2018, 12:31 AM
In T158243#3900947, @Paladox wrote:

Prods now using a zuul version that supports gerrit 2.14+ :).

Then can this task be closed?

Paladox added a comment.Jan 16 2018, 12:32 AM

We only updated to Zuul 2.5.1, but there’s 2.5.2 and 2.6.0. But this shouldn’t block gerrit’s 2.14 update now :)

TerraCodes removed a parent task: T156120: Update gerrit to 2.14.6.Jan 16 2018, 12:34 AM
TerraCodes mentioned this in T156120: Update gerrit to 2.14.6.
greg lowered the priority of this task from High to Medium.Jan 17 2018, 5:21 PM

Lowering priority to Normal as we now unblocked the gerrit upgrade.

gerritbot added a comment.Jan 21 2018, 1:30 PM

Change 388624 abandoned by Paladox:
Pin GitPython to >=0.3.3,<2.1.2

https://gerrit.wikimedia.org/r/388624

Paladox added a comment.Feb 3 2018, 12:17 PM

I doin't think we can upgrade to the latest zuul version now (we can upgrade to the 2.6 tagged version)

upstream support 3.x now only.

hashar lowered the priority of this task from Medium to Low.Feb 5 2018, 5:26 PM

status update

The main reason for this task was to overhaul the Debian packaging. That has been done.

I still want to update to the latest Zuul 2.5 patch (before 3.0 got merged in) so keeping this task opened.

Discussion regarding Zuul v3 will be in another task.

hashar renamed this task from Update zuul to upstream master to Update zuul to latest pre 3.0 commit.Feb 5 2018, 5:28 PM
hashar lowered the priority of this task from Low to Lowest.Feb 15 2018, 4:53 PM
greg moved this task from In-progress to Backlog on the Release-Engineering-Team (Kanban) board.Feb 21 2018, 5:06 PM
hashar added a comment.Mar 16 2018, 10:09 AM

Lack of upgrade caused T189859

greg edited projects, added Release-Engineering-Team (Next); removed Release-Engineering-Team (Kanban), Patch-For-Review.Jul 13 2018, 10:16 PM
hashar closed this task as Declined.Oct 17 2018, 7:22 AM

Will backport other commits as needed.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL