train-deploy-notes Jenkins job fails in conjunction with branch.py
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• brennen
	Jan 21 2020, 7:35 PM

Description

While deploying T233864, noted that the train-deploy-notes job currently fails:

Per @mmodell, this is a known issue with branching by way of branch.py. Additionally we seem to have deprecated a config file that the script relies on.

I created MediaWiki_1.35/wmf.16/Changelog manually using:

12:33:00 brennen@metaphor:~/release/make-deploy-notes (master m0/u1) ❂ python3 makedeploynotes.py 1.35.0-wmf.15 1.35.0-wmf.16

...after moving release/make-wmf-branch/config.json-deprecated to release/make-wmf-branch/config.json and deleting comments atop the file.

Details

Subject	Repo	Branch	Lines +/-
train-deploy-notes: Get rid of regex anchor	integration/config	master	+1 -1
train-deploy-notes: Skip initialization of mediawiki/core submodules	integration/config	master	+23 -9
train-deploy-notes: Run when wgVersion is changed from -alpha	integration/config	master	+48 -18

Customize query in gerrit

Related Objects

Mentioned In: T196516: Automate updating deployment notes
Mentioned Here: T233864: 1.35.0-wmf.16 deployment blockers

Event Timeline

• brennen created this task.Jan 21 2020, 7:35 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 21 2020, 7:35 PM

• brennen moved this task from Backlog to Logs/Train on the User-brennen board.Jan 21 2020, 7:36 PM

Assigning to @mmodell for the moment since he's been driving forward train automation. He and I will likely pair here since the deploy-notes job is a beast we paired to create in the first place.

Change 571849 had a related patch set uploaded (by Dduvall; owner: Dduvall):
[integration/config@master] train-deploy-notes: Run when wgVersion is changed from -alpha

https://gerrit.wikimedia.org/r/571849

gerritbot added a project: Patch-For-Review.Feb 13 2020, 1:48 AM

Change 571849 merged by jenkins-bot:
[integration/config@master] train-deploy-notes: Run when wgVersion is changed from -alpha

https://gerrit.wikimedia.org/r/571849

Maintenance_bot removed a project: Patch-For-Review.Feb 14 2020, 6:10 PM

Change 572496 had a related patch set uploaded (by Dduvall; owner: Dduvall):
[integration/config@master] train-deploy-notes: Skip initialization of mediawiki/core submodules

https://gerrit.wikimedia.org/r/572496

gerritbot added a project: Patch-For-Review.Feb 16 2020, 6:42 PM

Change 572496 merged by jenkins-bot:
[integration/config@master] train-deploy-notes: Skip initialization of mediawiki/core submodules

https://gerrit.wikimedia.org/r/572496

Looks like the job filter introduced to zuul/layout.yaml in https://gerrit.wikimedia.org/r/c/integration/config/+/571849 that was meant to limit scheduling of the job for only changes to the includes/DefaultSettings.php file on wmf/* branches is not working.

From the zuul debug logs:

2020-02-18 18:23:37,379 DEBUG zuul.IndependentPipelineManager: Starting queue processor: postmerge
2020-02-18 18:23:37,380 DEBUG zuul.IndependentPipelineManager: Checking for changes needed by <Change 0x7fae44aaa590 572931,1>:
2020-02-18 18:23:37,380 DEBUG zuul.IndependentPipelineManager:   No changes needed
2020-02-18 18:23:37,380 DEBUG zuul.IndependentPipelineManager: Preparing ref for: <Change 0x7fae44aaa590 572931,1>
2020-02-18 18:23:37,380 INFO zuul.IndependentPipelineManager: Change <Change 0x7fae44aaa590 572931,1> depends on changes []
2020-02-18 18:23:37,380 DEBUG zuul.MergeClient: Submitting job <gear.Job 0x7fae2c2d28d0 handle: None name: merger:merge unique: 8e2b885cf1ad4c92b1c903203806de24> with data {'items': [{'oldrev': None, 'newrev': None, 'refspec': u'refs/changes/31/572931/1', 'merge_mode': 2, 'number': '572931', 'connection_name': 'gerrit', 'project': 'mediawiki/core', 'url': [redacted], 'branch': u'wmf/1.35.0-wmf.20', 'patchset': 1, 'ref': 'Z519642dbd2bc430cbb826041285360dd'}]}
2020-02-18 18:23:37,381 DEBUG zuul.IndependentPipelineManager: Reporting change <Change 0x7fae44aaa590 572931,1>
2020-02-18 18:23:37,382 DEBUG zuul.IndependentPipelineManager: No jobs for change <Change 0x7fae44aaa590 572931,1>
2020-02-18 18:23:37,382 DEBUG zuul.IndependentPipelineManager: Removing change <Change 0x7fae44aaa590 572931,1> from queue
2020-02-18 18:23:37,382 DEBUG zuul.IndependentPipelineManager: Finished queue processor: postmerge (changed: True)

Maintenance_bot removed a project: Patch-For-Review.Feb 18 2020, 9:11 PM

Stealing this from @mmodell since I broke the job (or hit a Zuul bug, rather) with the recent Zuul layout changes.

hashar mentioned this in T196516: Automate updating deployment notes.Mar 3 2020, 8:16 PM

With @mmodell's latest change: https://gerrit.wikimedia.org/r/575575 I can now trigger the job:

[thcipriani@contint1001 ~]$ zuul enqueue --trigger gerrit --pipeline post --project mediawiki/core --change 576439,1

Caused this build to trigger: https://integration.wikimedia.org/ci/job/train-deploy-notes/4190/console

Of course that build is failing :\

While I can run the clone operation locally on contint1001, I can't run the same operation in docker container:

[thcipriani@contint1001 ~]$ docker run --rm -it --user root --entrypoint /bin/bash docker-registry.wikimedia.org/releng/ci-src-setup-simple:0.2.1
...
root@95015c10eb0d:/# nc -vz contint1001.wikimedia.org -w 1 80                                                                                          
nc: connect to contint1001.wikimedia.org port 80 (tcp) timed out: Operation now in progress                                                            
nc: connect to contint1001.wikimedia.org port 80 (tcp) failed: Cannot assign requested address

In T243330#5939220, @thcipriani wrote:

While I can run the clone operation locally on contint1001, I can't run the same operation in docker container:

[thcipriani@contint1001 ~]$ docker run --rm -it --user root --entrypoint /bin/bash docker-registry.wikimedia.org/releng/ci-src-setup-simple:0.2.1
...
root@95015c10eb0d:/# nc -vz contint1001.wikimedia.org -w 1 80                                                                                          
nc: connect to contint1001.wikimedia.org port 80 (tcp) timed out: Operation now in progress                                                            
nc: connect to contint1001.wikimedia.org port 80 (tcp) failed: Cannot assign requested address

Can containers in CI run with --network=host ?

Change 582111 had a related patch set uploaded (by Thcipriani; owner: Thcipriani):
[integration/config@master] train-deploy-notes: Get rid of regex anchor

https://gerrit.wikimedia.org/r/582111

gerritbot added a project: Patch-For-Review.Mar 20 2020, 4:43 PM

Change 582111 merged by jenkins-bot:
[integration/config@master] train-deploy-notes: Get rid of regex anchor

https://gerrit.wikimedia.org/r/582111

Mentioned in SAL (#wikimedia-releng) [2020-03-20T22:22:31Z] <James_F> Zuul: Unanchor the train-deploy-notes job regex filter 582111 T243330

Maintenance_bot removed a project: Patch-For-Review.Mar 20 2020, 11:10 PM

Jdforrester-WMF edited projects, added Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)); removed Release-Engineering-Team-TODO (2020-01 to 2020-03 (Q3)).Apr 1 2020, 3:04 PM

Jdforrester-WMF moved this task from INBOX to Maintenance on the Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)) board.

dduvall removed dduvall as the assignee of this task.May 26 2020, 5:44 PM

Works again \o/ https://www.mediawiki.org/w/index.php?title=MediaWiki_1.35/wmf.34/Changelog&action=history

train-deploy-notes Jenkins job fails in conjunction with branch.pyClosed, ResolvedPublicActions

Description

Details

Related Objects

Event Timeline

train-deploy-notes Jenkins job fails in conjunction with branch.py
Closed, ResolvedPublic
Actions