Page MenuHomePhabricator

wikidata-query-gui-build doesn’t work when latest commit is by dependabot (commit-msg hook adds Change-Id in wrong place)
Closed, ResolvedPublic

Description

Wikidata Query UI build #41 failed:

+ git -C ./src/build commit -m 'Merging from 4d63180156443b59eba6cb477f8b424c16e66e2b' -m 'Bump bootstrap-table from 1.19.0 to 1.19.1

Bumps [bootstrap-table](https://github.com/wenzhixin/bootstrap-table) from 1.19.0 to 1.19.1.
- [Release notes](https://github.com/wenzhixin/bootstrap-table/releases)
- [Changelog](https://github.com/wenzhixin/bootstrap-table/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/wenzhixin/bootstrap-table/compare/1.19.0...1.19.1)

---
updated-dependencies:
- dependency-name: bootstrap-table
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Pull Request: https://github.com/wikimedia/wikidata-query-gui/pull/103'
[production d5dffdd] Merging from 4d63180156443b59eba6cb477f8b424c16e66e2b
 6 files changed, 4 insertions(+), 4 deletions(-)
 delete mode 100644 js/embed.vendor.min.5f1c7180a4d05a87929b.js
 create mode 100644 js/embed.vendor.min.7cb558e46015da96d5ee.js
 delete mode 100644 js/vendor.min.4d04d4141e5811e6ae01.js
 create mode 100644 js/vendor.min.8784f6fc265e46709859.js
+ git -C ./src/build push origin HEAD:refs/for/production%ready
remote: 
remote: Processing changes: refs: 1
remote: Processing changes: refs: 1, done            
remote: ERROR: commit d5dffdd: Change-Id must be in message footer        
remote: 
remote: Hint: run        
remote:   git commit --amend        
remote: and move 'Change-Id: Ixxx..' to the bottom on a separate line        
remote: 
To ssh://gerrit.wikimedia.org:29418/wikidata/query/gui-deploy
 ! [remote rejected] HEAD -> refs/for/production%ready (commit d5dffdd: Change-Id must be in message footer)
error: failed to push some refs to 'ssh://wdqsguibuilder@gerrit.wikimedia.org:29418/wikidata/query/gui-deploy'

The problem is that Gerrit’s commit-msg hook gets confused where the Change-Id should go; if you get the commit-msg hook from Gerrit (like wikidata-query-gui-build does) and create a commit with the same commit message from the output above, the final resulting commit message is:

Bump bootstrap-table from 1.19.0 to 1.19.1

Bumps bootstrap-table from 1.19.0 to 1.19.1.

Change-Id: I031068


updated-dependencies:

  • dependency-name: bootstrap-table dependency-type: direct:production update-type: version-update:semver-patch

...

Signed-off-by: dependabot[bot] <support@github.com>
Pull Request: https://github.com/wikimedia/wikidata-query-gui/pull/103

The Change-Id is in the middle of the message, and Gerrit doesn’t like that.

Event Timeline

The above is with the commit-msg hook from Gerrit 3.3.6 (according to the comment in the file). With an older version of the hook that I have lying around locally – From Gerrit Code Review 2.13.4-13-gc0c5cc4742 – it actually looks better:

Bump bootstrap-table from 1.19.0 to 1.19.1

Bumps bootstrap-table from 1.19.0 to 1.19.1.


updated-dependencies:

  • dependency-name: bootstrap-table dependency-type: direct:production update-type: version-update:semver-patch

...

Signed-off-by: dependabot[bot] <support@github.com>
Pull Request: https://github.com/wikimedia/wikidata-query-gui/pull/103

Change-Id: I884024

The blank line between Pull Request and Change-Id isn’t exactly desirable, but that might just be due to the space in “Pull Request” (rather than “Pull-Request” as I’d expect from a trailer).

Edit: yeah, if you change the message from “Pull Request” to “Pull-Request”, then this version of the commit-msg hook correctly puts all the trailers together (in the order Change-Id, Signed-off-by, Pull-Request). The newest version still gets just as confused, thoguh.

The recommended workaround is to create and merge another commit with git commit --allow-empty, merge that, then trigger another deploy build.

It is not recommended to try a local build, manually adjust the commit message to put the Change-Id in the right place, and upload that (as I did in this change, later reverted). Local builds may include extra files or have other issues due to an unusual environment.

The hook in upstream is at https://gerrit.googlesource.com/gerrit/+/refs/heads/master/resources/com/google/gerrit/server/tools/root/hooks/commit-msg (git log).

The hook adds the Change-Id header using git-interpret-trailers which would be the command being confused by the Depend-a-bot commit message. The reason would the hook inserts the header before the line having --- which must be handled specifically by git-interpret-trailers?

Reproduction:

git version 2.34.1
$ 'echo '---' |git interpret-trailers --trailer "Change-Id: FOO"

Change-Id: FOO
---
$

From the git-interpret-trailers(1) man page:

The group must either be at the end of the message or be the last non-whitespace lines before a line that starts with --- (followed by a space or the end of the line). Such three minus signs start the patch part of the message. See also --no-divider below.

--- is used when show the whole commit which will have the commit message and the patch itself. The patch would show the diff eg:

...
    Some oneline message 

    Signed-off-by: whoever

diff --git a/Documentation/RelNotes/2.36.0.txt b/Documentation/RelNotes/2.36.0.txt
new file mode 100644
index 0000000000..c59e0c80e9
--- /dev/null
+++ b/Documentation/RelNotes/2.36.0.txt

So if you want add a trailer to a commit message to an input representing a full patch (commit + diff), that will work. In the case of dependend-a-bot the --- marker in the middle of the commit message cause git-interpret-trailers to stop processing.

The git command has the option:

--no-divider
Do not treat --- as the end of the commit message.
Use this when you know your input contains just the commit message itself (and not an email or the output of git format-patch).

Which sounds like the use case for the Gerrit commit-msg hook. There is unfortunately no git config setting to enable the option :-\

The git commit adding --no-divider is 1688c9a4894df517241026c7a3848bdc84607986 and has been released in cGit 2.19.2

I guess we can ask GitHub to amend their depend a bot to use four minus signs, they seem to use that to in markdown context to generate a <hr/> and they missed that is specially handled by git interpret-trailer. That is my favorite way to fix it. The alternative is to craft a patch for Gerrit upstream to amend the commit-msg hook and use --no-divider.

I supposed the code is somewhere in https://github.com/dependabot/dependabot-core/ but I could not find out how ---\n is injected :-\

The recommended workaround is to create and merge another commit with git commit --allow-empty, merge that, then trigger another deploy build.

You can save yourself the empty commit by not telling dependabot to rebase a dependency upgrade on master, and then triggering a deploy build when the current master commit (as a result of that) is a merge commit (i.e. the dependabot change wasn’t merged via fast-forward). That’s what I did for the last two builds.

Perhaps we should also reconsider if we need to include the full commit message of the commit-to-be-deployed at all. I think this made a lot more sense back when we built and (usually) merged one build commit per source commit (but didn’t necessarily deploy each build commit separately, unlike now); now that we’ve given that up, and only make occasional build commits that actually combine changes from several source commits, maybe it makes more sense to only include the commit subject (first line of the message) and commit hash and/or change ID.

I supposed the code is somewhere in https://github.com/dependabot/dependabot-core/ but I could not find out how ---\n is injected :-\

Could it be somehow coming from Addbot?

I supposed the code is somewhere in https://github.com/dependabot/dependabot-core/ but I could not find out how ---\n is injected :-\

Could it be somehow coming from Addbot?

As far as I see, it comes from dependabot itself, couldn't find actual code but here is test.

Maybe Addbot is not the part of the problem but solution. Alter commit part can be modified so it can remove/change --- before sending it to gerrit in the first place. Therefore we can solve it within ourselves.

@HasanAkgun_WMDE this seems to be approved, can you please merge the PR and move the ticket along?

Change 957274 had a related patch set uploaded (by Hasan Akgün (WMDE); author: Hasan Akgün (WMDE)):

[wikidata/query/gui@master] Add sanitize commit message option to dependabot-gerrit action

https://gerrit.wikimedia.org/r/957274

Change 957274 merged by jenkins-bot:

[wikidata/query/gui@master] Add sanitize commit message option to dependabot-gerrit action

https://gerrit.wikimedia.org/r/957274

Lucas_Werkmeister_WMDE changed the task status from Open to Stalled.EditedSep 25 2023, 1:47 PM

Can’t be tested at the moment – IIUC the gui-build job is currently fetching a particular commit rather than latest master, probably due to ongoing work at T328543: [QB] [WDQS-GUI] Move build scripts from CI to the repository. (Edit: I got that slightly wrong, see T328543#9195739 and possibly subsequent comments.)

Unstalling – in build #94, the “Change-Id must be in message footer” error still happened. It doesn’t look like the commit message of bump qunit replaced --- with ----.