Have we tried merge=union? That seems vastly simpler.

I think a custom merge tool is the right solution here. merge=union doesn't work in all cases, for reasons I outlined in T201514: "This isn't exactly what we want, because if one side removes a line and the other adds in the same place, it will re-add the removed line. Also, conflicting changes to bits of the file that aren't line-based will result in duplicate lines. This will happen silently without warning, without any merge conflict being reported."

MatmaRex wrote a custom merge tool: https://github.com/MatmaRex/mediawikireleasenotes-driver If it works flawlessly, we should check it in to our .gitconfig and .gitattributes files in the repo. If we don't require that developers have Ruby installed, which I don't think we do (do we?), we should rewrite it in Python or PHP.

For the record, I tried out the linked merge tool and it ate a line on the first trial run. If we do this, we need a test suite.

Ideally this would run on gerrit, not developers' machines. That does not seem to be case though.

It should definitely run on Gerrit. That doesn't mean it shouldn't run on developers' machines too, though. Merging/rebasing is part of normal workflow, e.g., rebasing your changes locally onto origin/master. If Gerrit doesn't support it yet, at least we can support it on developers' machines, so that when you resolve the conflict locally it's automatic.

In T200392#4488442, @Simetrical wrote:

MatmaRex wrote a custom merge tool: https://github.com/MatmaRex/mediawikireleasenotes-driver If it works flawlessly, we should check it in to our .gitconfig and .gitattributes files in the repo. If we don't require that developers have Ruby installed, which I don't think we do (do we?), we should rewrite it in Python or PHP.

For the record, I tried out the linked merge tool and it ate a line on the first trial run. If we do this, we need a test suite.

I'd be interested to see your example (maybe file an issue on GitHub, since this is getting off-topic here), I've been using it ever since I wrote it (5 years ago) and I've never seen any incorrect merges. I guess I only rebase release notes maybe once a month, though.

https://github.com/MatmaRex/mediawikireleasenotes-driver/issues/2

(end off-topic)

In T200392#4488468, @Tgr wrote:

Ideally this would run on gerrit, not developers' machines. That does not seem to be case though.

Yeah, ideally we'd be able to install a custom merge driver into Gerrit, and then "rebase" would work just fine, and we'd adapt Zuul to use it as well. But given that it hasn't happened for the past 5 years, I'd like something in the meantime, which is this proposal.

I use MatmaRex's merge driver as well, but I find it pretty annoying to edit an older patch, push it, get a V-1 doesn't merge from Jenkins, then have to rebase, and re-push. Or you're ready to merge someone's patch, hit CR+2, and then see the V-1, have to pull it locally, rebase, re-push, and then CR+2 again. So I'd really like a solution that doesn't require rebasing locally.

Why can't Jenkins use the merge driver too, so that it will auto-merge these as well?

It probably could, I suggested it some years ago, but AFAIK no one has tried to implement this. It would have to be able to push the rebased changes for review, which AFAIK is also a feature that doesn't exist yet.

I don't think Jenkins has anything to do with merging; it's a job triggering framework. AIUI merging is done by Gerrit (when you click "Rebase") and Zuul (when you click +2 and it creates the revision to run gate-and-submit jobs on). Gerrit uses JGit which has no merge driver support. Zuul uses GitPython, which uses command line git, and even has merge drivers in one of its unit tests so that could probably work but how useful is it without support in Gerrit?

Maybe we could have a bot which automatically updates patches where only the release notes file is in conflict, and does the rebase locally.

Can we get MatmaRex's merge driver installed by default for all checkouts, maybe using composer (just because people are already running composer update)? It probably needs to be rewritten in Python first.

TechCom hosted an IRC meeting regarding this today. @Legoktm and I were the only ones in attendance. Below are the minutes and the log, but there isn't much in there.

• Minutes: https://tools.wmflabs.org/meetbot/wikimedia-office/2018/wikimedia-office.2018-10-16-21.00.html
• Log: https://tools.wmflabs.org/meetbot/wikimedia-office/2018/wikimedia-office.2018-10-16-21.00.log.html

The meat of it was

<legoktm> on the task there's some discussion about integrating merge drivers into Gerrit and Zuul, which I'm all for, but I have no idea how to do that technically, and I'm not sure anyone else does either
<legoktm> [...] I don't expect that to happen anytime soon... :(
<legoktm> so my bot proposal would just require me writing the bot and having it run on a cron

Which sounds reasonable (as I said above, integrating into Gerrit doesn't seem feasible anyway). Is it easier to have that bot merge release notes written in the custom syntax than have it rebase conflicted release notes, though? Granted the latter only works if nothing else is in conflict, but fixing release notes conflicts is easy enough that it doesn't make much difference when you have to fix conflicts in other files anyway.

TechCom is moving this to Last Call ending Wednesday 31 October 2pm PST (21:00 UTC, 22:00 CET)

Approved per today's TechCom meeting.

So how do we begin implementing this? It's a great idea and it seems like we just need to promote it?

Does the bot exist already or would that have to be written first?

The bot still needs to be written, but we can start implementing the commit messages (socializing through code-review) now.

A requisite bot / automation seems like something that we (Release-Engineering-Team) could build.

If we wanted a simpler change that would solve the immediate problem, I suggest just adding new release-notes lines in separate files in a designated directory. Before a release, a simple script could aggregate them into a traditional RELEASE-NOTES file. E.g., instead of adding * MediaWikiServices is now deprecated to the 'Deprecations in 1.x' section of RELEASE-NOTES, I'd put a file in release-notes/deprecations named whatever I feel like (name will be ignored by the script) with my release-notes line. Then to accumulate all the deprecations before making a release could be as simple as cat $(ls -tr release-notes/deprecations) or such.

In T200392#5421324, @Simetrical wrote:

Then to accumulate all the deprecations before making a release could be as simple as cat $(ls -tr release-notes/deprecations) or such.

I don't think local last modified times are related to repo last modified times in any way. You'd probably have to use git log (this SO answer has some examples).