Figure out how to make separate subtree include both test and includes files in the history if possible.
Also move both of those directories in future to one. We suggested that name could be "Lib/packages/Changes" because "Lib/libraries" seemed confusing and silly.
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Move \Wikibase\Lib\Changes to separate package | mediawiki/extensions/Wikibase | master | +9 -0 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T256058 Extract PHP package/library for "changes" as a "subtree" from the Wikibase monorepo | |||
| Resolved | Lucas_Werkmeister_WMDE | T258395 Make git play nice |
I’ve been playing with git subtree and found something that works. Basically, it’s a bunch of git subtree splits (or pushes), wired together using git replace – two git replaces are necessary to connect the history past the changes → Changes directory rename, and one git replace connects the those old histories to the new history where in Wikibase.git the library is now under lib/packages/changes/. git replace refs are not pushed or pulled by default, but can be transferred explicitly, so we would add something to the README advising users to run git fetch origin 'refs/replace/*:refs/replace/*' if they want nice Git history (compare ceylon.formatter instructions).
And now, any further commits made that affect lib/packages/changes/ (I tested with an “Add README”) can be pushed using git subtree push -P lib/packages/changes/ /tmp/changes/ moved, and the history will do the right thing.
That said, this isn’t a very nice history – because src/ and tests/ were exported separately, there are two copies of most commits, one touching src/ and the other touching tests/. I’ll see if I can find a solution for that – maybe it can be done using git filter-branch instead of git subtree split, “filtering” the working tree by removing all directories except the few that we want to keep. (Doing this via the real git filter-branch promises to be extremely slow, because it would actually check out the full working tree each time and then shell out to /bin/rm to remove it again, but I remember hearing about a utility that should be able to do it faster.)
Edit: looking through the git replace manpage, it seems git replace --graft 2fdc9baef4 tests-lcase would’ve been an easier alternative to step 9. 🤷
Yes, git filter-repo worked very nicely (and finished in less than four seconds, extremely impressive):
And now, like before, any further changes commits can be pushed with git subtree push, and the history will do the right thing, thanks to the git replace. (At some point, you’ll want to get rid of the current master branch and rename moved to master, because moved is where the history continues.)
(Also, git filter-repo preserves all the other branches – wmf/, REL_, and some random stuff we should probably throw away even in Wikibase? – which we probably want to delete.)
As for the directory name within Wikibase, maybe it should be closer to the packagist package name? I assume that will be something like wikibase/changes, so maybe the directory would be lib/packages/wikibase/changes? (I imagine we might also extract some packages into the wmde namespace instead of wikibase, so we probably don’t want to remove the “wikibase” path component.) Or possibly just packages/wikibase/changes, without lib.
I don’t think we have any tags in Wikibase.git at the moment. For branches, we could mirror them (git filter-repo preserves them), but I’m not sure if that would be useful… maybe we could keep REL_ branches but discard wmf. branches?
I also just realized that we should be able to use git-filter-repo instead of git subtree push for the later updates too, since the commit hashes it produces are also stable. (In that case, we might not even need a git replace.)
I have some repos locally, should I push them to GitHub? (They’re probably not the final form yet.)
(I’m hesitant to push them because that might trigger notifications for people or issues mentioned in the commit messages :/)
Oh, I meant in the library. When you want to make a new release, you'd push a git tag to the split repo, right? Or do you need to push it to Wikibase and have it get subtree'd over to the split repo?
Oh, I see. Yes, I think we’d push tags directly to the split repo, that should work. (Though I’m not yet sure how often we’ll release the library at all – Wikibase will always use the “master” version, if I understand correctly, not the one from packagist.)
Yeah under your account (and not org like wmde/wikimedia) would not cause notifications (I have done it in github.com/Ladsgroup/tainted-refs
Alright, I pushed that repository to https://github.com/lucaswerkmeister/wikibase-changes. If you clone it, and then also do git fetch origin 'refs/replace/*:refs/replace/*', you should get a full history.
That said, if we’d be using git-filter-repo anyways, I’m now wondering if we need git replace at all? After all, git-filter-repo can do the rename already, so I’m not sure what we need a git replace for… might look into this again tomorrow and create a second demo repository.
I think git-filter-repo would work for now. We can revisit if we still need git replace for whatever reason.
Just one note: I think the files first need to move to somewhere else. Like containing all of them in one place. Similar to this POC: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/593557
Like files being packages/wikibase-changes/src and packages/wikibase-changes/test etc. in Wikibase itself (we can make extension-repo.json and extension-client.json understand the php files easily) and we also can keep the namespace so they stay in the same address but like this:
"AutoloadNamespaces": { "Wikibase\\Lib\\": "lib/includes/", "Wikibase\\Lib\\Changes": "packages/wikibase-changes/src/" },
With this:
Regardless of what version Wikibase uses, I think for a library to be valuable it needs proper releases...
Change 616117 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/Wikibase@master] Move \Wikibase\Lib\Changes to separate package
The git-filter-repo incantation to extract the Changes history is:
git-filter-repo --path=lib/{includes,tests/phpunit}/{c,C}hanges/ --path lib/packages/wikibase/changes/ --path .mailmap --path-rename=lib/includes/{c,C}hanges:src --path-rename=lib/tests/phpunit/{c,C}hanges:tests --path-rename lib/packages/wikibase/changes/: --message-callback 'return re.sub(b"^changes: ", b"", message)'
--path selects the paths to include; --path-rename renames them accordingly; the --message-callback turns commit messages like “changes: Something” into just “Something”, removing the “changes:” prefix. The result can be found at https://github.com/lucaswerkmeister/wikibase-changes/.
Change 616117 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Move \Wikibase\Lib\Changes to separate package