Searched For: gitsubmodules
## Context
The Wikibase and WikibaseLexeme extension Git repositories have several submodules: some libraries and the new (mobil … :NewLexeme page in WikibaseLexeme. Git repository options for them includ … be used as the upstream URL of the submodules (T301273, [I7b1e3cbafc](https://ge … dules` URL for the three remaining submodules, but enforces a different developm … GitLab: Currently not used for any submodules, but allows a mostly GitHub-like d …
## Main Objectives
* Move the …
org/r/#/c/96544/ for an example. The submodule is incorrectly removed.
I've detected that we keep forgetting to register extension submodules at {rMEXT} while creating new repo … s doesn't seem to like each other).
y also add archived/readonly repos which ain't really necessary there.
Found when working with @jnuche on adding `git LFS` to a repository deployed by s … file://`. That is not supported by Git LFS 2.7.1 (but is by 2.10).
The c … whether LFS would work this way.
kibase/serialization-javascript
[x] data-values/data-types (JS part)
been recently failing CI with errors like this:
```
21:39:56 Running "git-status" task
21:39:56 >> Unstaged … 29bc58b343
21:39:57 Warning: Task "git-status" failed. Use --force to co … o idea how it could be related.
`git-status` comes from:
```
name=Grunt … s,lang=js
grunt.registerTask( 'git-status', function () {
var … require( 'child_process' ).exec( 'git ls-files --modified', function ( e … require( 'child_process' ).exec( '…
e-data-values-value-view'. Retry scheduled
Retried attempts also fail
We don't, which means some of our tests are skipped, which seems bad.
The following submodules are good candidates to be imported … It also happened in the past that git submodule sha updates in operation … tages that could have been avoided.
ment branch of mediawiki core, rather than branching and cloning 100+ submodules each week. Changes to the extensi … 1. One view of deployed code - the git history of the deployment branches … nterested parties can see what the git history looks like with this arran … g to be, suggestions are welcomed.
cluster, I'm blocked by this error on each machine:
```
Cloning into 'submodules/ores'...
Submodule path 'submodules/ores': checked out '2c54f421755349 … a4ce35e6ce92d2bfd92'
Cloning into 'submodules/wheels'...
Submodule path 'submodules/wheels': checked out 'd7fa640c59ae … 3516c76235e1fca5cf9'
Cloning into 'submodules/wikiclass'...
error: RPC failed; r … tor.wikimedia.org/source/wikiclass.git' into submodule path 'submodules…
/shell/limit.sh --whitelist=/srv/mediawiki/core/extensions/Linter/../.git/modules/Linter --noroot --seccomp= … diawiki/core/extensions/Linter/../.git/modules/Linter'\'' '\''/usr/bin/git'\'' '\''show'\'' '\''-s'\'' '\''-- … diawiki/core/extensions/Linter/../.git/modules/Linter --noroot --seccomp= … diawiki/core/extensions/Linter/../.git/modules/Linter'\'' '\''/usr/bin/git'\'' '\''show'\'' '\''-s'\'' '\''-- … diawiki/core/extensions/Linter/../.…
------------------------
**Version**: unspecified
**Severity**: normal
reaking code updates on beta:
```
10:23:02 INFO:mwextupdate:running: git submodule update --init --recursiv … ribe'
```
And then locally
```
$ git submodule update --init --recursiv … 'vendor/coderkungfu/php-queue'
```
```==> default: Notice: /Stage[main]/Role::Apex/Mediawiki::Skin[Apex]/Git::Clone[mediawiki/skins/Apex]/File[ … ]/Role::Apex/Mediawiki::Skin[Apex]/Git::Clone[mediawiki/skins/Apex]/Exec[ … ]/Role::Apex/Mediawiki::Skin[Apex]/Git::Clone[mediawiki/skins/Apex]/Exec[ … pex]/returns: fatal: remote error: Git repository not found
==> default: Error: /usr/bin/git clone --recurse-submodules https://gerrit.wikimedia.org/r/p/mediawiki/skins/Apex.…
While cleaning slaves disk I found out mediawiki-core-npm job process git submodules. On wmf branches that means cloni … /
The configuration does show the git submodule processing enabled.
Jobs running on Ubuntu Trusty for repositories with submodules are currently broken due to a bug in the Jenkins GIT Plugin.
We use Jenkins GIT Plugin 1.5.0, which uses git-client-plugin 1.0.6:
https://github.com/jenkinsci/git-client-plugin/blob/git-client-1.0.6/src/main/java/org/jen … ive");
}
args.add("git reset");
if (hard) {
… and "submodule foreach --recursive git reset --hard" returned status code … Entering 'lib/ve'
stderr: /usr/lib/…
The translatewiki repository relies on git submodules hosted on github using either http or the git protocol:
https://github.com/puppetlabs/puppetlabs-apt.git
git://github.com/tPl0ch/puppet-composer.git
Since lanthanum does not have dir … is run on that host, the clone of submodules fails:
https://integration.wikim … sole
But then that fails fetching git://github.com..
In man git-config, there seems to be possibil … ity to set proxy for both HTTP and …
d by MediaWiki's GitInfo class without shell (it just reads from the .git/HEAD file directly), I additionall … calcuating the commit dates with `git show`) has also not been disabled … (tested on tin.eqiad.wmnet).
The .git directory itself is not excluded f … -file will naturally not sync the .git directory, when using scap or sync … t, [[Special:Version]] does show a git hash (maybe not the best one, but … 't work for extensions because the …
The integration/jenkins.git project is available for deploymen … cripts
That repositories has some submodules, to deploy it I went with:
git deploy start
git submodule update --init
git deploy sync
All the target have received a copy of the main git repository but the submodules ARE NOT THERE.
Expected behavior: `git deploy sync` should replicate the … wmf-deployment
**Severity**: normal
Some extensions, such as Extension:GuidedTour, require git submodules.
It would be a very useful enhanc … Distributor#Doesn.27t_do_submodules
d the easyrdf submodule is not checked as it is not pulled as recurse submodules are not pulled
------------------ … wmf-deployment
**Severity**: minor
Jenkins git plugin has an issue that prevents us from using submodules in the two repositories. The plugin set the URL for the submodules to be /var/lib/zuul/git/... instead of the supplied url in .gitmodules.
Running "git submodule sync" does fix it though … **: unspecified
**Severity**: major
On my local wiki, Special:Version only shows git SHA1 hashes for extensions that have local .git subdirectories because I checked o … icular version. Where I relied on `git submodule update --init`, the extension's .git is a plain file containing only a … ine, for example extension/Vector/.git contains only
gitdir: /home/spage/projects/core/.git/modules/extensions/Vector
and it … thing special when an extension's .…
When git submodule update fails, Quibble do … ing:
```
INFO:quibble.cmd:Updating git submodules of extensions and skins
extensions … pace path set to: ./
```
Probably git submodule just exit 0 when that sh … ):
self.log.info('Updating git submodules of extensions and skins')
# From JJB macro ext-skins-submodules-update
# jjb/mediawiki-ext … r that will process mediawiki/core submodules in
# wmf branches whic … '\n'.join([
'…
ync copies of schema definitions, we should include them as submodule.
The Refreshed skin doesn't include the WikiFont git submodule in it's own repository s … /Brickimedia/brickimedia/issues/397
e `--branch REL1_25`. Since the release branch has the extensions as submodules, we should really use git submodule update instead.
e.php:424
01:00:48 /workspace/src/maintenance/doMaintenance.php:94
```
Per https://www.mediawiki.org/wiki/Review_queue#Deploy_to_Beta_Cluster
We're getting failures due to the issues in T218844...
Can we add `git submodule sync --recursive` into the mix of git clone/checkout/update for the extensions.git repo?
https://git-scm.com/docs/git-submodule
```
sync [--recursive] [--] [<path>…]
Synchronizes submodules' remote URL configuration setting … modules. It will only affect those submodules which already have a URL entry in .git/config (that is the case when they … r local repositories accordingly.
…
ry repository. ResourceLoader definitions are to be moved to Wikibase.git instead. DataTypesModule class should also be moved to Wikibase.git
The repo of the npm package should be a submodule in Wikibase git repo.
Wikibase should define Resou … 18n files from the npm package dir.
ry repository. ResourceLoader definitions are to be moved to Wikibase.git instead.
The repo of the npm package should be a submodule in Wikibase git repo.
Wikibase should define Resou … he npm package, that Wikibase uses.
ry repository. ResourceLoader definitions are to be moved to Wikibase.git instead.
The repo of the npm package should be a submodule in Wikibase git repo.
Wikibase should define Resou … he npm package, that Wikibase uses.
ry repository. ResourceLoader definitions are to be moved to Wikibase.git instead.
Make sure there is no MW … fore turning it into a npm package.
ry repository. ResourceLoader definitions are to be moved to Wikibase.git instead.
The repo of the npm package should be a submodule in Wikibase git repo.
Wikibase should define Resou … he npm package, that Wikibase uses.
ry repository. ResourceLoader definitions are to be moved to Wikibase.git instead.
The repo of the npm package should be a submodule in Wikibase git repo.
Wikibase should define Resou … he npm package, that Wikibase uses.
The repo of the npm package should be a submodule in Wikibase git repo.
Wikibase should define Resou … he npm package, that Wikibase uses.
iled to recurse into submodule path 'scripts/i18n'
The command "eval git submodule update --init --recursiv … 'scripts/i18n'
The command "eval git submodule update --init --recursiv … 'scripts/i18n'
The command "eval git submodule update --init --recursive" failed 3 times.
The command "git submodule update --init --recursiv … .
Your build has been stopped.
```
ry repository. ResourceLoader definitions are to be moved to Wikibase.git instead.
`git submodule update --init --recursiv … e into submodule path 'Widgets'
```
If the git submodule i18n has not been initia … wikimedia.org/show_bug.cgi?id=63327
When rewriting git submodule URLs on targets to point … actual path to the module in the .git/modules dir.
```
[@tin:/srv/deplo … eventbus] (master)[4c422c3] ± ls .git/modules/config/
event-schemas
```
… eqiad.wmnet/eventlogging/eventbus/.git/modules/config/schemas
```
`http: … eqiad.wmnet/eventlogging/eventbus/.git/modules/config/schemas` is not the … eqiad.wmnet/eventlogging/eventbus/.git/modules/config/event-schemas`.
…
re's also this:
```krenair@deployment-bastion:/srv/mediawiki-staging$ git diff
diff --git a/portals b/portals
index 551a95c. … not update properly on beta:
```
$ git log --oneline --color --decorate H … primary colors and search icon
```
I try to download all extension via git
```
mkdir mediawiki && cd mediawiki
git clone https://gerrit.wikimedia.org/r/p/mediawiki/extensions.git
cd extensions
git submodule update --init --recursiv … same issue. Any solution?
Thanks.
pdated.
On tin:
```
kartik@tin:/srv/deployment/cxserver/deploy/src$ git show HEAD
commit 84ad47245b278a520 … rv/deployment/cxserver/deploy/src$ git show HEAD
commit a3c21d2b5c1363ade … a2de889d4e10821e1
```
While main git repo (cxserver/deploy) is updated on tin and targets.
(from bug 64371 comment #3)
> Bah the job uses a different git-deploy target (..)
> /srv/deployme … submodule from integration/jenkins.git if it is
> not used :)
Let's fin … wmf-deployment
**Severity**: normal
make mwmultiversion a submodule of operations/mediawiki-config.git
Will be easier to fetches it that … **: unspecified
**Severity**: minor
g to master on #bluespice repos causes updating `mediawiki/extensions.git` to fail.
Please stop it.
---+-----------------------------------+
9 rows in set (0.249 sec)
```
fc
Previously, Gerrit HTTP 502 errors in CI from 2020-2022: {T246763}
19:51:33 Finished cache_git_info (duration: 12m 12s)
```
12 minutes :(
Skins have been removed from my machine, due to the introduction of "submodules" within the main MediaWiki reposit … ome people may lose work like I did
===Description
Currently we have 2 submodules that need to be manually updated t … eir commit sha's updated with:
```
git submodule update --remote
```
Inc … CI
===Acceptance Criteria
[] Both submodules in the xLab code base are automati … on include the latest upstream data
OUNTEREXAMPLE
15:40:05 15:40:05 prep failed: <FailedCommand> Command 'git fetch --tags --jobs 2 --no-recurse-submodules origin master' failed with exit co … shell' marked build as failure
```
T296022 has removed the SSH git-ssh.wikimedia.org endpoint which w … ntication.
It pushed to ssh://vcs@git-ssh.wikimedia.org/source/phabricator-translations.git
The repository is a submodule of … oyment repo is on Gerrit and other submodules are on Gerrit, we should migrate ssh://vcs@git-ssh.wikimedia.org/source/phabricator-translations.git to Gerrit with push rights granted … )** https://www.mediawiki.org/wiki/Git/Creating_new_repositories#Importing_from_an_existing_repository…
is is probably not be the fault of the release notes, but the release git tagging. Tag `wmf/1.45.0-wmf.24` … om the including parent repository.
he clone started after 03:00:56 UTC and errored out at 03:01:25 UTC.
te the issue**:
* Checkout any version of pywikibot since 7.0.0 from git
* Install it using pip (e.g., `pip … the same as it would be from PyPI.
This is tracking steps for splitting wikibase.git.
Jeroen De Dauw, 13 July 2014, "Splitting the Wikibase git repo":
> Now that we have code th … depends on things in the Wikibase git (ie Wikibase Query depends on Wiki … d Wikibase Lib), the state of this git repo is becoming problematic. It f … d co can then use
> * We split the git repo in 3: repo, client and lib
> … ctober 2014, "Testing the Wikibase.git split":
> Last week I created a … experimental split of the Wikibase.…
~~We run a custom git-fat package, which is in standard … tream: https://github.com/jedbrown/git-fat/issues/92, so we could also co … them if we want to continue to use git-fat.~~
We need to stop using git-fat. It's antiquated, based on Pyt … do this. Mostly, this means using git-lfs, but it may also mean just not … . Subtasks for each usage we have.
is task can skip the Tech Verification column and be closed directly.
On T224191 we detected again missing submodules missing from {rMEXT}. While I used … mediawiki/extensions/$extensioname.git` repo irrespective if it is archiv … logies if I missatributed you here.
new branch named `wmf/<version>`
- It checks out `mediawiki/core`
-- `git branch -B wmf/<version>`
-- Removes existing submodules (`git submodule deinit` && `git rm`)
-- Modifies `extensions/.giti … ions/.gitignore#1|line ]]
-- Does `git submodule add --branch <wmf/<versi … with `Implicit merge` (see T373425)
paths_urls = git.list_submodules_paths_urls(locatio … p/lib/python3.9/site-packages/scap/git.py", line 707, in list_submodules_ …
return _runcmd(["git", subcommand] + list(args), **kwar … cap.runcmd.FailedCommand: Command 'git submodule -q foreach echo -n "$PWD " && git remote get-url origin --recursive' …
usage: git remote get-url [--push] [--all] <n …
usage: …
What it says on the tin. git-lfs has become the de facto standard for large file support in Git. It works on Github, has support i … nts are far easier to install than git-fat or git-annex.
Documentation (Work in pro … ttps://wikitech.wikimedia.org/wiki/Git-lfs
This task is complete when sc … xisting repository, relying on LFS submodules and large files.
Reading https://github.com/git-lfs/git-lfs/wiki/Tutorial#migrating-existi … our options are either to rewrite git history in our repos, or create ne … that scap will rehydrate the repos.
What will happen if we try to checkout a project with git-lfs-enabled submodules on tin?
And will scap be able to f … and checkout on deployment targets?
is task can skip the Tech Verification column and be closed directly.
ing rules (particularly those which have repo name without trailing `.git`) should be fixed. Also the origin … ority of this task can be lowered).
sk if they need more work and comment the Phabricator Task-ID there.
iki-config/+/723652/5#message-68f2e2d7f4564e39a84b27703751cedee5f9e88d
resh box, trying to review a Vagrant patch (from the host box):
```
$ git review
The following command failed with exit code 128
"git submodule foreach cp -p .git/hooks/commit-msg "$(git rev-parse --git-dir)/hooks/""
-------------------- … t/modules/nginx'
cp: cannot stat '.git/hooks/commit-msg': Not a directory … --
```
It got over it after I ran `git-review -s` but it's not great user … automatically for new gerrit repos.…
Cloning into '/builds/wikimedia-enterprise/services/structured-data/submodules/parser/submodules/wmf'...
error: cannot run ssh: No … l: unable to fork
fatal: clone of 'git@gitlab.gluzdov.com:wikimedia-enterprise/general/wmf.git' into submodule path '/builds/wiki … nterprise/services/structured-data/submodules/parser/submodules/wmf' failed
Failed to clone 'submodules/wmf'. Retry scheduled
Cloning into … nterprise/services/structured-data/…
Gerrit logs on logstash ]], I found out two WMCS instances are doing git fetches against the primary Gerrit … uring upload-pack from /srv/gerrit/git/mediawiki/core.git
| type | org.eclipse.jetty.io.EofE … read | HTTP POST /r/mediawiki/core/git-upload-pack
The traffic comes fr … ate the repository (most probably `git`)
* get the git version being used (`git --version`)
** we might want to try a newer git version from `-backports`
* check whether …
Our git repo structure is currently that w … r. The upstream repo DOES have a a git repo - but it doesn't actually mat … e easily fixable once that is done)
sk if they need more work and comment the Phabricator Task-ID there.
onds=total_seconds)))
```
It seems like more than half of the total `git gc` time is spent on that repo:
` … ub.blog/2021-08-16-highlights-from-git-2-33/#geometric-repacking (cGit NO … gerrit/+/refs/heads/master/contrib/git-exproll.sh
gc log for scoring/or … sizeOfPackedObjects=2617021456
```
```
vagrant@mediawiki-vagrant:/vagrant$ git status
fatal: Not a git repository: /home/gtisza/Wikimedia/vagrant/TemplateStyles/.git/modules/puppet/modules/cdh
```
Caused by the puppet submodules' `.git` files containing that path (three … bash history, it was created with `git clone --recursive https://gerrit.w … diawiki/vagrant .`. Is it possible git behavior changed recently and it i … where it used to use relative ones?…
s the error I had previously https://phabricator.wikimedia.org/T235909
file=language` parameter
* [ ] The submodule-commit in WikibaseLexeme.git as bumped to include this function … ) and does not cause errors there
xtension, while it should be showing 7aad7f5. Bryan says he knows why.
t was needed to fix it (thanks @thcipriani for the help!) was to run `git update-server-info` inside the repository on `tin`, and also in its git submodule, given that the repo has a git submodule.
According to @thcipria … orReturnCode_128>
RAN: /usr/bin/git clone --jobs 1 http://tin.eqiad.wmnet/debmonitor/deploy/.git /srv/deployment/debmonitor/deploy- … tin.eqiad.wmnet/debmonitor/deploy/.git/' not found
```
es:
- [ ] Updating local npm and composer dependencies
- [ ] Updating submodules
- [x] Producing static assests for … egration test running instructions
en commits after that.
For example 5 days ago:
```
cd 'phabricator'; git add .; if ! git diff --cached --quiet; then git commit -m 'Localisation updates from https://translatewiki.net.'; git rebase 'origin/wmf/stable' && git push origin 'wmf/stable'; fi
[wmf/ … wmf/stable is up to date.
To ssh://git-ssh.wikimedia.org/source/phabricator-translations.git
93f0042..1f0d46f wmf/stable -> … d1179222043c still exists somewhere…
`integration/docroot.git` is used as the Apache DocumentRoo … we just had a Jenkins job to just `git pull`.
With time we have moved do … a ssh to the machine and issuing a git pull.
We also have two CI masters … ld migrate to use #scap instead.
ke to be able to use ExtensionJson.json to install any extension from git. I thought the git url might already be in it (since …
What needs to be done to add the git repo?
I have been running the git version of Pywikibot on several ac … 'm on the latest version now
```
$ git pull origin master
From https://ge … HEAD
Already up to date.
```
```
$ git rev-parse HEAD
2cf5b090d79cd8a27ad8c446286bc89a7f1083ff
```
```
$ git submodule update --recursive --rem … * AttributeError should be resolved
at the time we ran it).
# Analysis
The `do_core_work` method runs `git('checkout', '-B', branch)` against … anch` method doesn't pass through).
f the CI images entry points share the same common code:
```lang=bash
git init
git fetch --depth 2 --quiet "${ZUUL_URL}/${ZUUL_PROJECT}" "$ZUUL_REF"
git checkout --quiet FETCH_HEAD
git submodule --quiet update --init -- … a feature flag to skip processing submodules when `GIT_NO_SUBMODULES` is set:
` … MODULES:-}" ]; then
set -x
git submodule --quiet update --init -- … "\$GIT_NO_SUBMODULES set, skipping submodules…
be just fine, and actually have read the code and it looked like the git url will be properly forged.
Even … mple
Error: Execution of '/usr/bin/git -c core.sharedRepository=group clone --recurse-submodules
https://gerrit.wikimedia.org/r/m … awiki/services/integration/docroot.git /srv/deployment/integration/docroo … e redundant `repository` parameter.
mediawiki/extensions repo hasn't been updated since the Gerrit upgrade