Page MenuHomePhabricator

Missing dependencies in extension snapshots
Closed, ResolvedPublic

Description

There is composer/, module/ and pimple/ in the vendor directory of this link https://extdist.wmflabs.org/dist/extensions/Flow-REL1_31-a7bc00b.tar.gz, but composer/ and pimple/ is missing in this link https://extdist.wmflabs.org/dist/extensions/Flow-REL1_32-bf651cf.tar.gz

Eventually, I am given below error:

PHP Fatal error:  Class 'Pimple\Container' not found in /mywikis/installation/path/extensions/Flow/includes/Container.php on line 5

... and cannot access or use Structured Discussion pages at all, before I executed manually composer install --no-dev in the extensions/Flow directory.

UPDATE I am encountered same problem in TemplateStyles of REL1_32

Event Timeline

Lens0021 created this task.Feb 10 2019, 4:29 AM
Restricted Application added a project: Growth-Team. · View Herald TranscriptFeb 10 2019, 4:29 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Lens0021 updated the task description. (Show Details)Feb 10 2019, 4:30 AM
Lens0021 renamed this task from Missing dependencies in the REL1_32 snapshot of the Structured Discussions to Missing dependencies in the REL1_32 snapshot of Structured Discussions.Feb 10 2019, 4:38 AM
Lens0021 renamed this task from Missing dependencies in the REL1_32 snapshot of Structured Discussions to Missing dependencies in REL1_32 snapshot of some extensions.Feb 10 2019, 3:15 PM
Lens0021 updated the task description. (Show Details)
Lens0021 added a project: TemplateStyles.
Legoktm claimed this task.Feb 10 2019, 7:16 PM
Legoktm added a subscriber: Legoktm.
2019-02-07 06:10:26,244 DEBUG:Running composer install for TemplateStyles
2019-02-07 06:10:31,408 ERROR:Traceback (most recent call last):
  File "/srv/extdist/nightly.py", line 190, in update_extension
    self.shell_exec([self.COMPOSER, 'install', '--no-dev'])
  File "/srv/extdist/nightly.py", line 142, in shell_exec
    return subprocess.check_output(args, **kwargs).decode('utf-8')
  File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command '['/srv/composer/vendor/bin/composer', 'install', '--no-dev']' returned non-zero exit status 2

2019-02-07 06:10:31,408 ERROR:composer install failed

Need to figure out why...

Lens0021 updated the task description. (Show Details)Feb 11 2019, 2:29 AM
Lens0021 renamed this task from Missing dependencies in REL1_32 snapshot of some extensions to Missing dependencies in REL1_32 & REL1_33 snapshot of some extensions.EditedJul 5 2019, 3:28 PM
Lens0021 added a project: MW-1.33-release.

Same issue in REL1_33

If it has not high priority, It would be better to have Template:ExtensionInstall run composer install on the users at all times.

Lens0021 renamed this task from Missing dependencies in REL1_32 & REL1_33 snapshot of some extensions to Missing dependencies in REL1_32-REL1_34 snapshot of some extensions.Dec 21 2019, 4:50 PM
Lens0021 added a project: MW-1.34-release.
Tgr added a subscriber: Tgr.Apr 16 2020, 5:16 PM
Reedy renamed this task from Missing dependencies in REL1_32-REL1_34 snapshot of some extensions to Missing dependencies in extension snapshots.May 4 2020, 5:21 PM
Reedy added subscribers: MichaelLoucks, Reedy, Ammarpad.
Ammarpad removed a subscriber: Ammarpad.May 5 2020, 6:30 AM

Appears there are two issues:

  • Composer is checking platform-requirements. The php cli doesn't have very many extensions enabled so some are missing, some things want php 7.2, etc
  • Composer wants the php-zip extension in order to unzip downloaded dependencies

Change 598283 had a related patch set uploaded (by Brian Wolff; owner: Brian Wolff):
[labs/tools/extdist@master] Allow installing composer dependencies even if missing php extensions

https://gerrit.wikimedia.org/r/598283

Change 598283 merged by jenkins-bot:
[labs/tools/extdist@master] Allow installing composer dependencies even if missing php extensions

https://gerrit.wikimedia.org/r/598283

Change 598284 had a related patch set uploaded (by Brian Wolff; owner: Brian Wolff):
[operations/puppet@production] Include unzip package on extdist cloud vps tool for composer

https://gerrit.wikimedia.org/r/598284

Change 598284 merged by Muehlenhoff:
[operations/puppet@production] Include unzip package on extdist cloud vps tool for composer

https://gerrit.wikimedia.org/r/598284

Mentioned in SAL (#wikimedia-cloud) [2020-05-25T17:50:56Z] <bawolff> running nightly.py with --force option to fix composer issue (T215713)

New issue:

Updating dependencies
Package operations: 27 installs, 0 updates, 0 removals
  - Installing composer/installers (v1.9.0): Loading from cache
  - Installing mediawiki/semantic-forms (3.7): Downloading (failed)    Failed to download mediawiki/semantic-forms from dist: The "https://api.github.com/repos/wikimedia/mediawiki-extensions-SemanticForms/zipball/7e9e627853dea94d6b1344383513927fd037903b" file could not be downloaded (HTTP/1.1 404 Not Found)
    Now trying to download from source
  - Installing mediawiki/semantic-forms (3.7): The authenticity of host 'github.com (140.82.114.4)' can't be established.
RSA key fingerprint is SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8.
Are you sure you want to continue connecting (yes/no)? yes
Cloning failed using an ssh key for authentication, enter your GitHub credentials to access private repos
Head to https://github.com/settings/tokens/new?scopes=repo&description=Composer+on+extdist-04+2020-05-25+1834
to retrieve a token. It will be stored in "/home/extdist/.composer/auth.json" for future use by Composer.
Token (hidden): 
Invalid token provided.
You can also add it manually later by using "composer config --global --auth github-oauth.github.com <token>"
Cloning 7e9e627853

                                                                                                                     
  [RuntimeException]                                                                                                 
  Failed to execute git clone --no-checkout 'https://no:***@github.com/wikimedia/mediawiki-extensions-SemanticForms  
  .git.git' 'extensions/SemanticForms/' && cd 'extensions/SemanticForms/' && git remote add composer 'https://no:**  
  *@github.com/wikimedia/mediawiki-extensions-SemanticForms.git.git' && git fetch composer && git remote set-url or  
  igin 'https://github.com/wikimedia/mediawiki-extensions-SemanticForms.git.git' && git remote set-url composer 'ht  
  tps://github.com/wikimedia/mediawiki-extensions-SemanticForms.git.git'                                             
                                                                                                                     
  Cloning into 'extensions/SemanticForms'...                                                                         
  remote: Invalid username or password.                                                                              
  fatal:***@github.com/wikimedia/mediawiki-extensions-SemanticForms.git.git/'
Bawolff closed this task as Resolved.May 25 2020, 7:42 PM
Bawolff claimed this task.

This should be fixed now