Page MenuHomePhabricator
Create Task
Maniphest T215713

Missing dependencies in extension snapshots
Closed, ResolvedPublic
Actions

Description

There is composer/, module/ and pimple/ in the vendor directory of this link https://extdist.wmflabs.org/dist/extensions/Flow-REL1_31-a7bc00b.tar.gz, but composer/ and pimple/ is missing in this link https://extdist.wmflabs.org/dist/extensions/Flow-REL1_32-bf651cf.tar.gz

Eventually, I am given below error:

PHP Fatal error:  Class 'Pimple\Container' not found in /mywikis/installation/path/extensions/Flow/includes/Container.php on line 5

... and cannot access or use Structured Discussion pages at all, before I executed manually composer install --no-dev in the extensions/Flow directory.

UPDATE I am encountered same problem in TemplateStyles of REL1_32

Details

SubjectRepoBranchLines +/-
Include unzip package on extdist cloud vps tool for composeroperations/puppetproduction+1 -1
Allow installing composer dependencies even if missing php extensionslabs/tools/extdistmaster+1 -1
Customize query in gerrit

Related Objects

Event Timeline

Lens0021 created this task.Feb 10 2019, 4:29 AM
Restricted Application added a project: Growth-Team. · View Herald TranscriptFeb 10 2019, 4:29 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Lens0021 updated the task description. (Show Details)Feb 10 2019, 4:30 AM
Lens0021 renamed this task from Missing dependencies in the REL1_32 snapshot of the Structured Discussions to Missing dependencies in the REL1_32 snapshot of Structured Discussions.Feb 10 2019, 4:38 AM
Lens0021 renamed this task from Missing dependencies in the REL1_32 snapshot of Structured Discussions to Missing dependencies in REL1_32 snapshot of some extensions.Feb 10 2019, 3:15 PM
Lens0021 updated the task description. (Show Details)
Lens0021 added a project: TemplateStyles.
Legoktm claimed this task.Feb 10 2019, 7:16 PM
Legoktm edited projects, added VPS-project-Extdist; removed TemplateStyles, Growth-Team, StructuredDiscussions.
Legoktm subscribed.
2019-02-07 06:10:26,244 DEBUG:Running composer install for TemplateStyles
2019-02-07 06:10:31,408 ERROR:Traceback (most recent call last):
  File "/srv/extdist/nightly.py", line 190, in update_extension
    self.shell_exec([self.COMPOSER, 'install', '--no-dev'])
  File "/srv/extdist/nightly.py", line 142, in shell_exec
    return subprocess.check_output(args, **kwargs).decode('utf-8')
  File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command '['/srv/composer/vendor/bin/composer', 'install', '--no-dev']' returned non-zero exit status 2

2019-02-07 06:10:31,408 ERROR:composer install failed

Need to figure out why...

Lens0021 updated the task description. (Show Details)Feb 11 2019, 2:29 AM
• dcausse mentioned this in T216612: Build failures for Elastica and CirrusSearch on REL1_32.Feb 20 2019, 1:23 PM
Lens0021 mentioned this in T217328: Error in REL1_32-bf651cf: "Class 'Pimple\Container' not found".Feb 28 2019, 4:42 PM
Reedy merged a task: T217328: Error in REL1_32-bf651cf: "Class 'Pimple\Container' not found".Feb 28 2019, 8:29 PM
Reedy added a subscriber: Carlo666.
Urbanecm subscribed.Apr 15 2019, 1:17 PM
Extarys subscribed.May 23 2019, 3:32 AM
Lens0021 renamed this task from Missing dependencies in REL1_32 snapshot of some extensions to Missing dependencies in REL1_32 & REL1_33 snapshot of some extensions.EditedJul 5 2019, 3:28 PM
Lens0021 added a project: MW-1.33-release.

Same issue in REL1_33

Lens0021 added a comment.Oct 25 2019, 9:38 AM

If it has not high priority, It would be better to have Template:ExtensionInstall run composer install on the users at all times.

Lens0021 renamed this task from Missing dependencies in REL1_32 & REL1_33 snapshot of some extensions to Missing dependencies in REL1_32-REL1_34 snapshot of some extensions.Dec 21 2019, 4:50 PM
Lens0021 added a project: MW-1.34-release.
Bawolff awarded a token.Mar 11 2020, 3:23 PM
Reedy merged a task: T227362: composer dependencies missing for TimedMediaHandler tarball.Apr 7 2020, 1:12 AM
Reedy added subscribers: TheDJ, Ciencia_Al_Poder.
Tgr subscribed.Apr 16 2020, 5:16 PM
Physikerwelt subscribed.Apr 23 2020, 1:05 PM
Reedy mentioned this in T250406: RFC: Hybrid extension management.Apr 24 2020, 12:25 PM
Reedy renamed this task from Missing dependencies in REL1_32-REL1_34 snapshot of some extensions to Missing dependencies in extension snapshots.May 4 2020, 5:21 PM
Reedy merged a task: T251775: Attempting to install/use OATHAuth extension results in an error with "Class 'Base32\Base32'" not found.
Reedy added subscribers: MichaelLoucks, Reedy, Ammarpad.
RazeSoldier subscribed.May 4 2020, 5:42 PM
Ammarpad unsubscribed.May 5 2020, 6:30 AM
Bawolff subscribed.May 24 2020, 10:10 PM

Appears there are two issues:

  • Composer is checking platform-requirements. The php cli doesn't have very many extensions enabled so some are missing, some things want php 7.2, etc
  • Composer wants the php-zip extension in order to unzip downloaded dependencies
gerritbot added a comment.May 24 2020, 10:13 PM

Change 598283 had a related patch set uploaded (by Brian Wolff; owner: Brian Wolff):
[labs/tools/extdist@master] Allow installing composer dependencies even if missing php extensions

https://gerrit.wikimedia.org/r/598283

gerritbot added a project: Patch-For-Review.May 24 2020, 10:13 PM
gerritbot added a comment.May 24 2020, 10:16 PM

Change 598283 merged by jenkins-bot:
[labs/tools/extdist@master] Allow installing composer dependencies even if missing php extensions

https://gerrit.wikimedia.org/r/598283

gerritbot added a comment.May 24 2020, 10:25 PM

Change 598284 had a related patch set uploaded (by Brian Wolff; owner: Brian Wolff):
[operations/puppet@production] Include unzip package on extdist cloud vps tool for composer

https://gerrit.wikimedia.org/r/598284

gerritbot added a comment.May 25 2020, 2:29 PM

Change 598284 merged by Muehlenhoff:
[operations/puppet@production] Include unzip package on extdist cloud vps tool for composer

https://gerrit.wikimedia.org/r/598284

Maintenance_bot removed a project: Patch-For-Review.May 25 2020, 3:10 PM
Stashbot added a comment.May 25 2020, 5:50 PM

Mentioned in SAL (#wikimedia-cloud) [2020-05-25T17:50:56Z] <bawolff> running nightly.py with --force option to fix composer issue (T215713)

Bawolff added a comment.May 25 2020, 6:37 PM

New issue:

Updating dependencies
Package operations: 27 installs, 0 updates, 0 removals
  - Installing composer/installers (v1.9.0): Loading from cache
  - Installing mediawiki/semantic-forms (3.7): Downloading (failed)    Failed to download mediawiki/semantic-forms from dist: The "https://api.github.com/repos/wikimedia/mediawiki-extensions-SemanticForms/zipball/7e9e627853dea94d6b1344383513927fd037903b" file could not be downloaded (HTTP/1.1 404 Not Found)
    Now trying to download from source
  - Installing mediawiki/semantic-forms (3.7): The authenticity of host 'github.com (140.82.114.4)' can't be established.
RSA key fingerprint is SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8.
Are you sure you want to continue connecting (yes/no)? yes
Cloning failed using an ssh key for authentication, enter your GitHub credentials to access private repos
Head to https://github.com/settings/tokens/new?scopes=repo&description=Composer+on+extdist-04+2020-05-25+1834
to retrieve a token. It will be stored in "/home/extdist/.composer/auth.json" for future use by Composer.
Token (hidden): 
Invalid token provided.
You can also add it manually later by using "composer config --global --auth github-oauth.github.com <token>"
Cloning 7e9e627853

                                                                                                                     
  [RuntimeException]                                                                                                 
  Failed to execute git clone --no-checkout 'https://no:***@github.com/wikimedia/mediawiki-extensions-SemanticForms  
  .git.git' 'extensions/SemanticForms/' && cd 'extensions/SemanticForms/' && git remote add composer 'https://no:**  
  *@github.com/wikimedia/mediawiki-extensions-SemanticForms.git.git' && git fetch composer && git remote set-url or  
  igin 'https://github.com/wikimedia/mediawiki-extensions-SemanticForms.git.git' && git remote set-url composer 'ht  
  tps://github.com/wikimedia/mediawiki-extensions-SemanticForms.git.git'                                             
                                                                                                                     
  Cloning into 'extensions/SemanticForms'...                                                                         
  remote: Invalid username or password.                                                                              
  fatal:***@github.com/wikimedia/mediawiki-extensions-SemanticForms.git.git/'
Bawolff mentioned this in T253586: extensions including semantic-forms via composer are broken and breaking extension distributor.May 25 2020, 7:02 PM
Bawolff closed this task as Resolved.May 25 2020, 7:42 PM
Bawolff claimed this task.

This should be fixed now

TheDJ added a comment.May 26 2020, 10:46 AM

Thank you @Bawolff

Lens0021 awarded a token.May 27 2020, 11:14 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits