Page MenuHomePhabricator
Create Task
Maniphest T203084

Migrate DonationInterface CI job to Docker
Closed, ResolvedPublic
Actions

Description

For mediawiki/extensions/DonationInterface we have specific jobs. They use specific branches and specially run patchsets against REL1_27 branch of MediaWiki (among other tricks).

Moreover, the tests are running with PHP Zend 5.6.

Jobs to migrate:

mwext-donationinterfacecore-REL1_27-zend56-jessie
mwext-donationinterfacecore-REL1_28-zend56-jessie

Dummy test change: https://gerrit.wikimedia.org/r/#/c/324066

Repro simulating a CI job for a DonationInterface patch:

ZUUL_PROJECT=mediawiki/extensions/DonationInterface quibble \
  --packages-source composer \
  --db mysql \
  --run=phpunit \
  --branch REL1_27 \
  --project-branch mediawiki/extensions/DonationInterface=master \
  --project-branch mediawiki/vendor=fundraising/REL1_27 \
  mediawiki/vendor \
  mediawiki/extensions/DonationInterface \
  mediawiki/extensions/FundraisingEmailUnsubscribe \
  mediawiki/extensions/ContributionTracking \
  mediawiki/extensions/ParserFunctions \
  mediawiki/extensions/cldr

Note: the job clones mediawiki/vendor but also run composer install from the root of mediawiki/core which generates a file for the composer merge plugin:

composer.local.json
{
   "extra" : {
      "merge-plugin" : {
         "include" : [
            "extensions/DonationInterface/composer.json",
            "extensions/FundraisingEmailUnsubscribe/composer.json",
            "extensions/ContributionTracking/composer.json",
            "extensions/ParserFunctions/composer.json",
            "extensions/cldr/composer.json"
         ]
      }
   }
}

Details

Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

hashar triaged this task as Medium priority.Aug 29 2018, 1:26 PM
hashar created this task.
gerritbot added a comment.Aug 29 2018, 1:27 PM

Change 456141 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: Quibble container with PHP 5.6

https://gerrit.wikimedia.org/r/456141

gerritbot added a project: Patch-For-Review.Aug 29 2018, 1:27 PM
gerritbot added a comment.Aug 29 2018, 1:38 PM

Change 456141 merged by jenkins-bot:
[integration/config@master] docker: Quibble container with PHP 5.6

https://gerrit.wikimedia.org/r/456141

gerritbot added a comment.Aug 29 2018, 2:13 PM

Change 456148 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Docker jobs for DonationInterface

https://gerrit.wikimedia.org/r/456148

gerritbot added a comment.Aug 29 2018, 2:24 PM

Change 456152 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: quibble-jessie-php56 fix package name

https://gerrit.wikimedia.org/r/456152

gerritbot added a comment.Aug 29 2018, 2:28 PM

Change 456152 merged by jenkins-bot:
[integration/config@master] docker: quibble-jessie-php56 fix package name

https://gerrit.wikimedia.org/r/456152

gerritbot added a comment.Aug 29 2018, 2:43 PM

Change 456148 merged by jenkins-bot:
[integration/config@master] Docker jobs for DonationInterface

https://gerrit.wikimedia.org/r/456148

hashar updated the task description. (Show Details)Aug 29 2018, 2:47 PM
hashar added a comment.Aug 29 2018, 2:53 PM

I have deployed the new container that ships Zend php 5.6. I have added the new job to Zuul and it can be triggered on any change by commenting check experimental. I did that on the dummy change #324066 and it fails:

https://integration.wikimedia.org/ci/job/quibble-donationinterface-REL1_27-zend56-docker/2/console

1) AmazonApiTest::testDoPaymentSuccess
ReflectionException: Class PayWithAmazon\Mocks\MockPaymentsClient does not exist

/workspace/src/vendor/wikimedia/smash-pig/Core/Configuration.php:178
/workspace/src/extensions/DonationInterface/amazon_gateway/amazon.adapter.php:164
/workspace/src/extensions/DonationInterface/amazon_gateway/amazon.adapter.php:134
/workspace/src/extensions/DonationInterface/amazon_gateway/amazon.api.php:50
/workspace/src/includes/api/ApiMain.php:1362
/workspace/src/includes/api/ApiMain.php:437
/workspace/src/tests/phpunit/includes/api/ApiTestCase.php:112
/workspace/src/extensions/DonationInterface/tests/phpunit/Adapter/Amazon/AmazonApiTest.php:43
/workspace/src/tests/phpunit/MediaWikiTestCase.php:156
/workspace/src/tests/phpunit/phpunit.php:262

2) AmazonApiTest::testDoPaymentErrors
ReflectionException: Class PayWithAmazon\Mocks\MockPaymentsClient does not exist

/workspace/src/vendor/wikimedia/smash-pig/Core/Configuration.php:178
/workspace/src/extensions/DonationInterface/tests/phpunit/Adapter/Amazon/AmazonApiTest.php:81
/workspace/src/tests/phpunit/MediaWikiTestCase.php:156
/workspace/src/tests/phpunit/phpunit.php:262

3) DonationInterface_Adapter_Amazon_Test::testDoPaymentSuccess
ReflectionException: Class PayWithAmazon\Mocks\MockPaymentsClient does not exist

/workspace/src/vendor/wikimedia/smash-pig/Core/Configuration.php:178
/workspace/src/extensions/DonationInterface/amazon_gateway/amazon.adapter.php:164
/workspace/src/extensions/DonationInterface/amazon_gateway/amazon.adapter.php:134
/workspace/src/extensions/DonationInterface/tests/phpunit/Adapter/Amazon/AmazonTest.php:154
/workspace/src/tests/phpunit/MediaWikiTestCase.php:156
/workspace/src/tests/phpunit/phpunit.php:262

4) DonationInterface_Adapter_Amazon_Test::testDoPaymentDeclined
ReflectionException: Class PayWithAmazon\Mocks\MockPaymentsClient does not exist

/workspace/src/vendor/wikimedia/smash-pig/Core/Configuration.php:178
/workspace/src/extensions/DonationInterface/tests/phpunit/Adapter/Amazon/AmazonTest.php:183
/workspace/src/tests/phpunit/MediaWikiTestCase.php:156
/workspace/src/tests/phpunit/phpunit.php:262

5) DonationInterface_Adapter_Amazon_Test::testFailOnAmazonRejected
ReflectionException: Class PayWithAmazon\Mocks\MockPaymentsClient does not exist

/workspace/src/vendor/wikimedia/smash-pig/Core/Configuration.php:178
/workspace/src/extensions/DonationInterface/tests/phpunit/Adapter/Amazon/AmazonTest.php:210
/workspace/src/tests/phpunit/MediaWikiTestCase.php:156
/workspace/src/tests/phpunit/phpunit.php:262

6) DonationInterface_Adapter_Amazon_Test::testTransactionTimedOut
ReflectionException: Class PayWithAmazon\Mocks\MockPaymentsClient does not exist

/workspace/src/vendor/wikimedia/smash-pig/Core/Configuration.php:178
/workspace/src/extensions/DonationInterface/tests/phpunit/Adapter/Amazon/AmazonTest.php:235
/workspace/src/tests/phpunit/MediaWikiTestCase.php:156
/workspace/src/tests/phpunit/phpunit.php:262

7) DonationInterface_Adapter_Amazon_Test::testClientException
ReflectionException: Class PayWithAmazon\Mocks\MockPaymentsClient does not exist

/workspace/src/vendor/wikimedia/smash-pig/Core/Configuration.php:178
/workspace/src/extensions/DonationInterface/tests/phpunit/Adapter/Amazon/AmazonTest.php:261
/workspace/src/tests/phpunit/MediaWikiTestCase.php:156
/workspace/src/tests/phpunit/phpunit.php:262

8) DonationInterface_Adapter_Amazon_Test::testDoRecurringPaymentSuccess
ReflectionException: Class PayWithAmazon\Mocks\MockPaymentsClient does not exist

/workspace/src/vendor/wikimedia/smash-pig/Core/Configuration.php:178
/workspace/src/extensions/DonationInterface/amazon_gateway/amazon.adapter.php:164
/workspace/src/extensions/DonationInterface/amazon_gateway/amazon.adapter.php:134
/workspace/src/extensions/DonationInterface/tests/phpunit/Adapter/Amazon/AmazonTest.php:290
/workspace/src/tests/phpunit/MediaWikiTestCase.php:156
/workspace/src/tests/phpunit/phpunit.php:262

Seems like an issue with composer and autoloading :-(

hashar updated the task description. (Show Details)Aug 29 2018, 3:38 PM
hashar updated the task description. (Show Details)
hashar added a comment.Aug 29 2018, 3:56 PM

The original job relies on mediawiki/vendor and then run composer install inside the DonationInterface extension:

cd $WORKSPACE/src/extensions/DonationInterface
composer install

Then the MediaWiki install is run.

Quibble only supports using the composer merge plugin to ship dependencies (via composer update).

I guess I will need a hack to handle DonationInterface as a special edge case.

hashar mentioned this in T183512: [EPIC] Migrate Mediawiki jobs from Nodepool to Docker.Aug 29 2018, 4:11 PM
hashar added a comment.Aug 29 2018, 5:07 PM

Based on a discussion with @Ejegg , the issues would be because of amzn/login-and-pay-with-amazon-sdk-php. DonationInterface composer.json has:

"require": {
    "amzn/login-and-pay-with-amazon-sdk-php": "dev-master",
},
"repositories": [
    {
        "type": "git",
        "url": "https://github.com/ejegg/login-and-pay-with-amazon-sdk-php"
    }
],

And the composer.lock has:

{
    "name": "amzn/login-and-pay-with-amazon-sdk-php",
    "version": "dev-master",
    "source": {
        "type": "git",
        "url": "https://github.com/ejegg/login-and-pay-with-amazon-sdk-php",
        "reference": "e3f8d1082556104526999740beaf55b481aaf323"
    },

But when installing with the composer merge plugin, the repository is apparently ignored and it ends up doing:

Installing amzn/login-and-pay-with-amazon-sdk-php (dev-master 272469d): [257.2MB/29.88s]

That is dev-master on https://packagist.org/packages/amzn/login-and-pay-with-amazon-sdk-php#dev-master states 272469d62dfe8e027fa091442825c7d9ce449af4

That is apparently not the last commit of upstream lib https://github.com/amzn/amazon-pay-sdk-php/commits/master

gerritbot added a comment.Sep 11 2018, 10:57 AM

Change 459762 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/core@fundraising/REL1_27] Composer merge plugin now prepends VCS repositories

https://gerrit.wikimedia.org/r/459762

gerritbot added a comment.Sep 11 2018, 11:04 AM

Change 459765 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/extensions/DonationInterface@master] Test merge plugin prepend-repositories

https://gerrit.wikimedia.org/r/459765

gerritbot added a comment.Sep 11 2018, 11:06 AM

Change 459767 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Add experimental DonationInterface job mediawiki

https://gerrit.wikimedia.org/r/459767

gerritbot added a comment.Sep 11 2018, 11:07 AM

Change 459767 merged by jenkins-bot:
[integration/config@master] Add experimental DonationInterface job mediawiki

https://gerrit.wikimedia.org/r/459767

hashar added a comment.Sep 11 2018, 1:28 PM

So that will be solved by bumping wikimedia/composer-merge-plugin from 1.3.1 to 1.4.1. It would prepend the repositories before packagist which thus cause composer to prefer the VCS version over packagist.

I gave it a try locally and it installs:

Installing amzn/login-and-pay-with-amazon-sdk-php (dev-master e3f8d10)

Which is the master branch of https://github.com/ejegg/login-and-pay-with-amazon-sdk-php

https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/459762 Use composer-merge-plugin 1.4.1 to prepend VCS repos

I could not figure out how to update mediawiki/vendor@fundraising/REL1_27. composer always install unrelated dependencies. @Ejegg could update vendor please? :]

hashar moved this task from Backlog to In progress on the Continuous-Integration-Infrastructure (shipyard) board.Sep 11 2018, 1:42 PM
hashar added a comment.Sep 11 2018, 1:59 PM

And to confirm, I gave it a try with:

mediawiki/corefundraising/REL1_27Gerrit 459762Bump composer-merge-plugin to 1.4.1
mediawiki/vendorfundraising/REL1_26Gerrit 459792BAD change to update merge plugin
DonationInterfacemasterGerrit 459765Dummy with Depends-On to two previous changes

A check experimental on the DonationInterface change triggered https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/DonationInterface/+/459765/ which is passing since it did:

  • Installing amzn/login-and-pay-with-amazon-sdk-php (dev-master e3f8d10)

So I guess we want to merge the mediawiki/core patch https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/459762/ and then one has to craft an update for mediawiki/vendor. Once done, I will switch CI to use the Docker based job :]

hashar moved this task from Backlog to Blocked (externally) on the Release-Engineering-Team (Kanban) board.Sep 24 2018, 4:43 PM
gerritbot added a comment.Sep 26 2018, 9:22 AM

Change 459792 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/vendor@fundraising/REL1_27] Update wikimedia/composer-merge-plugin to 1.4.1

https://gerrit.wikimedia.org/r/459792

gerritbot added a comment.Sep 26 2018, 9:22 AM

Change 462888 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/vendor@fundraising/REL1_27] Sync composer.json/composer.lock

https://gerrit.wikimedia.org/r/462888

gerritbot added a comment.Oct 16 2018, 1:36 PM

Change 467682 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Fix DonationInterface skip-if rule

https://gerrit.wikimedia.org/r/467682

gerritbot added a comment.Oct 16 2018, 1:41 PM

Change 467682 merged by jenkins-bot:
[integration/config@master] Fix DonationInterface skip-if rule

https://gerrit.wikimedia.org/r/467682

gerritbot added a comment.Oct 17 2018, 4:39 PM

Change 468018 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] mediawiki/core has fundraising/RELxxx branches

https://gerrit.wikimedia.org/r/468018

gerritbot added a comment.Oct 17 2018, 4:54 PM

Change 468018 merged by jenkins-bot:
[integration/config@master] mediawiki/core has fundraising/RELxxx branches

https://gerrit.wikimedia.org/r/468018

hashar added a comment.Oct 17 2018, 6:19 PM

quibble-donationinterface-REL1_27-zend56-docker SUCCESS in 3m 33s \o/

That is using a dummy change https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/DonationInterface/+/459765/ that has the required dependencies.

hashar mentioned this in T124572: Move assert-phpflavor macro into a bash script in integration/jenkins.Nov 5 2018, 1:46 PM
gerritbot added a comment.Nov 13 2018, 10:48 AM

Change 473181 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Migrate DonationInterface to Docker job (non voting)

https://gerrit.wikimedia.org/r/473181

gerritbot added a comment.Nov 13 2018, 10:57 AM

Change 473181 merged by jenkins-bot:
[integration/config@master] Migrate DonationInterface to Docker job (non voting)

https://gerrit.wikimedia.org/r/473181

hashar edited projects, added Continuous-Integration-Infrastructure (Slipway); removed Continuous-Integration-Infrastructure (shipyard).Dec 31 2018, 3:19 PM
hashar moved this task from Backlog to Stalled on the Continuous-Integration-Infrastructure (Slipway) board.Jan 15 2019, 10:02 AM
hashar removed hashar as the assignee of this task.Mar 19 2019, 8:39 PM

There are patch aboves, including one to update the composer merge plugin. I haven't had any review or feedback on those though.

Anyway I have too many things to do and I am not actively working on this task.

gerritbot added a comment.Apr 11 2019, 2:32 PM

Change 459762 abandoned by Reedy:
Use composer-merge-plugin 1.4.1 to prepend VCS repos

Reason:
No one cares about fundraising/REL1_27 anymore :)

https://gerrit.wikimedia.org/r/459762

hashar closed this task as Resolved.Apr 11 2019, 7:06 PM
hashar claimed this task.

REL1_27 and the outdated composer-merge-plugin are gone.

DonationInterface is now using REL1_31 / Quibble / php7 on Docker https://gerrit.wikimedia.org/r/#/c/integration/config/+/499334/

\o/

gerritbot added a comment.Apr 11 2019, 7:07 PM

Change 459792 abandoned by Hashar:
Update wikimedia/composer-merge-plugin to 1.4.1

Reason:
No more needed. Got migrated to fundraising/REL1_31 https://gerrit.wikimedia.org/r/#/c/integration/config/ /499334/

https://gerrit.wikimedia.org/r/459792

gerritbot added a comment.Apr 11 2019, 7:07 PM

Change 462888 abandoned by Hashar:
Sync composer.json/composer.lock

Reason:
No more needed. Got migrated to fundraising/REL1_31 https://gerrit.wikimedia.org/r/#/c/integration/config/ /499334/

https://gerrit.wikimedia.org/r/462888

gerritbot added a comment.Apr 11 2019, 7:07 PM

Change 459765 abandoned by Hashar:
Test merge plugin prepend-repositories

Reason:
No more needed. Got migrated to fundraising/REL1_31 https://gerrit.wikimedia.org/r/#/c/integration/config/ /499334/

https://gerrit.wikimedia.org/r/459765

gerritbot added a comment.Aug 29 2019, 1:04 PM

Change 533205 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] DonationInterface job is now passing fine

https://gerrit.wikimedia.org/r/533205

gerritbot added a comment.Aug 29 2019, 3:16 PM

Change 533205 merged by jenkins-bot:
[integration/config@master] DonationInterface job is now passing fine

https://gerrit.wikimedia.org/r/533205

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits