Change 664990 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Minor improvements in test cleanup

https://gerrit.wikimedia.org/r/664990

Change 665063 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Use sample data that gets picked up by the cleanup

https://gerrit.wikimedia.org/r/665063

Change 665071 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Clean up anonymous too....

https://gerrit.wikimedia.org/r/665071

Change 665186 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Increase time limit on tests

https://gerrit.wikimedia.org/r/665186

Change 665187 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Improve test cleanup on geocoder test

https://gerrit.wikimedia.org/r/665187

Change 665194 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Improve Ponyo cleanup

https://gerrit.wikimedia.org/r/665194

Change 665195 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Remove now-obsolete PEAR error handling

https://gerrit.wikimedia.org/r/665195

Change 665196 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Improve Buffy cleanup

https://gerrit.wikimedia.org/r/665196

Change 665212 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Fix cleanup on email-amender

https://gerrit.wikimedia.org/r/665212

Change 665217 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Further minor fixes from upstream EmailAmmender repo

https://gerrit.wikimedia.org/r/665217

Change 664990 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Minor improvements in test cleanup

https://gerrit.wikimedia.org/r/664990

Change 665187 merged by Eileen:
[wikimedia/fundraising/crm@master] Improve test cleanup on geocoder test

https://gerrit.wikimedia.org/r/665187

Change 665071 merged by Eileen:
[wikimedia/fundraising/crm@master] Clean up anonymous too....

https://gerrit.wikimedia.org/r/665071

Change 665186 merged by Eileen:
[wikimedia/fundraising/crm@master] Increase time limit on tests

https://gerrit.wikimedia.org/r/665186

Change 665063 merged by Eileen:
[wikimedia/fundraising/crm@master] Use sample data that gets picked up by the cleanup

https://gerrit.wikimedia.org/r/665063

Change 665239 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Stick with a nice clean mouse

https://gerrit.wikimedia.org/r/665239

Change 665241 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] One less Test DAF 123827162

https://gerrit.wikimedia.org/r/665241

Change 665242 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Sorry Christine... you are to be deleted

https://gerrit.wikimedia.org/r/665242

Change 665243 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Reformat, remove silly function

https://gerrit.wikimedia.org/r/665243

Wow thanks for all this @Eileenmcnaughton! Even with a bunch of the patches not yet merged, on the master branch HEAD at this writing (898a31439725e), with a clean install using the most recent civibuild patches (b5a85eb0ddc), all tests are passing!

fundraising-dev$ ./civicrm-phpunit.sh wmff
PHPUnit 7.5.20 by Sebastian Bergmann and contributors.

...............................................................  63 / 515 ( 12%)
............................................................... 126 / 515 ( 24%)
............................................................... 189 / 515 ( 36%)
............................................................... 252 / 515 ( 48%)
............................................................... 315 / 515 ( 61%)
............................................................... 378 / 515 ( 73%)
............................................................... 441 / 515 ( 85%)
............................................................... 504 / 515 ( 97%)
...........                                                     515 / 515 (100%)

Time: 6.19 minutes, Memory: 119.00 MB

OK (515 tests, 5312 assertions)

@AndyRussG - yes most of the fixes are to reduce the number of failures you get when it is not a brand new install - by improving cleanup

In T274943#6847538, @Eileenmcnaughton wrote:

@AndyRussG - yes most of the fixes are to reduce the number of failures you get when it is not a brand new install - by improving cleanup

Ahhh right, thanks for the clarification! Indeed, on the second run, I got 3 failures...

@AndyRussG - yes - it might not quite get to zero - but the number of contacts left behind by the tests goes from triple digits to double digits with the chain merged - I left the scope as 'all the tests that inherit from the drupalBase test class' & then locked that in with a test that will fail if we start leaving behind test contacts again

Just a bunch of test cleanups in review

Change 665194 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Improve Ponyo cleanup

https://gerrit.wikimedia.org/r/665194

Change 665195 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Remove now-obsolete PEAR error handling

https://gerrit.wikimedia.org/r/665195

Change 665196 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Improve Buffy cleanup

https://gerrit.wikimedia.org/r/665196

Change 665212 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Fix cleanup on email-amender

https://gerrit.wikimedia.org/r/665212

Change 665239 merged by Eileen:
[wikimedia/fundraising/crm@master] Stick with a nice clean mouse

https://gerrit.wikimedia.org/r/665239

Change 665241 merged by Eileen:
[wikimedia/fundraising/crm@master] One less Test DAF 123827162

https://gerrit.wikimedia.org/r/665241

Change 665217 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Further minor fixes from upstream EmailAmmender repo

https://gerrit.wikimedia.org/r/665217

Change 665243 merged by Eileen:
[wikimedia/fundraising/crm@master] Reformat, remove silly function

https://gerrit.wikimedia.org/r/665243

Change 665242 merged by Eileen:
[wikimedia/fundraising/crm@master] Sorry Christine... you are to be deleted (test cleanup)

https://gerrit.wikimedia.org/r/665242

So with the last of this chain of patches now merged, here's how it went for me:

• Pulled the latest civibuildkit source.
• Using the setup.sh script:
  • Reset the database.
  • Ran composer and npm install for buildkit.
  • Re-cloned the wmff crm repo.
  • Ran the steps to recreate wmff (which includes buildkit create and deleting file artifacts).
• Ran all the unit tests, using the civicrm-phpunit.sh script. They all passed.
• Tried re-running the tests, again using the civicrm-phpunit.sh script. The first one failed, so I stopped them before they finished.
• Rebuilt the wmff site by running the command directly (docker-compose exec civicrm civibuild create wmff), so, this time, the file artifacts were not removed.
• Re-ran the tests using the civicrm-phpunit.sh script. They all passed again, but I got a weird error after the success message (see output pasted at the end of this comment).
• Rebuilt the wmff site again, as before, by running the command directly (docker-compose exec civicrm civibuild create wmff).
• Re-ran the tests using the civicrm-phpunit.sh script. They all passed again, and this time no weird error.

So, it seems fine, and the weird error is something flapping. And I don't doubt that we're now much closer to having a clean test suite, yaaaayyy!!! :)

@Eileenmcnaughton, is this sufficient to consider the test component of this task done?

Also, what more is still needed for the xdebug part?

Thanks so much for all your work on this!!!

Full output from the test run where I got the error (see above):

./civicrm-phpunit.sh wmff
PHPUnit 7.5.20 by Sebastian Bergmann and contributors.

...............................................................  63 / 515 ( 12%)
............................................................... 126 / 515 ( 24%)
............................................................... 189 / 515 ( 36%)
............................................................... 252 / 515 ( 48%)
............................................................... 315 / 515 ( 61%)
............................................................... 378 / 515 ( 73%)
............................................................... 441 / 515 ( 85%)
............................................................... 504 / 515 ( 97%)
...........                                                     515 / 515 (100%)

Time: 6.05 minutes, Memory: 117.00 MB

OK (515 tests, 5721 assertions)
PHP Fatal error:  Uncaught RuntimeException: Stream is detached in /srv/civi-sites/wmff/civicrm/vendor/guzzlehttp/psr7/src/Stream.php:89
Stack trace:
#0 /srv/civi-sites/wmff/vendor/mrmarkfrench/silverpop-php-connector/src/SilverpopConnector/SilverpopXmlConnector.php(1111): GuzzleHttp\Psr7\Stream->getContents()
#1 /srv/civi-sites/wmff/vendor/mrmarkfrench/silverpop-php-connector/src/SilverpopConnector/SilverpopXmlConnector.php(579): SilverpopConnector\SilverpopXmlConnector->post('<?xml version="...')
#2 /srv/civi-sites/wmff/vendor/mrmarkfrench/silverpop-php-connector/src/SilverpopConnector/SilverpopXmlConnector.php(47): SilverpopConnector\SilverpopXmlConnector->logout()
#3 [internal function]: SilverpopConnector\SilverpopXmlConnector->__destruct()
#4 {main}
  thrown in /srv/civi-sites/wmff/civicrm/vendor/guzzlehttp/psr7/src/Stream.php on line 89
PHP Stack trace:
PHP   1. {main}() /srv/civi-sites/wmff/vendor/phpunit/phpunit/phpunit:0
PHP   2. PHPUnit\TextUI\Command::main() /srv/civi-sites/wmff/vendor/phpunit/phpunit/phpunit:61
PHP   3. PHPUnit\TextUI\Command->run() /srv/civi-sites/wmff/vendor/phpunit/phpunit/src/TextUI/Command.php:162
PHP   4. PHPUnit\TextUI\TestRunner->doRun() /srv/civi-sites/wmff/vendor/phpunit/phpunit/src/TextUI/Command.php:206

Oh yeah - I think I remember something about that guzzle.

Note that when you get a fail on the first test it actually leaves the db better than before so re-running might work at that point

I think we should close this now