Looks like xdebug slow down composer/phpunit quite a bit. I noticed that when comparing:
mwext-Wikibase-repo-tests-sqlite-php55-trusty
mwext-Wikibase-repo-tests-sqlite-php55-jessie
trusty:
```
PHP 5.5.9-1ubuntu4.21 (cli) (built: Feb 9 2017 20:54:58)
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2014 Zend Technologies
with Zend OPcache v7.0.3, Copyright (c) 1999-2014, by Zend Technologies
with Xdebug v2.2.3, Copyright (c) 2002-2013, by Derick Rethans
```
Time: **2.6 minutes**, Memory: 1341.50MB
jessie:
```
PHP 5.5.38-4+wmf1+jessie (cli)
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2015 Zend Technologies
with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2015, by Zend Technologies
```
Time: **1.15 minutes**, Memory: 1306.25MB
When installing the Debian package php-xdebug, it adds an ini file for the cli as `/etc/php5/cli/conf.d/20-xdebug.ini`. We should drop that file and when we do need xdebug, we can explicitly enabled it via `php -d extension=xdebug.so`.