In T172165#4221992, @ArielGlenn wrote:As mentioned a couple of days ago, some jobs on terbium are still running with php5; those are the jobs that run via foreachwiki as www-data which calls foreachwikiindblist.
A list of those jobs is here: P7139There's two options.
One: fix foreachwikindblist so it doesn't use php5 but whatever /usr/bin/php points to, which on terbium would be hhvm. Someone(s) should feel confident that the above jobs run well on hhvm before such a change is deployed.
Two: wait for terbium's replacement (mwmaint1001) to become the live server. See T192092 This would happen sometime after the tin -> deploy1001 switchover, which is currently scheduled for May 25, see T175288#4205322 In this case, someone(s) should feel confident that the above jobs run well on php7.0.Can folks discuss and get us a plan?
Description
Details
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| s/php5/php/ in foreachwikiindblist | operations/puppet | production | +1 -1 |
Event Timeline
Change 434754 had a related patch set uploaded (by Reedy; owner: Reedy):
[operations/puppet@production] s/php5/php/ in foreachwikiindblist
Stuff that uses foreachwiki, and hence, php5
modules/mediawiki/manifests/maintenance/purge_abusefilter.pp: command => '/usr/local/bin/foreachwiki extensions/AbuseFilter/maintenance/purgeOldLogIPData.php >/var/log/mediawiki/purge_abusefilter.log 2>&1', modules/mediawiki/manifests/maintenance/purge_checkuser.pp: command => '/usr/local/bin/foreachwiki extensions/CheckUser/maintenance/purgeOldData.php >/dev/null 2>&1', modules/mediawiki/manifests/maintenance/cirrussearch.pp: command => '/usr/local/bin/foreachwiki extensions/CirrusSearch/maintenance/saneitizeJobs.php --push --refresh-freq=7200 >> /var/log/mediawiki/cirrus-sanitize/push-jobs.log 2>&1', modules/mediawiki/manifests/maintenance/initsitestats.pp: command => '/usr/local/bin/foreachwiki initSiteStats.php --update > /dev/null', modules/mediawiki/manifests/maintenance/echo_mail_batch.pp: command => '/usr/local/bin/foreachwikiindblist /srv/mediawiki/dblists/echo.dblist extensions/Echo/maintenance/processEchoEmailBatch.php >/dev/null', modules/mediawiki/manifests/maintenance/pageassessments.pp: command => '/usr/local/bin/foreachwikiindblist /srv/mediawiki/dblists/pageassessments.dblist extensions/PageAssessments/maintenance/purgeUnusedProjects.php > /dev/null', modules/mediawiki/manifests/maintenance/cleanup_upload_stash.pp: command => '/usr/local/bin/foreachwiki maintenance/cleanupUploadStash.php > /dev/null', modules/mediawiki/manifests/maintenance/purge_expired_userrights.pp: command => '/usr/local/bin/foreachwiki maintenance/purgeExpiredUserrights.php >/dev/null 2>&1', modules/mediawiki/manifests/maintenance/purge_securepoll.pp: command => '/usr/local/bin/foreachwiki extensions/SecurePoll/cli/purgePrivateVoteData.php >/dev/null 2>&1', modules/mediawiki/manifests/maintenance/update_special_pages.pp: command => 'flock -n /var/lock/update-special-pages /usr/local/bin/foreachwiki updateSpecialPages.php > /var/log/mediawiki/updateSpecialPages.log 2>&1',
So, running extensions/CheckUser/maintenance/purgeOldData.php under normal foreachwiki, and a hacked version of foreachwikiindblist to use hhvm gives some interesting (but expected? results).
php5: real 7m8.995s
hhvm: real 15m43.211s
HHVM takes twice as long. I guess it's the usual issue of hhvm startup being excessive for short running scripts.
Is this enough for us to care? Or for shorter running scripts, is this manageable, knowing that it will be switched to php7 eventually?
Most of the jobs aren't time sensitive... But I think the Echo and CirrusSearch ones are more important and run more regularly?
CirrusSearch runs every 2 hours, at 10 past
Echo just runs at midnight though, so less freuqent
If you're testing anyway, how long does maintenance/rebuildLocalisationCache.php take on PHP7 (ref. T191921: mwscript rebuildLocalisationCache.php takes 40 minutes on HHVM (rather than ~5 on PHP 5))? ;-)
reedy@mwmaint1001:/tmp$ php7.0 --version
PHP 7.0.27-0+deb9u1 (cli) (built: Jan 5 2018 13:51:52) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies
with Zend OPcache v7.0.27-0+deb9u1, Copyright (c) 1999-2017, by Zend Technologies
reedy@mwmaint1001:/tmp$ php -v
HipHop VM 3.18.6-dev (rel)
Compiler: 3.18.5+dfsg-1+wmf8+deb9u1
Repo schema: 7be0062745976798500274e76b3907ae9448a681
reedy@mwmaint1001:/tmp$ php7.0 --version
PHP 7.0.27-0+deb9u1 (cli) (built: Jan 5 2018 13:51:52) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies
with Zend OPcache v7.0.27-0+deb9u1, Copyright (c) 1999-2017, by Zend Technologies
reedy@mwmaint1001:/tmp$ PHP=php7.0 mwscript eval.php enwiki
> echo PHP_VERSION;
7.0.27-0+deb9u1
reedy@mwmaint1001:/tmp$ time PHP=php7.0 mwscript rebuildLocalisationCache.php --wiki=enwiki --outdir=/tmp/l10nstuff
<snip>
408 languages rebuilt out of 408
real 4m35.088s
reedy@mwmaint1001:/tmp$ ls -al l10ntsuff/l10n_cache-en.cdb
-rw-r--r-- 1 www-data www-data 3956618 May 24 12:08 l10nstuff/l10n_cache-en.cdbFile size looks good/sane
reedy@tin:/srv/mediawiki-staging$ ls -al php-1.32.0-wmf.5/cache/l10n/l10n_cache-en.cdb -rw-r--r-- 1 l10nupdate l10nupdate 3968828 May 24 03:11 php-1.32.0-wmf.5/cache/l10n/l10n_cache-en.cdb
So then compare with hhvm on the same host as php7
reedy@mwmaint1001:/tmp$ time PHP=hhvm mwscript rebuildLocalisationCache.php --wiki=enwiki --outdir=/tmp/l10nstuff <snip> 408 languages rebuilt out of 408 real 16m59.381s
And then php5 on tin...
reedy@tin:/tmp$ time PHP=php5 mwscript rebuildLocalisationCache.php --wiki=enwiki --outdir=/tmp/l10nstuff <snip> 408 languages rebuilt out of 408 real 24m13.240s
I, for one, welcome our new PHP7 overlords?
Change 434754 merged by Giuseppe Lavagetto:
[operations/puppet@production] s/php5/php/ in foreachwikiindblist
As of now, any new script will only run on php7 or HHVM. We can consider this task resolved.