Mediawiki uses several different ways of finding out about directories for temporary files. An array is created of the env vars TMPDIR, TMP and TEMP, and get_sys_temp_dir and upload_tmp_dir are added as well. Then all are checked whether they're not empty, they exist and whether they are writeable. As per https://github.com/wikimedia/mediawiki/blob/master/includes/libs/filebackend/fsfile/TempFSFile.php#L86

By itself, there is most certainly nothing wrong with that. It goes wrong however, when the localisation cache code assumes that it can just write to the temporary folder and create a file named l10n_cache_LANGCODE.cdb or use the existing one. https://github.com/wikimedia/mediawiki/blob/master/includes/cache/localisation/LCStoreCDB.php#L141

This can go wrong very easily if you use a secure setup with suexec+fcgid or in my case just php-fpm. If multiple users have a wiki installation with localisation (enabled by default when php intl is found), only the first wiki to create the file will be good by default.
The default on many different OSes is to return a centrale tmp folder for all users. In our, one user would be able to create /tmp/l10n_cache-en.cdb and then the others would try to use and overwrite the file, fail, and then throw an exception as they can't change a temporary file they don't own.

For example:

[2016-12-28 02:27:32.740954] [proxy_fcgi:error] [R:WGMVBMCoAI0AAFE9YPgAAACM] [H:bc1] AH01071: Got error 'PHP message: PHP Warning: rename(/tmp/l10n_cache-nl.cdb.tmp.208999319,/tmp/l10n_cache-nl.cdb): Operation not permitted in /home/boss/bert/www/wikitest/mediawiki-1.27.1/vendor/wikimedia/cdb/src/Writer/PHP.php on line 89\nPHP message: PHP Warning: fclose(): supplied resource is not a valid stream resource in /home/boss/bert/www/wikitest/mediawiki-1.27.1/vendor/wikimedia/cdb/src/Writer/PHP.php on line 236\nPHP message: PHP Warning: fwrite(): supplied resource is not a valid stream resource in /home/boss/bert/www/wikitest/mediawiki-1.27.1/vendor/wikimedia/cdb/src/Writer/PHP.php on line 100\nPHP message: PHP Warning: fclose(): supplied resource is not a valid stream resource in /home/boss/bert/www/wikitest/mediawiki-1.27.1/vendor/wikimedia/cdb/src/Writer/PHP.php on line 236\nPHP message: PHP Warning: unlink(/tmp/l10n_cache-nl.cdb.tmp.208999319): No such file or directory in /home/boss/bert/www/wikitest/mediawiki-1.27.1/vendor/wikimedia/cdb/src/Writer/PHP.php on line 237\nPHP message: PHP Fatal error: Uncaught Cdb\\Exception: Error writing to CDB file "/tmp/l10n_cache-nl.cdb.tmp.208999319". in /home/boss/bert/www/wikitest/mediawiki-1.27.1/vendor/wikimedia/cdb/src/Writer/PHP.php:239\nStack trace:\n#0 /home/boss/bert/www/wikitest/mediawiki-1.27.1/vendor/wikimedia/cdb/src/Writer/PHP.php(102): Cdb\\Writer\\PHP->throwException('Error writing t...')\n#1 /home/boss/bert/www/wikitest/mediawiki-1.27.1/vendor/wikimedia/cdb/src/Writer/PHP.php(215): Cdb\\Writer\\PHP->write('\\x002\\x95Sq\\x1C\\x07\\x00')\n#2 /home/boss/bert/www/wikitest/mediawiki-1.27.1/vendor/wikimedia/cdb/src/Writer/PHP.php(82): Cdb\\Writer\\PHP->finish()\n#3 /home/boss/bert/www/wikitest/mediawiki-1.27.1/vendor/wikimedia/cdb/src/Writer.php(88): Cdb\\Writer\\PHP->close()\n#4 [internal function]: Cdb\\Writer->__destruct()\n#5 {main}\n thrown in /home/boss/bert/www/wikitest/mediawiki-1.27.1/vendor/wikimedia/cdb/src/Writer/PHP.php on line 239\n'

This is not only a problem when using multiple users. Even when using a standard mod_php www-data own everything-style setup, multiple wikis would all use the same cache and cache files, that doesn't really sound like a sensible idea.

My suggestion therefore is to either implement namespaces on these kinds of files, or extremely well fit tempnam()-function https://secure.php.net/manual/en/function.tempnam.php
Tempnam would fit in perfectly with custom cache folders and the different ways of finding one automatically as described earlier, but would also make sure that name collisions can no longer take place.

I am not sure whether localisation is the only case of a semi-hardcoded filename. But I hope it is!

I'm not sure whether after this explenation it's still necesarry. But to reproduce this bug, simple install apache and php-fpm, use something like for example mod_proxy_fcgi or mod_fastcgi to hook up 2 vhosts each for a different user to 2 pools each for one of the users. Make sure php intl is installed, then install mediawiki on both vhosts. One will not be able to work as long as the other has the l10n cache files sitting in /tmp.