Page MenuHomePhabricator

Cloud-vagrant: shared directory (/vagrant/settings.d/) seen by CLI PHP only
Open, Needs TriagePublic

Description

After lots of debugging of an apparent wiki misconfiguration issue, I found that early on when Vagrant loads /var/www/w/dblist.php, and that file calls glob( '/vagrant/settings.d/wikis/dbConf/*dbConf.php' ), /vagrant/settings.d/wikis/dbConf/10-wiki-dbConf.php is not picked up (and so $wgLocalDatabases will be left empty and the site will fail to load). The file looks normal:

vagrant@mediawikivagrant:~$ ls -l /vagrant/settings.d/wikis/dbConf/*dbConf.php
-rw-r--r-- 1 vagrant_share vagrant_share 226 Sep 12 23:12 /vagrant/settings.d/wikis/dbConf/10-wiki-dbConf.php

although I don't quite get why it's owned by vagrant_share, but both the vagrant and www-data users can see it, and accessing it from PHP also works in most contexts (e.g. shell.php, plain CLI, built-in server with php -s). The web-based PHP processes don't see it though:

var_dump(['u' => get_current_user(), 'glob' => glob( '/vagrant/settings.d/wikis/dbConf/*dbConf.php' ), 'c' => file_get_contents('/vagrant/settings.d/wikis/dbConf/10-wiki-dbConf.php') ]);

Warning: file_get_contents(/vagrant/settings.d/wikis/dbConf/10-wiki-dbConf.php): failed to open stream: No such file or directory in /var/www/w/dblist.php on line 9

array (size=3)
  'u' => string 'vagrant' (length=7)
  'glob' => 
    array (size=0)
      empty
  'c' => boolean false

Event Timeline

Tgr renamed this task from Labs-vagrant: /vagrant/settings.d/wikis/dbConf/10-wiki-dbConf.php seen by CLI PHP only to Cloud-vagrant: shared directory (/vagrant/settings.d/) seen by CLI PHP only.Oct 4 2019, 9:23 PM

Worked around via vagrant config nfs_shares false. But that's slower and NFS worked until know so it would be nice to know what's going on...

(This was for the cloud-vagrant instance running on roebling.wikispore.eqiad.wmflabs, by the way.)

Probably not related. NFS issues there were with the shared NFS servers that are used for things like $HOME in Toolforge. The NFS here would be the instance providing it's local disk to the Vagrant managed LXC container hosted on the same instance.

This sounds more like the issue we have seen in multiple places where Debian's systemd unit for Apache starts it before the Vagrant managed shared are mounted. This in turn makes lots of things look broken from mod_php's perspective. Usually restarting the apache2 process with something like vagrant ssh -- sudo service apache2 restart fixes it.

This sounds more like the issue we have seen in multiple places where Debian's systemd unit for Apache starts it before the Vagrant managed shared are mounted. This in turn makes lots of things look broken from mod_php's perspective. Usually restarting the apache2 process with something like vagrant ssh -- sudo service apache2 restart fixes it.

I think this is what we sometimes experience in T257322, and restarting apache works around. Maybe we can add a boot script which waits for the shared folders to mount?

I think this is what we sometimes experience in T257322, and restarting apache works around. Maybe we can add a boot script which waits for the shared folders to mount?

I have tried a whole lot of systemd configuration variations to get the apache2 unit to wait on the mount of /vagrant. So far none of them have been successful. In theory RequiresMountsFor=/vagrant or WantedBy=vagrant.mount should do the needful, but these do not seem to change anything in my tests. The closest I have gotten is using Restart=on-failure to keep retrying the apache2 startup until it works. This honestly might all be caused by the systemd version that is available in Debian Stretch.

We could try the complete hack from https://serverfault.com/a/814722/6479 of adding config.vm.provision "shell", run: "always", inline: "systemctl restart apache2" to our Vagrantfile. This makes me sad though because it just won't scale well. There are other services that roles optionally enable which also need to ensure that /vagrant is mounted before starting.

Change 619598 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[mediawiki/vagrant@master] apache2: wait for /vagrant/settings.d before starting

https://gerrit.wikimedia.org/r/619598

Change 619598 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[mediawiki/vagrant@master] apache2: wait for /vagrant/settings.d before starting

https://gerrit.wikimedia.org/r/619598

I think after N>50 tries I got something working. Testing and reviews welcome!