Page MenuHomePhabricator

Provide a consistent way to identify operation in toolforge (including k8s)
Closed, ResolvedPublic

Description

I used to use os.path.exists('/etc/wmcs-instancename') to identify whether my tool was running in Toolforge or on my laptop (e.g. point redis config at tools-redis or localhost). But that file doesn't exist in the k8s webservice container. Is there a common way to figure out whether I'm running in Toolforge or not? That works across k8s, grid, etc.

Event Timeline

One solution, although its a bit indirect, is checking for /etc/novaobserver.yaml. That's not a really obvious solution, but it is on par with /etc/wmcs-instancename and actually mounted into the containers that webservice manages with the current configuration.

On a more philosophical level I'm not personally excited by the idea of code that tries to guess where it is running. I'm more inclined to suggest that configuration local to the application deployment be used for this. Part of my objection here is the idea that each special thing like this that we add to Toolforge becomes something that will someday be broken by a system update, feature deprecation, or platform change.

See T113193: Detect Wikimedia Toolforge environment in pywikibot.version - already implemented in pywikibot.

import socket
if socket.getfqdn().endswith('.tools.eqiad.wmflabs'):
        return socket.gethostname()
    return None

Can you confirm that this test works on k8s too ?

tools.zhuyifei1999-test@tools-bastion-02:~$ webservice --backend=kubernetes python shell
If you don't see a command prompt, try pressing enter.
tools.zhuyifei1999-test@interactive:~$ 
tools.zhuyifei1999-test@interactive:~$ python
Python 2.7.9 (default, Jun 29 2016, 13:08:31) 
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.getfqdn()
'interactive'

Change 431285 had a related patch set uploaded (by Zhuyifei1999; owner: Zhuyifei1999):
[operations/puppet@production] toolforge k8s: allow /etc/wmcs-project to be mounted

https://gerrit.wikimedia.org/r/431285

Change 430647 had a related patch set uploaded (by Zhuyifei1999; owner: Zhuyifei1999):
[operations/software/tools-webservice@master] Mount & load project name dynamically from /etc/wmcs-project

https://gerrit.wikimedia.org/r/430647

Change 431285 merged by Andrew Bogott:
[operations/puppet@production] toolforge k8s: allow /etc/wmcs-project to be mounted

https://gerrit.wikimedia.org/r/431285

Change 430647 merged by jenkins-bot:
[operations/software/tools-webservice@master] Mount & load project name dynamically from /etc/wmcs-project

https://gerrit.wikimedia.org/r/430647

zhuyifei1999 claimed this task.
08:51:54 0 ✓ zhuyifei1999@tools-bastion-05: ~$ become video2commons
(venv)tools.video2commons@tools-bastion-05:~$ webservice shell
Pod is not ready in time
(venv)tools.video2commons@tools-bastion-05:~$ webservice shell
If you don't see a command prompt, try pressing enter.
(venv)tools.video2commons@interactive:~$ 
(venv)tools.video2commons@interactive:~$ ls /etc/wmcs-project 
/etc/wmcs-project
(venv)tools.video2commons@interactive:~$ cat /etc/wmcs-project 
tools