Page MenuHomePhabricator

Cumin: shell expression requires variables to be escaped
Closed, InvalidPublic

Description

I wanted to get the list of mounted partition by getting the sixth field of df using awk:

$ df|awk '{ print $6}'
Mounted
/dev
/run
/
/dev/shm
/run/lock
/sys/fs/cgroup

With cumin the whole line is printed:

sudo cumin --trace --force 'name:1053 'df| awk "{ print $6}"'
----- OUTPUT of 'df| awk "{ print $6}"' -----                                                                                                                                                                      
Filesystem                         1K-blocks    Used Available Use% Mounted on                                                                                                                                     
udev                                   10240       0     10240   0% /dev                                                                                                                                           
tmpfs                                 810132   82332    727800  11% /run                                                                                                                                           
/dev/vda3                           19479208 3585896  14880784  20% /                                                                                                                                              
tmpfs                                2025328       0   2025328   0% /dev/shm                                                                                                                                       
tmpfs                                   5120       0      5120   0% /run/lock                                                                                                                                      
tmpfs                                2025328       0   2025328   0% /sys/fs/cgroup                                                                                                                                 
/dev/mapper/vd-second--local--disk  21023324  834336  19098020   5% /srv                                                                                                                                           
================

And in the cumin log:

INFO cumin.cli.main] Cumin called by user 'hashar' with args:
Namespace(backend=None,
    batch_size={'value': None,
    'ratio': None},
    batch_sleep=None,
    commands=['df| awk "{ print $6}"'],
    config='/etc/cumin/config.yaml',
    debug=False,
    dry_run=False,
    force=True,
    global_timeout=None,
    hosts='name:docker-1053',
    ignore_exit_codes=False,
    interactive=False,
    mode='sync',
    output=None,
    success_percentage=100,
    timeout=None,
    trace=True,
    transport=None
)
INFO cumin.transports.clustershell.ClusterShellWorker.execute]
Executing commands [cumin.transports.Command('df| awk "{ print $6}"')]
on '1' hosts: integration-slave-docker-1053.integration.eqiad.wmflabs

I am not sure what happens to $6, but it seems to be expanded by the shell. I guess because it executes literally: df | awk "{ print $6}", the variable being in double quote, it gets expanded to an empty string.

If switch double and single quotes with: sudo cumin --trace --force 'name:docker-1053' "df| awk '{ print $6}'", the variable is not passed at all. From the output:

100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'df| awk '{ print }''.

In cumin.log:

commands=["df| awk '{ print }'"]

The not so straightforward way is to backslash the variable: 'df| awk "{ print \$6}"'.

That is with cumin 3.0.2-2

Event Timeline

hashar created this task.Apr 15 2019, 8:05 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 15 2019, 8:05 PM
hashar updated the task description. (Show Details)Apr 15 2019, 8:05 PM
Volans closed this task as Resolved.Apr 15 2019, 8:09 PM
Volans claimed this task.
Volans triaged this task as Normal priority.

Use "df | awk '{ print \$6}'"

Volans changed the task status from Resolved to Invalid.Apr 15 2019, 8:18 PM

Same with ssh:

$ ssh integration-slave-docker-1053.integration.eqiad.wmflabs "echo '$1'"

$

I don't quite know why it gets expanded though :/ Left to be figured out in another life.

@hashar this has nothing to do with Cumin but the local bash on the Cumin master.
If you use double quotes the bash interprets what's inside and replaces any variable with their value. If a variable is not defined it replaces it with empty string, so $foo will be replaced by empty string e Cumin will receive as parameter echo ''.

If instead you use single quotes outside, the content is passed verbatim to Cumin that executes it verbatim on the target hosts via ssh, hitting the remote bash on the target hosts. So executing 'echo "$foo"' will pass to Cumin echo "$foo" and when executing on the remote bash via ssh $foo will be replaced with the value for that variable on the remote bash, if defined. If not defined it will become again an empty string.
That's why the easiest way to pass a bash variable without being interpolated with its value is to use locally double quotes but escaping the $ sign to have the bash pass it verbatim to Cumin.