Page MenuHomePhabricator

Toolforge, version of jq varying
Closed, ResolvedPublic

Description

I had some tools that sometimes worked and sometimes didn't. After digging into it I found that some of the hosts of toolforge have jq version 1.3 installed, while others have jq version 1.4 installed.
So when I submit a job with jsub sometimes it's executed on a host with jq 1.3 and sometimes on a host with jq 1.4. Both are incompatible to each other. So a script designed to work with 1.4 might not run with 1.3 and vice versa.

Is it possible to update to jq 1.4 everywhere?

How to reproduce

$ cat test.sh
#!/bin/bash
jq -V >> testfile
$ /usr/bin/jsub -N test1 -once -quiet path/to/test.sh
$ /usr/bin/jsub -N test2 -once -quiet path/to/test.sh
$ /usr/bin/jsub -N test3 -once -quiet path/to/test.sh
# ...

will get something like

jq-1.4-1-e73951f
jq version 1.3
jq-1.4-1-e73951f
jq-1.4-1-e73951f
jq version 1.3

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 23 2018, 7:09 PM

For some reason on some hosts the latest version isn't the candidate:

07:25:52 1 ✗ zhuyifei1999@tools-bastion-02: ~$ for i in tools-exec-14{01..42}; do echo -n $i; ssh $i 'apt-cache policy jq' | grep Installed; done
tools-exec-1401  Installed: 1.4-2.1~ubuntu14.04.1
tools-exec-1402  Installed: 1.4-2.1~ubuntu14.04.1
tools-exec-1403  Installed: 1.4-2.1~ubuntu14.04.1
tools-exec-1404  Installed: 1.4-2.1~ubuntu14.04.1
tools-exec-1405  Installed: 1.4-2.1~ubuntu14.04.1
tools-exec-1406  Installed: 1.4-2.1~ubuntu14.04.1
tools-exec-1407  Installed: 1.4-2.1~ubuntu14.04.1
tools-exec-1408  Installed: 1.4-2.1~ubuntu14.04.1
tools-exec-1409  Installed: 1.4-2.1~ubuntu14.04.1
tools-exec-1410  Installed: 1.4-2.1~ubuntu14.04.1
tools-exec-1411  Installed: 1.3-1.1ubuntu1
tools-exec-1412  Installed: 1.4-2.1~ubuntu14.04.1
tools-exec-1413  Installed: 1.3-1.1ubuntu1
tools-exec-1414  Installed: 1.3-1.1ubuntu1
tools-exec-1415  Installed: 1.3-1.1ubuntu1
tools-exec-1416  Installed: 1.4-2.1~ubuntu14.04.1
tools-exec-1417  Installed: 1.3-1.1ubuntu1
tools-exec-1418  Installed: 1.3-1.1ubuntu1
tools-exec-1419  Installed: 1.3-1.1ubuntu1
tools-exec-1420  Installed: 1.4-2.1~ubuntu14.04.1
tools-exec-1421  Installed: 1.3-1.1ubuntu1
tools-exec-1422  Installed: 1.3-1.1ubuntu1
tools-exec-1423  Installed: 1.3-1.1ubuntu1
tools-exec-1424  Installed: 1.3-1.1ubuntu1
tools-exec-1425  Installed: 1.3-1.1ubuntu1
tools-exec-1426  Installed: 1.3-1.1ubuntu1
tools-exec-1427  Installed: 1.3-1.1ubuntu1
tools-exec-1428  Installed: 1.3-1.1ubuntu1
tools-exec-1429  Installed: 1.3-1.1ubuntu1
tools-exec-1430  Installed: 1.3-1.1ubuntu1
tools-exec-1431  Installed: 1.3-1.1ubuntu1
tools-exec-1432  Installed: 1.3-1.1ubuntu1
tools-exec-1433  Installed: 1.3-1.1ubuntu1
tools-exec-1434  Installed: 1.3-1.1ubuntu1
tools-exec-1435  Installed: 1.3-1.1ubuntu1
tools-exec-1436  Installed: 1.3-1.1ubuntu1
tools-exec-1437  Installed: 1.3-1.1ubuntu1
tools-exec-1438  Installed: 1.3-1.1ubuntu1
tools-exec-1439  Installed: 1.3-1.1ubuntu1
tools-exec-1440  Installed: 1.3-1.1ubuntu1
tools-exec-1441  Installed: 1.3-1.1ubuntu1
tools-exec-1442  Installed: 1.3-1.1ubuntu1
07:26:36 0 ✓ zhuyifei1999@tools-bastion-02: ~$ ssh tools-exec-1442 'apt-cache policy jq'
jq:
  Installed: 1.3-1.1ubuntu1
  Candidate: 1.3-1.1ubuntu1
  Version table:
     1.4-2.1~ubuntu14.04.1 0
        100 http://nova.clouds.archive.ubuntu.com/ubuntu/ trusty-backports/universe amd64 Packages
 *** 1.3-1.1ubuntu1 0
        500 http://nova.clouds.archive.ubuntu.com/ubuntu/ trusty/universe amd64 Packages
        100 /var/lib/dpkg/status

@aborrero Could you look into this?

aborrero claimed this task.Apr 30 2018, 9:47 AM
aborrero closed this task as Resolved.Apr 30 2018, 9:52 AM

Done!

aborrero@tools-clushmaster-01:~$ clush -w@exec-trusty "dpkg -s jq | grep -i version"
tools-exec-1411.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1413.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1412.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1416.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1415.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1402.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1404.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1414.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1429.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1430.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1432.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1431.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1433.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1434.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1417.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1405.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1418.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1428.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1435.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1436.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1409.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1437.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1410.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1420.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1421.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1406.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1438.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1419.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1403.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1423.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1408.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1440.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1401.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1424.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1439.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1407.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1442.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1441.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1426.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1422.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1427.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
tools-exec-1425.tools.eqiad.wmflabs: Version: 1.4-2.1~ubuntu14.04.1
Vvjjkkii renamed this task from Toolforge, version of jq varying to teeaaaaaaa.Jul 1 2018, 1:14 AM
Vvjjkkii reopened this task as Open.
Vvjjkkii triaged this task as High priority.
Vvjjkkii removed aborrero as the assignee of this task.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
AfroThundr3007730 renamed this task from teeaaaaaaa to Toolforge, version of jq varying.Jul 1 2018, 6:09 AM
AfroThundr3007730 closed this task as Resolved.
AfroThundr3007730 raised the priority of this task from High to Needs Triage.
AfroThundr3007730 assigned this task to aborrero.
AfroThundr3007730 updated the task description. (Show Details)
AfroThundr3007730 added a subscriber: Aklapper.