Page MenuHomePhabricator

Toolforge: introduce support for selecting grid queue release
Open, MediumPublic

Description

In the past we used to have a selector for the grid, which users could use to select the exec environment operating system.

When we migrated to Debian Stretch we mostly drop support for that, but we need it back to better handle the Stretch -> Buster migration.

Event Timeline

aborrero triaged this task as Medium priority.Mar 30 2021, 11:07 AM
aborrero created this task.
aborrero moved this task from Inbox to Doing on the cloud-services-team (Kanban) board.

Change 675733 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):
[labs/toollabs@master] Revert "jsub: Make release a deprecated noop"

https://gerrit.wikimedia.org/r/675733

Change 675734 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):
[operations/software/tools-webservice@master] Revert "Remove 'release' qsub label"

https://gerrit.wikimedia.org/r/675734

Change 675733 merged by Arturo Borrero Gonzalez:

[labs/toollabs@master] Revert "jsub: Make release a deprecated noop"

https://gerrit.wikimedia.org/r/675733

Change 675734 merged by Arturo Borrero Gonzalez:

[operations/software/tools-webservice@master] Revert "Remove 'release' qsub label"

https://gerrit.wikimedia.org/r/675734

Change 676033 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/software/tools-webservice@master] gridengine: cleaner release default

https://gerrit.wikimedia.org/r/676033

Change 676033 merged by Arturo Borrero Gonzalez:

[operations/software/tools-webservice@master] gridengine: cleaner release default

https://gerrit.wikimedia.org/r/676033

Change 676987 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[labs/toollabs@master] jsub: split_l(): the generator should return silently

https://gerrit.wikimedia.org/r/676987

Change 676989 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[labs/toollabs@master] tests/: update tests with the new default release argument

https://gerrit.wikimedia.org/r/676989

Change 676987 merged by Arturo Borrero Gonzalez:

[labs/toollabs@master] jsub: split_l(): the generator should return silently

https://gerrit.wikimedia.org/r/676987

Change 676989 merged by Arturo Borrero Gonzalez:

[labs/toollabs@master] tests/: update tests with the new default release argument

https://gerrit.wikimedia.org/r/676989

Mentioned in SAL (#wikimedia-cloud) [2021-04-06T10:21:41Z] <arturo> published jobutils & misctools 1.42 (T278748)

Is there anything left to do here, other than a tools-webservice release?

Is there anything left to do here, other than a tools-webservice release?

There is: we need to add release selection support to https://gerrit.wikimedia.org/r/plugins/gitiles/operations/software/tools-manifest/+/refs/heads/master/tools/manifest/webservicemonitor.py as well, which might need some changes in tools-webservice.

Change 713661 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/software/tools-webservice@master] Replace distro with os release

https://gerrit.wikimedia.org/r/713661

Change 713663 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/software/tools-manifest@master] Take OS codename into account for grid compatibility

https://gerrit.wikimedia.org/r/713663

Change 713663 merged by jenkins-bot:

[operations/software/tools-manifest@master] Do not compare OS versions

https://gerrit.wikimedia.org/r/713663

Change 713661 merged by jenkins-bot:

[operations/software/tools-webservice@master] Replace distro with os release

https://gerrit.wikimedia.org/r/713661

Found a related note in profile::toolforge::grid::cronrunner:

# We need to include exec environment here since the current
# version of jsub checks the local environment to find the full
# path to things before submitting them to the grid. This assumes
# that jsub is always run in an environment identical to the exec
# nodes. This is kind of terrible, so we need to fix that eventually.
# Until then...
include profile::toolforge::grid::exec_environ