Page MenuHomePhabricator

`webservice shell` sometimes loses output
Open, MediumPublicBUG REPORT

Description

Steps to replicate the issue (include links if applicable):

  • From the bastion, run something like webservice perl5.40 shell -- perl -lwe '$|=1; for($i=0; $i<3; $i++){ print "out line $i"; sleep 1; }'or webservice perl5.40 shell -- perl -lwe '$|=1; for($i=0; $i<3; $i++){ print STDERR "err line $i"; sleep 1; }'

What happens?:

Output printed often does not include the "out line 0" line.

What should have happened instead?:

All output lines should be printed.


Upstream : kubectl run -it loses some lines of the log #27264

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

webservice shell basically calls kubectl run under the hood (https://gitlab.wikimedia.org/repos/cloud/toolforge/webservice-cli/-/blob/main/toolsws/backends/kubernetes.py?ref_type=heads#L819)

Which is a bit funky at how the tty setup works (x-ref https://github.com/kubernetes/kubernetes/issues/27264).

Can you expand on your use case of needing to run a command/script?

Running stuff via webservice shell instead of on the bastion seems to be the way Toolforge's current admins intend for T360488: Missing Perl packages on dev.toolforge.org for anomiebot workflows to be resolved. I have a lot of scripts for starting, restarting, stopping, monitoring, and otherwise manipulating the jobs for the bot, because having to constantly figure out manually which jobs need to be started/stopped/etc (and the specific long command lines to properly do so) would be error-prone and time-wasting.

fgiunchedi triaged this task as Medium priority.Sep 1 2025, 2:32 PM