I thought ssh from all internal networks was still allowed in base::firewall, how come this even blocks ssh from tin?

Yeah, the firewall now does not accept connections from tin, but bast1001 is allowed. The dsh command works as documented on that wikitech page as long as it is run from tin. Perhaps updating the documentation is enough ?

@Dzahn, no, what would be the point of allowing all internal networks anyway ? We would not be achieving any kind of network separation.

@akosiaris, can we allow connections from tin, so that we can deploy from the deploy host?

Sure we can but isn't parsoid deployed via trebuchet which does not use SSH at all ?

@akosiaris, trebuchet (salt really) is not able to do rolling restarts reliably, so we are using dsh to actually apply the restart. See T63882.

Every time @ssastry has asked me to restart parsoid across the cluster I've always used salt (with a -b option) to do rolling restarts. What problems does it cause?

+1 to alex, this should be handled by the deployment setup itself.

@yuvipanda, see T63882. The problem is specific to using salt without root.

@GWicke, @ssastry, let's stall this for a while if you don't mind. As noted in T63882 the ability to add a timeout and hence fix the bug has been added in salt v2014.7.

Works for me. I'll continue to keep two shells open, one to bast1001 and another to tin and that should take care of it for deploys.

(a salt upgrade is in the works atm, thanks to @ArielGlenn)