deployment-tmh01.deployment-prep.eqiad.wmflabs refuses mwdeploy ssh connection
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	hashar
	Apr 27 2016, 9:50 AM

Description

https://integration.wikimedia.org/ci/job/beta-scap-eqiad/ broke because deployment-tmh01.deployment-prep.eqiad.wmflabs refuses mwdeploy ssh connection

That broke between on April 27th between 08:24:48 UTC and 08:34:45 UTC

From the puppet.log on that instance:

Notice: /Stage[main]/Mediawiki::Users/Ssh::Userkey[mwdeploy]/File[/etc/ssh/userkeys/mwdeploy]/content: 
--- /etc/ssh/userkeys/mwdeploy  2016-04-26 23:01:26.102259724 +0000
+++ /tmp/puppet-file20160427-27066-w4emgf       2016-04-27 08:31:23.205921570 +0000
@@ -1 +1 @@
-ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCtTt7udU8YI057E6lnIVrdL7eV0/SFLirFI0/iWkv3/M6Kh3P6zepWu/E0oujbw+DYLhtTY9Nh14UTTp7UE+TXNDmAtM
aRpAd8FzZQnQ8pzo+PO1AyBGh8necJh9kZpnEcuA23r59SWHQjMndqVGAoW90Aydd2lHWkMXXExGlsGuH8cdpF3+GOUsHAnodhXhDluvURFsEJwogy1Gfw25qgg0F3yfkBt
PapVCyNXvOMDA5w4KAbjTAHpEKdP0ep098GD4QzWigyfCVT+uimHXVtQTL8KwyYz8iPESEFon0bO79rFtA/ZHq7dDbFC0lNEhWiVj+1sCV0B+Ba5AUlj9QGDDC7oz1+vsfG
in/LyHk2U2n7Nus0K99F8gCxm3o5Xj9gMhsYOm5wja8FL31NEz1KNsLQggD5O2dTKokMCMP3D4SsN7zot6CaayQBAhtxycWiZJXPSx4ZKXoZqsKRZJ63qzSonMJVaPqPa44Cc1AwxEkMfJ59Jhgpw6mj+4Rz7CEbnzGPnGoeg+anreeww+af9ojqOFasJrucEMJR0QyeuuaRHheeMAU5v/R+uCyLnPbFQLl0kgT16gFx6wYGoX3K3XwRNrGATEJHEHAt2IEJOdoGFNgHFkHVTJUAiJ8nGMDtjsLxfRIlCF/P+seU9kXdAnBBqGkju7v2uvtMI1oseQ==
\ No newline at end of file
+ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFwlmBBBJAr1GI+vuYjFh5vq0YIVa5fqE5DZdpzUZISlQ0Kt+9bIr2qNHIj+Jl5Bc6ZY1mkh8l693tAHVx+8tayoiFWYNs9IVsxR+iHgOOhAdDIBXaHaUattdiye5bQmdvJVXaVegckNX2gbmUCOc09jvZvlk3blKFTSEpZRU8dmpXQzKdZgaAq2VTajAegoFnuN9FbC7hzBPA+1NxFNKn94eIeFPSlo5rWr44OEb5Uy3O0B5c6WPM+IgfiygetP+yGL4cKv7qEjZ0Sxok/Rh1lBh1vP1YQ/Mc6tMV0s+kOv7Wz+P88bfU1/uWvy479OZdfh3NQqDTrLzqHwVW1vef root@deployment-salt
\ No newline at end of file

Notice: /Stage[main]/Mediawiki::Users/Ssh::Userkey[mwdeploy]/File[/etc/ssh/userkeys/mwdeploy]/owner: owner changed 'mwdeploy' to 'root'
Notice: /Stage[main]/Mediawiki::Users/Ssh::Userkey[mwdeploy]/File[/etc/ssh/userkeys/mwdeploy]/mode: mode changed '0644' to '0444'

And later:

Notice: /Stage[main]/Mediawiki::Users/Ssh::Userkey[mwdeploy]/File[/etc/ssh/userkeys/mwdeploy.d/mwdeploy]/ensure: created
Info: /Stage[main]/Mediawiki::Users/Ssh::Userkey[mwdeploy]/File[/etc/ssh/userkeys/mwdeploy.d/mwdeploy]: Scheduling refresh of Exec[compile_ssh_userkeys_mwdeploy]
Notice: /Stage[main]/Mediawiki::Users/Ssh::Userkey[mwdeploy]/Exec[compile_ssh_userkeys_mwdeploy]: Triggered 'refresh' from 1 events
Notice: /Stage[main]/Mediawiki::Users/Ssh::Userkey[mwdeploy]/File[/etc/ssh/userkeys/mwdeploy]/owner: owner changed 'root' to 'mwdeploy'
Notice: /Stage[main]/Mediawiki::Users/Ssh::Userkey[mwdeploy]/File[/etc/ssh/userkeys/mwdeploy]/mode: mode changed '0444' to '0644'

Probably related to T132747

Related Objects

Mentioned In: T132747: scap::target shouldn't allow users to redefine the user's key
Mentioned Here: T132747: scap::target shouldn't allow users to redefine the user's key

Event Timeline

hashar created this task.Apr 27 2016, 9:50 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 27 2016, 9:50 AM

hashar mentioned this in T132747: scap::target shouldn't allow users to redefine the user's key.Apr 27 2016, 9:51 AM

I'm getting an odd error:

Error: Could not find command 'for'
Error: /Stage[main]/Mediawiki::Users/Ssh::Userkey[mwdeploy]/Exec[compile_ssh_userkeys_mwdeploy]/returns: change from notrun to 0 failed: Could not find command 'for'

it's supposed to be a bash for loop.

From I4086a12896e7e22004402dd0bc025896c037c746 for T132747 which is cherry picked on puppetmaster:

+        exec { "compile_ssh_userkeys_${user}":
+            path    => '/usr/bin:/bin',
+            command => "for f in ${userkeys_d}/*; do (cat \"\${f}\"; echo) ; done >${userkeys}",
+            creates => $userkeys,
+            user    => $user,
+        }

for is not a command but a shell builtin. So you want to do something like bash -c 'for ....' ?

• mmodell closed this task as Resolved.Apr 29 2016, 10:11 PM

• mmodell claimed this task.

deployment-tmh01.deployment-prep.eqiad.wmflabs refuses mwdeploy ssh connectionClosed, ResolvedPublicActions

Description

Related Objects

Event Timeline

deployment-tmh01.deployment-prep.eqiad.wmflabs refuses mwdeploy ssh connection
Closed, ResolvedPublic
Actions