Change Details

Past incidents in phab: T123924 T137647 T159870 (also observed on acamar today, no separate task for that) acpid_pad continues to occasionally cause problems, usually fixable by rmmod of the module with no apparent other ill effects in the general case. Dell has some older documentation that talks about blacklisting acpi_pad and a few other modules here: http://en.community.dell.com/techcenter/b/techcenter/archive/2012/08/27/ubuntu-on-dell-12g-poweredge-servers . I don't think we should blindly follow that advice, as it was written in the context of both older kernels and older servers, but I think at this point it might be reasonable for us to universally blacklist this module in base (or disable it on the kernel cmdline, if that's what's necessary to stop it from loading). --- list of Dell PowerEdge 320 hosts: (cumin/facter:` 'F:productname ~ "^PowerEdge R320"'`) (check box if firmware has been upgraded) [x] acamar.wikimedia.org [x] achernar.wikimedia.org [] baham.wikimedia.org [x] bast2001.wikimedia.org [x] cerium.eqiad.wmnet [x] heze.codfw.wmnet [x] labservices1002.wikimedia.org [x] logstash[1001-1003].eqiad.wmnet (declined, decom) [x] mira.codfw.wmnet (decom) [x] praseodymium.eqiad.wmnet [x] subra.codfw.wmnet (decom) [x] suhail.codfw.wmnet (decom) [x] tin.eqiad.wmnet [x] xenon.eqiad.wmnet

Past incidents in phab: T123924 T137647 T159870 (also observed on acamar today, no separate task for that) acpid_pad continues to occasionally cause problems, usually fixable by rmmod of the module with no apparent other ill effects in the general case. Dell has some older documentation that talks about blacklisting acpi_pad and a few other modules here: http://en.community.dell.com/techcenter/b/techcenter/archive/2012/08/27/ubuntu-on-dell-12g-poweredge-servers . I don't think we should blindly follow that advice, as it was written in the context of both older kernels and older servers, but I think at this point it might be reasonable for us to universally blacklist this module in base (or disable it on the kernel cmdline, if that's what's necessary to stop it from loading). --- list of Dell PowerEdge 320 hosts: (cumin/facter:` 'F:productname ~ "^PowerEdge R320"'`) (check box if firmware has been upgraded) [x] acamar.wikimedia.org [x] achernar.wikimedia.org [x] baham.wikimedia.org [x] bast2001.wikimedia.org [x] cerium.eqiad.wmnet [x] heze.codfw.wmnet [x] labservices1002.wikimedia.org [x] logstash[1001-1003].eqiad.wmnet (declined, decom) [x] mira.codfw.wmnet (decom) [x] praseodymium.eqiad.wmnet [x] subra.codfw.wmnet (decom) [x] suhail.codfw.wmnet (decom) [x] tin.eqiad.wmnet [x] xenon.eqiad.wmnet

Past incidents in phab: T123924 T137647 T159870 (also observed on acamar today, no separate task for that) acpid_pad continues to occasionally cause problems, usually fixable by rmmod of the module with no apparent other ill effects in the general case. Dell has some older documentation that talks about blacklisting acpi_pad and a few other modules here: http://en.community.dell.com/techcenter/b/techcenter/archive/2012/08/27/ubuntu-on-dell-12g-poweredge-servers . I don't think we should blindly follow that advice, as it was written in the context of both older kernels and older servers, but I think at this point it might be reasonable for us to universally blacklist this module in base (or disable it on the kernel cmdline, if that's what's necessary to stop it from loading). --- list of Dell PowerEdge 320 hosts: (cumin/facter:` 'F:productname ~ "^PowerEdge R320"'`) (check box if firmware has been upgraded) [x] acamar.wikimedia.org [x] achernar.wikimedia.org [x] baham.wikimedia.org [x] bast2001.wikimedia.org [x] cerium.eqiad.wmnet [x] heze.codfw.wmnet [x] labservices1002.wikimedia.org [x] logstash[1001-1003].eqiad.wmnet (declined, decom) [x] mira.codfw.wmnet (decom) [x] praseodymium.eqiad.wmnet [x] subra.codfw.wmnet (decom) [x] suhail.codfw.wmnet (decom) [x] tin.eqiad.wmnet [x] xenon.eqiad.wmnet