The Cirrus elasticsearch clusters are running Trusty. The logstash cluster is already on the newer Jessie and if I understand correctly, we are trying to standardise on Jessie. Migrating to Jessie should be mostly painless (the logstash cluster seems to be doing just fine) but we need some level of testing to validate that we don't have subtle differences on the search cluster.
Description
Details
Status | Subtype | Assigned | Task | ||
---|---|---|---|---|---|
Resolved | debt | T151324 [epic] System level upgrade for cirrus / elasticsearch | |||
Resolved | Gehel | T151326 Upgrade cirrus / elasticsearch to Jessie | |||
Resolved | • Cmjohnson | T156663 Degraded RAID on relforge1001 |
Event Timeline
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2009.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702081705_gehel_6415.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2011.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702081710_gehel_7774.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2010.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702081711_gehel_8116.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2012.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702081727_gehel_13072.log.
Completed auto-reimage of hosts:
['elastic2009.codfw.wmnet']
and were ALL successful.
Completed auto-reimage of hosts:
['elastic2011.codfw.wmnet']
Of which those FAILED:
set(['elastic2011.codfw.wmnet'])
Completed auto-reimage of hosts:
['elastic2010.codfw.wmnet']
and were ALL successful.
Completed auto-reimage of hosts:
['elastic2012.codfw.wmnet']
Of which those FAILED:
set(['elastic2012.codfw.wmnet'])
Change 336679 had a related patch set uploaded (by Gehel):
elasticsearch - reimage elastic20(13|14|15|16) to jessie and move data to /srv
Change 336679 merged by Gehel:
elasticsearch - reimage elastic20(13|14|15|16) to jessie and move data to /srv
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2013.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702082057_gehel_5758.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2014.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702082058_gehel_5826.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2015.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702082058_gehel_5839.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2016.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702082058_gehel_5854.log.
Completed auto-reimage of hosts:
['elastic2013.codfw.wmnet']
and were ALL successful.
Completed auto-reimage of hosts:
['elastic2014.codfw.wmnet']
and were ALL successful.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2015.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702082140_gehel_17779.log.
Completed auto-reimage of hosts:
['elastic2015.codfw.wmnet']
Of which those FAILED:
set(['elastic2015.codfw.wmnet'])
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2016.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702082226_gehel_31103.log.
Change 336765 had a related patch set uploaded (by Gehel):
elasticsearch - reimage elastic20(17|18|19|20) to jessie and move data to /srv
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2017.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702090934_gehel_20928.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2018.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702090936_gehel_21386.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2019.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702090936_gehel_21456.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2020.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702090936_gehel_21552.log.
Change 336765 merged by Gehel:
elasticsearch - reimage elastic20(17|18|19|20) to jessie and move data to /srv
Completed auto-reimage of hosts:
['elastic2017.codfw.wmnet']
Of which those FAILED:
set(['elastic2017.codfw.wmnet'])
Completed auto-reimage of hosts:
['elastic2020.codfw.wmnet']
and were ALL successful.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2018.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702091139_gehel_27767.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2019.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702091247_gehel_14629.log.
Completed auto-reimage of hosts:
['elastic2019.codfw.wmnet']
Of which those FAILED:
set(['elastic2019.codfw.wmnet'])
Change 336795 had a related patch set uploaded (by Gehel):
elasticsearch - reimage elastic20(21|22|23|24) to jessie and move data to /srv
Change 336795 merged by Gehel:
elasticsearch - reimage elastic20(21|22|23|24) to jessie and move data to /srv
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2021.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702091402_gehel_3926.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2022.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702091403_gehel_4080.log.
Completed auto-reimage of hosts:
['elastic2021.codfw.wmnet']
and were ALL successful.
Completed auto-reimage of hosts:
['elastic2022.codfw.wmnet']
and were ALL successful.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2024.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702091533_gehel_30735.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2023.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702091544_gehel_1612.log.
Completed auto-reimage of hosts:
['elastic2024.codfw.wmnet']
Of which those FAILED:
set(['elastic2024.codfw.wmnet'])
Completed auto-reimage of hosts:
['elastic2023.codfw.wmnet']
and were ALL successful.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2018.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702091637_gehel_18129.log.
Completed auto-reimage of hosts:
['elastic2018.codfw.wmnet']
and were ALL successful.
Change 336973 had a related patch set uploaded (by Gehel):
elasticsearch - reimage elastic20(25|26|27|28) to jessie and move data to /srv
Change 336973 merged by Gehel:
elasticsearch - reimage elastic20(25|26|27|28) to jessie and move data to /srv
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2025.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702100857_gehel_26722.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2026.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702100858_gehel_26938.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2028.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702100858_gehel_27063.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2027.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702100858_gehel_27019.log.
Completed auto-reimage of hosts:
['elastic2026.codfw.wmnet']
and were ALL successful.
Completed auto-reimage of hosts:
['elastic2025.codfw.wmnet']
and were ALL successful.
Completed auto-reimage of hosts:
['elastic2028.codfw.wmnet']
Of which those FAILED:
set(['elastic2028.codfw.wmnet'])
Completed auto-reimage of hosts:
['elastic2027.codfw.wmnet']
Of which those FAILED:
set(['elastic2027.codfw.wmnet'])
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2027.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702101419_gehel_20955.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2028.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702101420_gehel_21237.log.
Completed auto-reimage of hosts:
['elastic2027.codfw.wmnet']
and were ALL successful.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2028.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702101450_gehel_29598.log.
Completed auto-reimage of hosts:
['elastic2028.codfw.wmnet']
Of which those FAILED:
set(['elastic2028.codfw.wmnet'])
Change 337060 had a related patch set uploaded (by Gehel):
elasticsearch - reimage elastic20(29|30|31|32) to jessie and move data to /srv
Change 337060 merged by Gehel:
elasticsearch - reimage elastic20(29|30|31|32) to jessie and move data to /srv
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2029.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702101830_gehel_27474.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2030.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702101830_gehel_27486.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2031.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702101830_gehel_27720.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2032.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702101831_gehel_27761.log.
Completed auto-reimage of hosts:
['elastic2032.codfw.wmnet']
Of which those FAILED:
set(['elastic2032.codfw.wmnet'])
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2032.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702101844_gehel_30908.log.
Completed auto-reimage of hosts:
['elastic2029.codfw.wmnet']
and were ALL successful.
Completed auto-reimage of hosts:
['elastic2032.codfw.wmnet']
and were ALL successful.
Completed auto-reimage of hosts:
['elastic2030.codfw.wmnet']
Of which those FAILED:
set(['elastic2030.codfw.wmnet'])
Completed auto-reimage of hosts:
['elastic2031.codfw.wmnet']
and were ALL successful.
Change 337218 had a related patch set uploaded (by Gehel):
elasticsearch - reimage elastic20(33|34|35|36) to jessie and move data to /srv
Change 337218 merged by Gehel:
elasticsearch - reimage elastic20(33|34|35|36) to jessie and move data to /srv
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2034.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702110916_gehel_10144.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2033.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702110915_gehel_10106.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2035.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702110916_gehel_10291.log.
Script wmf_auto_reimage was launched by gehel on neodymium.eqiad.wmnet for hosts:
['elastic2036.codfw.wmnet']
The log can be found in /var/log/wmf-auto-reimage/201702110916_gehel_10342.log.
Completed auto-reimage of hosts:
['elastic2033.codfw.wmnet']
Of which those FAILED:
set(['elastic2033.codfw.wmnet'])
Completed auto-reimage of hosts:
['elastic2036.codfw.wmnet']
and were ALL successful.
Completed auto-reimage of hosts:
['elastic2035.codfw.wmnet']
and were ALL successful.
Completed auto-reimage of hosts:
['elastic2034.codfw.wmnet']
and were ALL successful.
Change 337378 had a related patch set uploaded (by Gehel):
elasticsearch - reimage to jessie and move data to /srv - preliminary work
Change 337378 merged by Gehel:
elasticsearch - reimage to jessie and move data to /srv - preliminary work
Change 338327 had a related patch set uploaded (by Gehel):
elasticsearch - reimage elastic10(17|18|19|20) to jessie and move data to /srv
Change 338327 merged by Gehel:
elasticsearch - reimage elastic10(17|18|19|20) to jessie and move data to /srv
Change 338392 had a related patch set uploaded (by Gehel):
elasticsearch - reimage elastic10(21|22|23|24) to jessie and move data to /srv
Change 338392 merged by Gehel:
elasticsearch - reimage elastic10(21|22|23|24) to jessie and move data to /srv
Change 338761 had a related patch set uploaded (by Gehel):
elasticsearch - reimage elastic10(25|28|29|30) to jessie and move data to /srv
Change 338761 merged by Gehel:
elasticsearch - reimage elastic10(25|28|29|30) to jessie and move data to /srv
Change 338768 had a related patch set uploaded (by Gehel):
elasticsearch - reimage elastic10(26|31|36|40) to jessie and move data to /srv
Change 338768 merged by Gehel:
elasticsearch - reimage elastic10(26|31|36|40) to jessie and move data to /srv
Change 338811 had a related patch set uploaded (by Gehel):
elasticsearch - reimage elastic10(27|32|37|41) to jessie and move data to /srv
Change 338811 merged by Gehel:
elasticsearch - reimage elastic10(27|32|37|41) to jessie and move data to /srv
Change 338977 had a related patch set uploaded (by Gehel):
elasticsearch - reimage elastic10(33|34|38|42) to jessie and move data to /srv
Change 338977 merged by Gehel:
elasticsearch - reimage elastic10(33|34|38|42) to jessie and move data to /srv
Change 339017 had a related patch set uploaded (by Gehel):
elasticsearch - reimage elastic10(35|39|43|44) to jessie and move data to /srv
Change 339017 merged by Gehel:
elasticsearch - reimage elastic10(35|39|43|44) to jessie and move data to /srv
Change 339130 had a related patch set uploaded (by Gehel):
elasticsearch - reimage elastic10(45|46|47) to jessie and move data to /srv
Change 339130 merged by Gehel:
elasticsearch - reimage elastic10(45|46|47) to jessie and move data to /srv