Page MenuHomePhabricator

Beta cluster puppetmaster lost cherry picked patches due to disk corruption
Closed, ResolvedPublic

Description

Due to labvirt1007 having a full disk, we have lost the cherry picked patches on deployment-puppetmaster. I have reset the git repo to origin/production but we should recover cherry picked patches.

Candidates are potentially dangling commits:

root@deployment-puppetmaster:/var/lib/git/operations/puppet# git fsck
Checking object directories: 100% (256/256), done.
Checking objects: 100% (187017/187017), done.
dangling commit 505570b8bc36136d40753d7b0601d1a2df198454
dangling commit c19430dcc99ac327857b1b3b3ed66c7d2971b0c9
dangling commit 92ba48b7f0e2d4c014043db316b76a7fcc2da7bf
dangling commit d217d9a32934450200d0a29bf0f2a646b22efd94
dangling commit 852fb92ca3af5b9481099b28cd6af9c7992f0069
dangling blob 699a01f65c3680b48a45232d15475dd7dd10e8ec
dangling blob 24f8996eca7201b6c83e773631cd2215a2d8f976
dangling commit fd292ab1969ec30edeb6b5897407e9cdfb5da9e7
dangling blob f70eac79141a2e035eeebdb00c50e9132f5e8cf4
dangling blob 0c225c21cef2f91f04dd554cbde2141f6ad3e822
dangling commit 7c02b57d6c64690cea43154fa9a8df75e19c2e70
dangling blob 7aba5505e728ab1be9ec36a9552765a9ebad2a7b
dangling commit 2dd035b56d3af7bb60ace55658b761504ce0f76d
dangling commit 7404b68ebf264f7692d810afd13f0022c182d641
dangling commit 7f75a6effd4ab012aa7fda12f900f23f1eb5a900
dangling commit ed12a7f658fe9d33a45e7e573bd3966354d5ea79
dangling blob 4029df429328e06376f92877543e31e38c1b2fef
dangling commit 39d10f9c39dab05c421e6d330f114f64d928f571
root@deployment-puppetmaster:/var/lib/git/operations/puppet#
for i in 2dd035b56d3af7bb60ace55658b761504ce0f76d 39d10f9c39dab05c421e6d330f114f64d928f571 505570b8bc36136d40753d7b0601d1a2df198454 7404b68ebf264f7692d810afd13f0022c182d641 7c02b57d6c64690cea43154fa9a8df75e19c2e70 7f75a6effd4ab012aa7fda12f900f23f1eb5a900 852fb92ca3af5b9481099b28cd6af9c7992f0069 92ba48b7f0e2d4c014043db316b76a7fcc2da7bf c19430dcc99ac327857b1b3b3ed66c7d2971b0c9 d217d9a32934450200d0a29bf0f2a646b22efd94 ed12a7f658fe9d33a45e7e573bd3966354d5ea79 fd292ab1969ec30edeb6b5897407e9cdfb5da9e7; do echo "Checking $i"; git cherry -v production $i; done;
Checking 2dd035b56d3af7bb60ace55658b761504ce0f76d
+ 2dd035b56d3af7bb60ace55658b761504ce0f76d Move web::sites to web::prod_sites; begin unification in new class
Checking 39d10f9c39dab05c421e6d330f114f64d928f571
+ 4802114333fd1dc29183845a2128343a05ef85f4 logstash: Count MediaWiki log events with statsd
+ 39d10f9c39dab05c421e6d330f114f64d928f571 logstash: normalize "level" fields across log types
Checking 505570b8bc36136d40753d7b0601d1a2df198454
- c2897c687f48a38a85f352200001cffe293e4433 [WIP] Update configuration for logstash 1.5.3
+ 505570b8bc36136d40753d7b0601d1a2df198454 labs: new role::logstash::stashbot class
Checking 7404b68ebf264f7692d810afd13f0022c182d641
- 7404b68ebf264f7692d810afd13f0022c182d641 beta: copy prod bits apache config
Checking 7c02b57d6c64690cea43154fa9a8df75e19c2e70
- 7c02b57d6c64690cea43154fa9a8df75e19c2e70 beta: Disable authentication for Kibana
Checking 7f75a6effd4ab012aa7fda12f900f23f1eb5a900
+ 7f75a6effd4ab012aa7fda12f900f23f1eb5a900 logstash: normalize "level" fields across log types
Checking 852fb92ca3af5b9481099b28cd6af9c7992f0069
- 852fb92ca3af5b9481099b28cd6af9c7992f0069 RESTBase: Add MobileApps endpoints and back-end config
Checking 92ba48b7f0e2d4c014043db316b76a7fcc2da7bf
- 92ba48b7f0e2d4c014043db316b76a7fcc2da7bf logstash: normalize "level" fields across log types
Checking c19430dcc99ac327857b1b3b3ed66c7d2971b0c9
- c19430dcc99ac327857b1b3b3ed66c7d2971b0c9 Followup Ibd58670e: Also set auto_create_index for beta logstash
Checking d217d9a32934450200d0a29bf0f2a646b22efd94
+ d217d9a32934450200d0a29bf0f2a646b22efd94 beta: varnish backend/director for isolated security audits
Checking ed12a7f658fe9d33a45e7e573bd3966354d5ea79
- c3b87bfa93d056e5774c3253cda7c788f3ec03a4 Update configuration for logstash 1.5.3
- 73037ea837afd8c7d052752d3df408178dce3b9a labs: new role::logstash::stashbot class
+ ed12a7f658fe9d33a45e7e573bd3966354d5ea79 logstash: Count MediaWiki log events with statsd
Checking fd292ab1969ec30edeb6b5897407e9cdfb5da9e7
- 56c6664b70041421a12d3ac2fed5dde92808c824 Update configuration for logstash 1.5.3
+ fd292ab1969ec30edeb6b5897407e9cdfb5da9e7 labs: new role::logstash::stashbot class
root@deployment-puppetmaster:/var/lib/git/operations/puppet#

Event Timeline

hashar created this task.Aug 26 2015, 7:31 AM
hashar raised the priority of this task from to Unbreak Now!.
hashar updated the task description. (Show Details)
hashar added subscribers: hashar, Aklapper.
Restricted Application added a subscriber: Luke081515. · View Herald TranscriptAug 26 2015, 7:31 AM
hashar added a subscriber: demon.Aug 26 2015, 7:37 AM

The only lost patch is apparently https://gerrit.wikimedia.org/r/#/c/197655/ Move web::sites to web::prod_sites; begin unification in new class by @demon. I haven't reapplied it since it is in merge conflict.

hashar closed this task as Resolved.Aug 26 2015, 7:38 AM
hashar claimed this task.

Seems all good now