Change Details

#### Current status **Current status:** - **Deployed mitigation** (via reverting to old checkpoint as well as https://gerrit.wikimedia.org/r/c/wikidata/query/rdf/+/764864, deployed in `0.3.104`) - **root cause still unknown** - leaving **wdqs@codfw depooled** through the night to buy more time to investigate #### What's left to do? [] Draft lightweight incident report (see form at https://wikitech.wikimedia.org/wiki/Incident_status; must be logged in to wikitech to see the form) [] Identify root cause of failure (currently we only have a mitigation in place) [] (after fixing root cause) re-enable traffic to codfw && revert patch for puppet metrics change: https://gerrit.wikimedia.org/r/c/operations/puppet/+/764830/

#### Current status **Current status:** - **Deployed mitigation** (via reverting to old checkpoint as well as https://gerrit.wikimedia.org/r/c/wikidata/query/rdf/+/764864, deployed in `0.3.104`) - **root cause still unknown** - leaving **wdqs@codfw depooled** through the night to buy more time to investigate #### What's left to do? [x] Draft lightweight incident report (see form at https://wikitech.wikimedia.org/wiki/Incident_status; must be logged in to wikitech to see the form) [] Identify root cause of failure (currently we only have a mitigation in place) [] (after fixing root cause) re-enable traffic to codfw && revert patch for puppet metrics change: https://gerrit.wikimedia.org/r/c/operations/puppet/+/764830/

#### Current status **Current status:** - **Deployed mitigation** (via reverting to old checkpoint as well as https://gerrit.wikimedia.org/r/c/wikidata/query/rdf/+/764864, deployed in `0.3.104`) - **root cause still unknown** - leaving **wdqs@codfw depooled** through the night to buy more time to investigate #### What's left to do? [x] Draft lightweight incident report (see form at https://wikitech.wikimedia.org/wiki/Incident_status; must be logged in to wikitech to see the form) [] Identify root cause of failure (currently we only have a mitigation in place) [] (after fixing root cause) re-enable traffic to codfw && revert patch for puppet metrics change: https://gerrit.wikimedia.org/r/c/operations/puppet/+/764830/