The webrequest mobile partition for 2014-12-29T17/1H has not been
marked successful.
What happened?
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Ottomata | T72085 Raw webrequest partitions that were not marked successful | |||
| Resolved | Ottomata | T74300 Raw webrequest partitions that were not marked successful due to configuration updates | |||
| Resolved | Ottomata | T74299 Raw webrequest partitions that were not marked successful due to deployments gone wrong | |||
| Resolved | QChris | T85695 Raw mobile webrequest partitions for 2014-12-29T17/1H not marked successful |
Event Timeline
Comment Actions
Around 2014-12-29T17:30, commit 64c7ea2ca798666bdd1638f2fa5423a226610688 broke varnish's reporting to the correct kafka topic on 2014-12-29T18:22. It was fixed by commit ffb852628fd69d65e12a4569b0b0c0fd1b86c6c1 immediately afterwards on 2014-12-29T18:26.
From the mobile caches, three hosts picked up the faulty configuration
and consequently produced to the wrong topic during
2014-12-29T17:23:21 -- 2014-12-29T17:45:22.
| Host | Start of issue | End of issue |
|---|---|---|
| cp1046.eqiad.wmnet | 2014-12-29T17:23:25 | 2014-12-29T17:26:17 |
| cp1047.eqiad.wmnet | 2014-12-29T17:23:21 | 2014-12-29T17:42:28 |
| cp3014.esams.wmnet | 2014-12-29T17:26:10 | 2014-12-29T17:45:22 |
The caches reported about 4 minutes worth of total mobile traffic to
the wrong topic during the affected period.
(It seems no text, and no upload partitions picked up the faulty config. Two bits host might be affected too, but as we do not care about bits at this point, I'll leave that to a separate bug)