Raw upload webrequest partitions for 2014-12-08T17/1H not marked successful
Closed, DeclinedPublic
Actions

Assigned To

None

Authored By

	QChris
	Dec 9 2014, 10:30 PM

Description

The webrequest upload partition *) for 2014-12-08T17/1H has not been
marked successful.

What happened?

_________________________________________________________________
qchris@stat1002 // jobs: 0 // time: 13:19:26 // exit code: 0
cwd: ~
~/cluster-scripts/dump_webrequest_status.sh
  +------------------+--------+--------+--------+--------+
  | Date             |  bits  | mobile |  text  | upload |
  +------------------+--------+--------+--------+--------+
[...]
  | 2014-12-08T15/1H |    .   |    .   |    .   |    X   |
  | 2014-12-08T16/1H |    .   |    .   |    .   |    .   |
  | 2014-12-08T17/1H |    .   |    .   |    .   |    X   |
  | 2014-12-08T18/1H |    .   |    .   |    .   |    X   |
  | 2014-12-08T19/1H |    .   |    .   |    .   |    .   |
[...]
  +------------------+--------+--------+--------+--------+


Statuses:

  . --> Partition is ok
  M --> Partition manually marked ok
  X --> Partition is not ok (duplicates, missing, or nulls)

Related Objects
Search...

Status	Assigned	Task
Resolved	Ottomata	T72085 Raw webrequest partitions that were not marked successful
Resolved	Ottomata	T72087 Kafka partition leader elections causing a drop of a few log lines
Declined	None	T78081 Raw upload webrequest partitions for 2014-12-08T17/1H not marked successful

Event Timeline

QChris created this task.Dec 9 2014, 10:30 PM

QChris raised the priority of this task from to Medium.

QChris updated the task description. (Show Details)

QChris added a project: Analytics-Clusters.

QChris changed Security from none to None.

QChris updated the task description. (Show Details)

QChris added subscribers: Unknown Object (MLST), • kevinator, QChris and 2 others.

2014-12-08T15/1H is tracked in T77999
2014-12-08T18/1H is tracked in T78000

The partition is affected only during the 4 seconds from 2014-12-08T17:17:19 to 2014-12-08T17:17:22.
This period matches kafka leader re-election to bring analytics1021 back into the game after its disk issue.

It caused in total 992 missing log lines from four hosts.

Host	Start of issue	End of issue
cp3005.esams.wikimedia.org	2014-12-08T17:17:19	2014-12-08T17:17:21
cp3010.esams.wikimedia.org	2014-12-08T17:17:20	2014-12-08T17:17:21
cp3017.esams.wmnet	2014-12-08T17:17:21	2014-12-08T17:17:21
cp4013.ulsfo.wmnet	2014-12-08T17:17:20	2014-12-08T17:17:22

Ottomata closed this task as Declined.Mar 31 2015, 2:09 PM

Raw upload webrequest partitions for 2014-12-08T17/1H not marked successfulClosed, DeclinedPublicActions

Description

Related ObjectsSearch...

Event Timeline

Raw upload webrequest partitions for 2014-12-08T17/1H not marked successful
Closed, DeclinedPublic
Actions

Related Objects
Search...