Page MenuHomePhabricator

Raw webrequest partitions for 2014-12-03T17/1H not marked successful
Closed, ResolvedPublic

Description

None of the webrequest partitions *) for 2014-12-03T17/1H have been
been marked successful.

What happened?

*)


qchris@stat1002 // jobs: 0 // time: 12:50:17 // exit code: 0
cwd: ~
~/cluster-scripts/dump_webrequest_status.sh 
  +------------------+--------+--------+--------+--------+
  | Date             |  bits  | mobile |  text  | upload |
  +------------------+--------+--------+--------+--------+
[...]
  | 2014-12-03T15/1H |    .   |    .   |    .   |    .   |
  | 2014-12-03T16/1H |    .   |    .   |    .   |    .   |
  | 2014-12-03T17/1H |    X   |    X   |    X   |    X   |
  | 2014-12-03T18/1H |    .   |    .   |    .   |    .   |
  | 2014-12-03T19/1H |    .   |    .   |    .   |    .   |
[...]
  +------------------+--------+--------+--------+--------+


Statuses:

  . --> Partition is ok
  M --> Partition manually marked ok
  X --> Partition is not ok (duplicates, missing, or nulls)

Event Timeline

QChris claimed this task.
QChris raised the priority of this task from to Needs Triage.
QChris updated the task description. (Show Details)
QChris added a project: Analytics-Clusters.
QChris changed Security from none to None.
QChris subscribed.

All partitions show ~16% duplicates (no missing lines) for all hosts.

Since analytics1027 got upgraded during that hour, I attribute it to the upgrade (although I could not find failed camus jobs).

Ganglia shows underreplicated partitions during that hour.

I deduped the partition and we now have a clean partition.