About half of the raw webrequest partitions for 2014-12-25T16/15H not marked successful
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	QChris
	Jan 2 2015, 7:10 PM

Description

About half of the raw webrequest partitions *) for 2014-12-25T16/15H have not
been marked successful.

What happened?

_________________________________________________________________
qchris@stat1002 // jobs: 0 // time: 14:59:21 // exit code: 0
cwd: ~
~/cluster-scripts/dump_webrequest_status.sh 600
  +------------------+--------+--------+--------+--------+
  | Date             |  bits  | mobile |  text  | upload |
  +------------------+--------+--------+--------+--------+
[...]
  | 2014-12-25T16/1H |    X   |    .   |    X   |    X   |
  | 2014-12-25T17/1H |    .   |    .   |    X   |    X   |
  | 2014-12-25T18/1H |    .   |    X   |    .   |    .   |
  | 2014-12-25T19/1H |    X   |    X   |    X   |    X   |
  | 2014-12-25T20/1H |    X   |    X   |    X   |    X   |
  | 2014-12-25T21/1H |    X   |    .   |    X   |    X   |
  | 2014-12-25T22/1H |    X   |    X   |    X   |    .   |
  | 2014-12-25T23/1H |    .   |    .   |    X   |    .   |
  | 2014-12-26T00/1H |    .   |    .   |    .   |    X   |
  | 2014-12-26T01/1H |    .   |    X   |    .   |    X   |
  | 2014-12-26T02/1H |    .   |    X   |    .   |    X   |
  | 2014-12-26T03/1H |    .   |    X   |    .   |    X   |
  | 2014-12-26T04/1H |    .   |    .   |    .   |    .   |
  | 2014-12-26T05/1H |    X   |    X   |    X   |    .   |
  | 2014-12-26T06/1H |    X   |    X   |    X   |    X   |
[...]
  +------------------+--------+--------+--------+--------+

Related Objects
Search...

Status	Assigned	Task
Resolved	Ottomata	T72085 Raw webrequest partitions that were not marked successful
Resolved	Ottomata	T85704 Raw webrequest partitions that were not marked successful due to oozie being overwhelmed
Resolved	QChris	T85710 About half of the raw webrequest partitions for 2014-12-25T16/15H not marked successful

Event Timeline

QChris created this task.Jan 2 2015, 7:10 PM

QChris raised the priority of this task from to Medium.

QChris updated the task description. (Show Details)

QChris added a project: Analytics-Clusters.

QChris added subscribers: Aklapper, Unknown Object (MLST), QChris and 3 others.

Resources on the cluster went scarce, and the jobs could not launch before they timed out.

I re-started them by manually, and the issues with the bits, mobile, and text partitions got resolved thereby, except for three partitions *).
Those additional failures are unrelated to the cluster being overwhelmed.

The 2014-12-26T06/1H partitions are handled in T85709.

*) After rerunning, the stats look like:

| 2014-12-25T16/1H |    .   |    .   |    .   |    .   |
| 2014-12-25T17/1H |    .   |    .   |    .   |    X   |
| 2014-12-25T18/1H |    .   |    .   |    .   |    .   |
| 2014-12-25T19/1H |    .   |    .   |    .   |    .   |
| 2014-12-25T20/1H |    .   |    .   |    .   |    .   |
| 2014-12-25T21/1H |    .   |    .   |    .   |    .   |
| 2014-12-25T22/1H |    .   |    .   |    .   |    .   |
| 2014-12-25T23/1H |    .   |    .   |    .   |    .   |
| 2014-12-26T00/1H |    .   |    .   |    .   |    .   |
| 2014-12-26T01/1H |    .   |    .   |    .   |    .   |
| 2014-12-26T02/1H |    .   |    .   |    .   |    .   |
| 2014-12-26T03/1H |    .   |    .   |    .   |    .   |
| 2014-12-26T04/1H |    .   |    .   |    .   |    .   |
| 2014-12-26T05/1H |    .   |    .   |    .   |    .   |
| 2014-12-26T06/1H |    .   |    .   |    X   |    X   |

About half of the raw webrequest partitions for 2014-12-25T16/15H not marked successfulClosed, ResolvedPublicActions

Description

Related ObjectsSearch...

Event Timeline

About half of the raw webrequest partitions for 2014-12-25T16/15H not marked successful
Closed, ResolvedPublic
Actions

Related Objects
Search...