Maniphest T141270

Aggregate system logs from kubernetes nodes
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	yuvipanda
	Jul 25 2016, 4:16 PM

Tags

Referenced Files

None

Subscribers

Description

Since we otherwise have log emptiness + death and can not debug, would be nice to centrally aggregate all these logs.

Details

	Subject	Repo	Branch	Lines +/-
	tools: Centrally collect all logs from k8s related nodes	operations/puppet	production	+39 -0
	tools: Reduce number of days central logs are kept to 3	operations/puppet	production	+2 -2
	tools: Add simple loggingreceiver role	operations/puppet	production	+23 -0

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		yuvipanda	T141270 Aggregate system logs from kubernetes nodes
		Resolved		fgiunchedi	T141498 Puppet fails with 'no user syslog' when setting up syslog receiver role on Debian Jessie

Event Timeline

yuvipanda created this task.Jul 25 2016, 4:16 PM

Restricted Application added a project: Cloud-Services. · View Herald TranscriptJul 25 2016, 4:16 PM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

• chasemp triaged this task as High priority.Jul 25 2016, 4:17 PM

• chasemp mentioned this in T141017: Kubernetes worker nodes hanging.

Change 301503 had a related patch set uploaded (by Yuvipanda):
tools: Add simple loggingreceiver role

https://gerrit.wikimedia.org/r/301503

yuvipanda mentioned this in rOPUPf39149fad04c: tools: Add simple loggingreceiver role.Jul 27 2016, 10:06 PM

yuvipanda mentioned this in rOPUP0ae32826864c: tools: Add simple loggingreceiver role.Jul 27 2016, 10:12 PM

Change 301503 merged by Yuvipanda:
tools: Add simple loggingreceiver role

https://gerrit.wikimedia.org/r/301503

yuvipanda mentioned this in rOPUP9a0c7540cc98: tools: Add simple loggingreceiver role.Jul 27 2016, 10:41 PM

yuvipanda created subtask T141498: Puppet fails with 'no user syslog' when setting up syslog receiver role on Debian Jessie.Jul 27 2016, 11:48 PM

Change 301529 had a related patch set uploaded (by Yuvipanda):
tools: Reduce number of days central logs are kept to 3

https://gerrit.wikimedia.org/r/301529

yuvipanda mentioned this in rOPUPa9629f18a60b: tools: Reduce number of days central logs are kept to 3.Jul 28 2016, 12:47 AM

yuvipanda mentioned this in rOPUP11640acfeae8: tools: Reduce number of days central logs are kept to 3.

Change 301529 merged by Yuvipanda:
tools: Reduce number of days central logs are kept to 3

https://gerrit.wikimedia.org/r/301529

yuvipanda mentioned this in rOPUPbbeca20b7fcb: tools: Reduce number of days central logs are kept to 3.Jul 28 2016, 1:02 AM

Change 301532 had a related patch set uploaded (by Yuvipanda):
tools: Centrally collect all logs from k8s related nodes

https://gerrit.wikimedia.org/r/301532

yuvipanda mentioned this in rOPUPf92f639657d4: tools: Centrally collect all logs from k8s related nodes.Jul 28 2016, 1:14 AM

yuvipanda mentioned this in rOPUP24c5ff32b8df: tools: Centrally collect all logs from k8s related nodes.Jul 28 2016, 1:20 AM

yuvipanda mentioned this in rOPUPae0e32330964: tools: Centrally collect all logs from k8s related nodes.

Change 301532 merged by Yuvipanda:
tools: Centrally collect all logs from k8s related nodes

https://gerrit.wikimedia.org/r/301532

yuvipanda mentioned this in rOPUP33dbc8424cd3: tools: Centrally collect all logs from k8s related nodes.Jul 28 2016, 1:37 AM

Done and documented in https://wikitech.wikimedia.org/wiki/Tools_Kubernetes#Infrastructure_centralized_Logging

yuvipanda mentioned this in T141500: [toolforge.infra] Setup centralized logging for the infra (ELK possibly).Jul 28 2016, 1:48 AM

• chasemp mentioned this in T141673: Track labs instances hanging .Aug 3 2016, 3:23 AM

scfc closed subtask T141498: Puppet fails with 'no user syslog' when setting up syslog receiver role on Debian Jessie as Resolved.Dec 4 2016, 8:14 PM

• Phabricator_maintenance removed a subscriber: yuvipanda.Jun 7 2017, 6:41 PM