Ship peopleweb apache2 logs to logstash.
This is a low-volume apache2 log producer which makes it a good early candidate for log shipping via the logging pipeline built in Q2 2018.
Description
Description
Details
Details
Customize query in gerrit
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | fgiunchedi | T205849 Begin the implementation of Q1's Logging Infrastructure design (2018-19 Q2 Goal) | |||
| Resolved | fgiunchedi | T205852 Onboard at least 10 new non-sensitive log producers to the logging pipeline | |||
| Resolved | herron | T209860 Ship peopleweb apache2 error logs to ELK |
Event Timeline
Comment Actions
Change 474760 had a related patch set uploaded (by Herron; owner: Herron):
[operations/puppet@production] rsyslog::input::file add rsyslog imfile wrapper for file ingestion
Comment Actions
Change 474813 had a related patch set uploaded (by Herron; owner: Herron):
[operations/puppet@production] logstash: add type "apache-error" and use logstash core patterns
Comment Actions
Change 474760 merged by Herron:
[operations/puppet@production] rsyslog::input::file add rsyslog imfile wrapper for file ingestion
Comment Actions
Change 474813 merged by Herron:
[operations/puppet@production] logstash: add type "apache2-error" and use logstash core patterns
Comment Actions
Change 474973 had a related patch set uploaded (by Herron; owner: Herron):
[operations/puppet@production] peopleweb: ingest apache error logs with rsyslog for shipping
Comment Actions
Change 474974 had a related patch set uploaded (by Herron; owner: Herron):
[operations/puppet@production] rsyslog: output logs tagged 'input-file-apache2-error' to kafka
Comment Actions
Change 474974 merged by Herron:
[operations/puppet@production] rsyslog: output logs tagged 'input-file-apache2-error' to kafka
Comment Actions
Change 474973 merged by Herron:
[operations/puppet@production] peopleweb: ingest apache error logs with rsyslog for shipping
Comment Actions
Apache error logs for peopleweb (rutherfordium) are now present in ELK. These can be found by searching in Kibana for host:rutherfordium, or program.raw:input-file-apache2-error for example. The volume of errors is low so it may help to search across a wide time range.
Comment Actions
@herron please note that rutherfordium is being replaced by people1001 as we speak (jessie -> stretch). Currently in progress. The new host already has the role and i will switch Varnish backend this week. If there are any manual steps involved or the hostname rutherfordium is hardcoded, that would need changes. [T210036]
Comment Actions
Ah! Good to know. The log shipping configs were applied at the role level so no host specific changes would be needed, but do let me know if any related issues arise during the stretch upgrade!