Page MenuHomePhabricator

Deprecate all non-Kafka logstash inputs
Open, MediumPublic

Description

Umbrella task to track status of applications using non-kafka inputs and thus need to be migrated to the new logging infrastructure.

See also "logstash server statistics" Kibana dashboard for a breakdown of kafka vs non-kafka input traffic https://logstash.wikimedia.org/goto/7900d9a70d24467c2a1f4545651a0b16

Related Objects

StatusSubtypeAssignedTask
InvalidNone
OpenNone
OpenNone
Resolvedfgiunchedi
OpenNone
OpenNone
OpenNone
Resolvedfgiunchedi
Resolvedbd808
ResolvedEBernhardson
Resolvedfgiunchedi
Resolvedfgiunchedi
Resolvedfgiunchedi
InvalidNone
OpenNone
ResolvedPchelolo
ResolvedMvolz
Resolvedakosiaris
DeclinedOttomata
DeclinedNone
Resolvedfgiunchedi
Resolvedfgiunchedi
Resolvedbmansurov
ResolvedArlolra
Resolvedelukey
ResolvedMilimetric
ResolvedMSantos
ResolvedOttomata
Resolvedfgiunchedi
Resolvedfgiunchedi
DeclinedNone
Resolvedakosiaris
Resolvedfgiunchedi
Resolvedfgiunchedi
Resolvedakosiaris

Event Timeline

Change 574467 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] logstash: write kafka inputs to es by default

https://gerrit.wikimedia.org/r/574467

Change 574485 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] scap: enable logging to syslog

https://gerrit.wikimedia.org/r/574485

Change 574467 merged by Filippo Giunchedi:
[operations/puppet@production] logstash: write kafka inputs to es by default

https://gerrit.wikimedia.org/r/574467

Mentioned in SAL (#wikimedia-operations) [2020-02-25T13:42:05Z] <godog> roll-restart logstash in eqiad/codfw - T227080

As far as gelf is concerned we're left with:

  • elasticsearch: (T225125) pending ES7 upgrade to get native json logging
  • graphoid: will be undeployed
  • kartotherian: (T222377) blocked by unrelated changes pending deployment

Change 657136 had a related patch set uploaded (by Effie Mouzeli; owner: Effie Mouzeli):
[operations/puppet@production] scap: disable udp logging

https://gerrit.wikimedia.org/r/657136

Change 574485 merged by Effie Mouzeli:
[operations/puppet@production] scap: enable logging to syslog

https://gerrit.wikimedia.org/r/574485

Change 657136 merged by Effie Mouzeli:
[operations/puppet@production] scap: disable udp logging

https://gerrit.wikimedia.org/r/657136

Change 663689 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[wikimedia/slimapp@master] use monolog SyslogHandler and add CeeFormatter

https://gerrit.wikimedia.org/r/663689