Page MenuHomePhabricator

Deprecate all non-Kafka logstash inputs
Open, MediumPublic

Description

Umbrella task to track status of applications using non-kafka inputs and thus need to be migrated to the new logging infrastructure.

See also "logstash server statistics" Kibana dashboard for a breakdown of kafka vs non-kafka input traffic https://logstash.wikimedia.org/goto/7900d9a70d24467c2a1f4545651a0b16

Related Objects

StatusSubtypeAssignedTask
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
Resolvedfgiunchedi
Resolvedbd808
ResolvedEBernhardson
Resolvedfgiunchedi
Resolvedfgiunchedi
Resolvedfgiunchedi
InvalidNone
OpenNone
OpenPchelolo
ResolvedMvolz
Resolvedakosiaris
DeclinedOttomata
DeclinedNone
Resolvedfgiunchedi
Resolvedfgiunchedi
Resolvedbmansurov
ResolvedArlolra
Resolvedelukey
ResolvedMilimetric
OpenMSantos
ResolvedOttomata
Resolvedfgiunchedi
Resolvedfgiunchedi
DeclinedNone
Resolvedakosiaris
Resolvedfgiunchedi
Resolvedfgiunchedi
OpenNone

Event Timeline

fgiunchedi updated the task description. (Show Details)Jul 2 2019, 1:21 PM
MoritzMuehlenhoff triaged this task as Medium priority.Jul 3 2019, 7:59 AM
fgiunchedi moved this task from Backlog to Doing on the User-fgiunchedi board.Jan 13 2020, 10:08 AM

Change 574467 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] logstash: write kafka inputs to es by default

https://gerrit.wikimedia.org/r/574467

Change 574485 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] scap: enable logging to syslog

https://gerrit.wikimedia.org/r/574485

Change 574467 merged by Filippo Giunchedi:
[operations/puppet@production] logstash: write kafka inputs to es by default

https://gerrit.wikimedia.org/r/574467

Mentioned in SAL (#wikimedia-operations) [2020-02-25T13:42:05Z] <godog> roll-restart logstash in eqiad/codfw - T227080

fgiunchedi moved this task from Inbox to Backlog on the observability board.Apr 6 2020, 12:36 PM
fgiunchedi moved this task from Doing to Up next on the User-fgiunchedi board.Apr 27 2020, 12:12 PM

As far as gelf is concerned we're left with:

  • elasticsearch: (T225125) pending ES7 upgrade to get native json logging
  • graphoid: will be undeployed
  • kartotherian: (T222377) blocked by unrelated changes pending deployment