Maniphest T320099

Error log events for Striker not decorated with request_id value as expected
Closed, ResolvedPublicBUG REPORT
Actions

Assigned To

Authored By

	bd808
	Oct 6 2022, 9:43 PM

Description

In T319500#8294677, @bd808 wrote:

In T319500#8294202, @bd808 wrote:

Striker still seems to struggle with getting its larger error messages to end up in the ELK cluster.

It turns out the message was in the ELK cluster. I was not finding it because the messages there do not carry the req_id value from the crash report as I expected. That is another bug to investigate.

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	striker: Bump container version to 2022-10-18-161910-production	operations/puppet	production	+2 -2
	fix: Add request_id filter to 'cee' log stream	labs/striker	master	+1 -0

Customize query in gerrit

Related Objects

Mentioned Here: T217932: Change log routing to ELK cluster to use rsyslog->kafka rather than talking directly to the ELK cluster
T319500: Attaching Phabricator account to a second Developer account via Striker results in a fatal error

Event Timeline

bd808 created this task.Oct 6 2022, 9:43 PM

Striker's settings file includes this in it's LOGGING config:

'filters': {
    'request_id': {
        '()': 'log_request_id.filters.RequestIDFilter'
    }
},

The bit that seems to be missing is a 'filters': ['request_id'], setting attached to the 'cee' log handler. Both the console and file handlers do have that configuration. This looks to be my oversight when implementing T217932: Change log routing to ELK cluster to use rsyslog->kafka rather than talking directly to the ELK cluster.

bd808 changed the task status from Open to In Progress.Oct 17 2022, 8:29 PM

bd808 claimed this task.

bd808 triaged this task as Medium priority.

bd808 moved this task from Backlog to Doing on the Striker board.

Restricted Application added a project: User-bd808. · View Herald TranscriptOct 17 2022, 8:31 PM

Change 843578 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[labs/striker@master] fix: Add request_id filter to 'cee' log stream

https://gerrit.wikimedia.org/r/843578

gerritbot added a project: Patch-For-Review.Oct 17 2022, 8:31 PM

Change 843578 merged by jenkins-bot:

[labs/striker@master] fix: Add request_id filter to 'cee' log stream

https://gerrit.wikimedia.org/r/843578

Maintenance_bot removed a project: Patch-For-Review.Oct 17 2022, 11:30 PM

Change 844063 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[operations/puppet@production] striker: Bump container version to 2022-10-18-161910-production

https://gerrit.wikimedia.org/r/844063

gerritbot added a project: Patch-For-Review.Oct 18 2022, 8:38 PM

Change 844063 merged by Andrew Bogott: