Page MenuHomePhabricator

ipoid logs not visible in Logstash
Closed, ResolvedPublic

Description

After merging the MRs in T351430: Improve logging for ipoid app to use Elastic Common Search scheme, we no longer see any of those logs in Logstash. https://wikimedia.slack.com/archives/C05EP1X6K09/p1705502786904129 contains a link to the "Discover" view in Logstash where I've filtered out a bunch of noise, but can't find any trace of the updated logs.

Event Timeline

kostajh triaged this task as High priority.Jan 17 2024, 4:56 PM
kostajh created this task.

Using the orchestrator.namespace: ipoid filter instead of kubernetes.label.app: ipoid works but only the logs for ipoid-production-daily-updates appear.

I'll dig into why the main app logs aren't showing up.

The main app logs aren't yet in ECS format, cf https://gitlab.wikimedia.org/repos/mediawiki/services/ipoid/-/merge_requests/213
A split index dashboard isn't possible as far as I know, so I fixed the dashboard to use the ECS orchestrator.namespace filter, the main app logs can be found here in the meantime.

kostajh claimed this task.
kostajh added a subscriber: akosiaris.

Thanks. In the meantime, I think we can mark this as resolved. Thanks to you and @akosiaris for reworking the ipoid dashboard to use ECS https://logstash.wikimedia.org/app/dashboards#/view/0011d930-7eee-11ee-9c86-cf41f505a7c5