In order to debug replication errors, it would be helpful if we were archiving postgres logs from the maps replica nodes, in our searchable backend.
Description
Description
Details
Details
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Send PostgreSQL logs to logstash | operations/puppet | production | +12 -0 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T321959 Tech Wishes - Maps service infrastructure deprecations | |||
| Resolved | None | T316365 Restore the map data health and parity between clusters | |||
| Open | None | T321887 Export maps postgres logs to logstash |
Event Timeline
Comment Actions
Do we have to wait for an upgrade to PostgreSQL 15 in order to get JSON logging that can be picked up by logstash? https://www.postgresql.org/docs/current/runtime-config-logging.html#RUNTIME-CONFIG-LOGGING-JSONLOG Otherwise we could work around with an extension like https://github.com/michaelpq/pg_plugins/tree/main/jsonlog
If we can't emit json logs, it seems there are connectors for flat logging, https://wikitech.wikimedia.org/wiki/Logstash/Interface
Comment Actions
I'm not sure Data-Persistence team is a good fit for this. We don't own the PG instance.
Comment Actions
Change 853941 had a related patch set uploaded (by Awight; author: Awight):
[operations/puppet@production] Send PostgreSQL logs to logstash
Comment Actions
Change 853941 abandoned by Awight:
[operations/puppet@production] Send PostgreSQL logs to logstash
Reason:
This was just a POC.