Page MenuHomePhabricator

Airflow scheduler and webserver logs should be readable by airflow instance admins
Open, Needs TriagePublic

Description

Right now, airflow process logs are only in journalctl. We should

  1. make these readable on the servers by admins. Either:

    1a. Put these in rotated files using airflow logging settings or rsyslog

    1b. add airflow admins to 'systemd-journal' posix group to allow them to use journalctl
  1. send airflow process logs to logstash

We should for sure do 1. 2. would be nice to have, but maybe not necessary.

Note that this are about airflow process logs, not DAG/job logs, which are already available on disk and in the Airflow webserver UI.