Python logs produced with logging are being printed by Flink as "INFO" all the time, no matter the level they produce. It looks like the message produced by Python is wrapped with the Java logger, so the message becomes a JSON with INFO level where the message contains the whole Python logs as plain text.
Some examples:
Debug as Info:
{"@timestamp":"2026-03-12T15:48:57.088Z","log.level": "INFO","message":"DEBUG:root:No need to resolve https://meta.wikimedia.org/w/api.php, it is already absolute", "ecs.version": "1.2.0","process.thread.name":"flink-pekko.actor.default-dispatcher-11","log.logger":"org.apache.flink.client.python.PythonDriver"}Error as Info:
{"@timestamp":"2026-03-12T15:48:57.093Z","log.level": "INFO","message":"error: Option 'enrich.mediawiki_api_endpoint_template' is not accepted", "ecs.version": "1.2.0","process.thread.name":"flink-pekko.actor.default-dispatcher-11","log.logger":"org.apache.flink.client.python.PythonDriver"}We aren't able to create alerts based on logs for these PyFlink applications as they always produce "INFO". The only errors that Flink produce are when the Python application finally dies, at that point Flink produces errors. But we might want to create some monitor based on regular and recoverable errors, or warnings, so we need to ensure PyFlink reports the right level.
Task is done if:
- Python logs are produced with the right level.
- Python logs are JSON with standard format and fields.
