When designing a program for use in WMF Production environments, there are certain logging standards that we need to meet, by sending the logs through a supported interface (systemd) and should be of a certain format
The standards can be found here
https://wikitech.wikimedia.org/wiki/Logstash/Interface
For Python program, we should follow the example as shown in the page (code below) -- by using a log structurer (pythonjsonlogger)
This ticket will involve implementing this log structurer functionality in both api_db and webapp (webapp is the proposed name for the Flask app that would serve the frontend)
import logging
import logging.config
from pythonjsonlogger import jsonlogger
class CustomJsonFormatter(jsonlogger.JsonFormatter):
def add_fields(self, log_record, record, message_dict):
super(CustomJsonFormatter, self).add_fields(log_record, record, message_dict)
log_record['level'] = record.levelname.upper()
class StructuredLoggingHandler(logging.StreamHandler):
def __init__(self, rsyslog=False):
super(StructuredLoggingHandler, self).__init__()
if rsyslog:
prefix = '@cee: '
else:
prefix = ''
self.formatter = CustomJsonFormatter(prefix=prefix)
# Demo code below
if __name__ == '__main__':
logging.config.dictConfig({
'version': 1,
'disable_existing_loggers': False,
'root': {
'handlers': ['demo'],
'level': 'DEBUG'
},
'handlers': {
'demo': {
'class': 'logger_demo.StructuredLoggingHandler',
'rsyslog': True
}
}
})
logging.info('It\'s log!')Tasks
- Implement JSON based logger in api_db according to above
- Implement the same for webapp as well.