Page MenuHomePhabricator

Update TWLight to send nginx log data to matomo
Closed, ResolvedPublic

Description

The matomo folks maintain a python etl script that uses the same api as the javascript tracker. To send our nginx log data to matomo, we need to

  • add a syslog container to twlight
  • configure twlight nginx to ship logs there
  • continuously run the etl script on the syslog container

Our matomo instance will treat this exactly like it would incoming javascript client data, so there won't need to be any changes there. We won't be dumping log files onto external storage, so there are no new log retention concerns.

https://github.com/matomo-org/matomo-log-analytics/tree/4.x-dev

Event Timeline

I already created sites in matomo for staging and production. I've now created auth keys in matomo and added docker swarm secrets to those environments for the site id and auth key. Whenever this deploys, it will start sending data to matomo.

updated TWLight wiki with config details for matomo

had to do a little hotfixing and quay.io image cleanup, looks good now!