Page MenuHomePhabricator

Review need for Library Card platform logs
Closed, ResolvedPublic

Description

The Library Card platform generates a number of logs and we have no clear retention policy (T191778). The logs get cleared out occasionally, but on an ad-hoc basis and without a clear 'After X days' policy.

We want to improve this situation, implementing a clear retention policy, but first need to evaluate what we need these logs for.

This investigation should answer:

  • What logs do we currently generate?
  • Which of these logs is useful to us, and how long are they useful for?

We can then move forward with an informed decision about which logs to keep and for how long.

Event Timeline

Samwalton9-WMF renamed this task from Review and improve platform logging procedures to Review need for Library Card platform logs.Jul 16 2020, 10:32 AM
Samwalton9-WMF triaged this task as Medium priority.
Samwalton9-WMF updated the task description. (Show Details)

We discussed this in our engineering meeting yesterday and concluded that we have two types of logs currently, and a future third.

  1. Django logs - these are retained based on deployments, being cleared out when we deploy new code. This currently happens on a regular basis but is variable. Old Django volumes/containers are deleted every week.
    1. We've never needed to go back further than a week in these logs, since they're usually used to fix immediate problems.
  2. Requests module. We think these logs might only have been used for pageview data, which we don't think we need anymore (T272575). We've never used this for development purposes and currently drop the data manually on a periodic basis.
  3. Matomo logs. This is configurable in Matomo.
Samwalton9-WMF claimed this task.

Filed T272578 for django logs, T272575 for requests. Matomo will be configured when setup.