Page MenuHomePhabricator

hupu.wikidocumentaries.eqiad1.wikimedia.cloud disk full
Closed, ResolvedPublic

Description

The 'hupu' VM is failing in a bunch of ways due to too many docker images filling up the hard drive. Time to delete some old ones!

Event Timeline

Most of the space was taken by docker logs of the running containers. Easiest solution was to recreate the containers that had long logs, which cleared the logs.

A permanent solution would be to set up rotation of the docker logs, but I remember looking into this earlier and it was not straightforward.

Ok, the following settings for each service in the docker-compose.yml file may have fixed this:

logging:
    driver: json-file
    options:
        max-size: 10m
        max-file: "20"
        compress: "true"
Andrew claimed this task.