Page MenuHomePhabricator

MariaDB missing logrotate for error and slow logs
Closed, DeclinedPublic


In the DBs configurations we don't set the log_error and slow_query_log_file variables so the default values are used and the files are saved in the data directory and we don't configure a custom logrotate.

Although the slow_query_log is OFF by default and the error log is usually small, in some servers we have it quite big, particularly on masters:

-rw-r----- 1 mysql root    12G Feb 21 09:53 db1052.err
-rw-r----- 1 mysql root    27G Feb 21 10:11 db1038.err

I think we should configure some logrotation policy.

Event Timeline

We should send those to syslog/ELK.

Change 272639 had a related patch set uploaded (by Volans):
mariadb: Moved error logs to syslog

Change 272639 abandoned by Volans:
mariadb: Moved error logs to syslog

Complete list of larger error files on the fleet:

db1038.eqiad.wmnet:         33G
dbstore2002.codfw.wmnet:    19G
db1052.eqiad.wmnet:         13G
dbstore2001.codfw.wmnet:    3.7G
db1051.eqiad.wmnet:         2.3G
labsdb1001.eqiad.wmnet:     1.5G
db1011.eqiad.wmnet:         1.3G
db1028.eqiad.wmnet:         839M
db1033.eqiad.wmnet:         626M
db1058.eqiad.wmnet:         386M
db1047.eqiad.wmnet:         364M
db2029.codfw.wmnet:         359M
dbstore1001.eqiad.wmnet:    129M
dbstore1002.eqiad.wmnet:    104M

Some of the above have been reimaged already/there are blocking tasks to do to reduce its verbosity in the first place.

As I mention on T127638 I would like to work towards ELK if possible.

Marostegui added a subscriber: Marostegui.

After my chat with @Volans this is no longer needed as with the migration to stretch logs are no longer stored on /srv/ as they are being handled by journald