Investigate why this happened and provide a viable solution for this as this problem is not seen in maps eqiad
|Resolved||Gehel||T224395 Maps004 /srv disk space is critical|
|Resolved||• Mathew.onipe||T224874 Maps2004 ran into disk space issues again after reimaging with new partitioning scheme|
Looking deep into maps2004 postgres database, the problem was traced to gis.planet_osm_line table being larger than normal when compared with its eqiad(maps1004) version.
We are still investigating why this is so.
maps2004 kept track osm2pgsql script for the last 5 days, all of them ended with failures during replication due to disk space. It seems that at some point osm2pgsql failed to roll back and didn't clean up some records causing some tables to grow indefinitely. Even though, the logs are inconclusive.
- fix DB with data available in eqiad, see https://wikitech.wikimedia.org/wiki/Maps#Postgres or recreate OSM database with the initial import script
- regenerate tiles from last osm2pgsql attempt, the list is available at /srv/osm_expire/expire.list.201905282106