maps NFS volume filling up
Open, MediumPublic
Actions

Assigned To

None

Authored By

	Andrew
	Aug 25 2025, 1:04 PM

Description

The NFS volume that backs up the cloud-vps maps project is beginning to fill up.

/dev/sdb        8.0T  6.7T  950G  88% /srv/maps

Here's a summary of file usage:

root@maps-nfs-1:/srv/maps/project# du -h -d1 
414G	./wma
83M	./styles
998M	./warper
254G	./hillshading
4.0G	./chippy
145G	./osmdb
4.1T	./tiles
4.9T	.

root@maps-nfs-1:/srv/maps/home# du -h -d1 .
23M	./kolossos
36K	./vagrant
52K	./akosiaris
36K	./ori
32K	./yurik
1.4G	./dschwen
28K	./marc
24K	./fnegri
9.2G	./chippy
44K	./maxsem
44K	./andrew
24K	./madhuvishy
48M	./hartman
2.6G	./apmon
20K	./bstorm
1.8T	./warperdata
56K	./nosy
1.8T	.

I suspect that the low-hanging fruit is in /home, as files there tend to be transient and/or forgotten.

Event Timeline

Andrew created this task.Aug 25 2025, 1:04 PM

Restricted Application added a project: cloud-services-team. · View Herald TranscriptAug 25 2025, 1:04 PM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Andrew triaged this task as Medium priority.Aug 25 2025, 1:05 PM

Andrew updated the task description. (Show Details)

@Chippyy can you check why warper data and your home directory have this much data stored ?

I also think we can delete the tiles directory, as we no longer run a tiles server as before in that group.

Do you agree @dschwen ?

taavi removed a project: Cloud-VPS.Aug 25 2025, 1:18 PM

taavi moved this task from Inbox to Watching on the cloud-services-team board.

In T402799#11114779, @TheDJ wrote:

@Chippyy can you check why warper data and your home directory have this much data stored ?

/home/warperdata is the main storage for the Wikimaps Warper application: https://warper.wmflabs.org/ It's where the store of converted map images from Commons and the resulting geo-referenced Geotiff files are stored. As more maps are georeferenced using the Warper in Commons, the storage will increase, which it slowly does. There's around 7800 maps in the wikimaps warper.

My home directory was filling with some database backups that were never cleared out, various copies of tiff files and jpeg files from commons which were used for debugging edge issues. I have just cleaned those up, many thanks!
There remains some ruby libraries and some code which appears to be taking a fair bit of space but I need to double check are not needed. They and other files are there from from the process of upgrading and replacing the new instances of the application over the years.

I don't see 1.4GB in my home dir when I log onto maps-wma2. The big chunk on nfs are map tiles for the WikiMiniAtlas

dschwen@maps-wma2:/mnt/nfs/secondary-maps/home/dschwen$ du -sch *
4.0K	README.check_apaches
54M	apache_heartbeat.log
4.0K	apache_heartbeat.sh
13M	bin
7.3M	git
20K	hosts
4.0K	install.sh
4.0K	iptables.conf
4.0K	kill_old_apaches.sh
0	osmdb
28K	pgconf
28K	postgresql.conf
4.0K	replica.my.cnf
0	wma
4.0K	www.sh
74M	total

root@maps-wma2:/home/dschwen# du -h -d1 .
8.0K	./.config
7.3M	./git
13M	./bin
8.0K	./.gnupg
57M	./.cache
772M	./.vscode-server
36K	./.ssh
223M	./.local
4.0K	./.nano
16K	./.myconfig
1.1G	.

/home/warperdata is the main storage for the Wikimaps Warper application: https://warper.wmflabs.org/

FYI: I don't think we should have 1.8TB of persisting data in a home directory... this is what we have projects directories for in /srv/maps/project

[edit] I notice that the warper instance doesn't mount the /srv/maps endpoint, I guess that explains why this came to be.

D'Uh. Ok, I can delete vscode server, but it'll be redownloaded when I connect again.

side-note: this server is due for a rebuild on Trixie. If you wind up doing anything that requires scheduled downtime let me know and we can do the rebuild at the same time

In T402799#11115098, @dschwen wrote:

D'Uh. Ok, I can delete vscode server, but it'll be redownloaded when I connect again.

Instead of worrying about 1.1GB, I suggest we delete: 4.1T ./tiles from the old tiles server.

In T402799#11115098, @dschwen wrote:

D'Uh. Ok, I can delete vscode server, but it'll be redownloaded when I connect again.

The VS Code Server is non-free software and thus cannot be used on WMCS.

maps NFS volume filling upOpen, MediumPublicActions

Description

Event Timeline

maps NFS volume filling up
Open, MediumPublic
Actions