Page MenuHomePhabricator

Check whether maps project requires NFS or not
Open, Needs TriagePublic

Description

This project has an entry in modules/labstore/files/nfs-mounts.yaml but no task under T102240

Event Timeline

Krenair created this task.Oct 31 2018, 1:13 PM

dumps: true
scratch: true
maps: true

Mholloway moved this task from All map-related tasks to Tracking on the Maps board.Nov 6 2018, 4:11 PM

This caused confusion today relating to replacing the existing maps instances with new ones running stretch.

Info about the maps-warper2 instance. it is soon to be replaced by maps-warper3 running Debian Stretch.
The maps project usage for this instance is to 1) to store about 230G of processed map images, 2) image thumbnails from the application (~600mb) 3) application database backups (~2G) and 4) the instance (and I imagine other maps instances?) has it's /home directory mapped to /mnt/nfs/labstore1003-maps/home/
In the old labs situation it had issues with running out of disk space, and hardware disk failures of additional local disks. I think using the project store to keep data was used after that in 2015.
This runs the http://warper.wmflabs.org/ application, started in 2014 and is separate from the old and new OSM work.
I think it would be okay for this the warper to run from an attached disk assuming it has enough space for all the existing and future additional map (let alone having /home on nfs is bonkers tho) It would be nice to have an off instance backup location.

For the OSM tiles, it looks to me as if they are stored in the /mnt/nfs/labstore1003-maps/project/tiles directory?

Krenair added a comment.EditedDec 13 2018, 8:41 PM

You'd need a special flavour to be created but you could have one with that much local storage (e.g. there is a bigdisk2 flavour with 300G disk space). You could create a second instance from the same flavour as a place to store backups (you'd want to ensure this instance ends up running on a different hypervisor host to the first).

I think it's inevitable that it will exceed the 300G space in less than a year as more maps and images get added, so maybe a custom one would be better.

Bstorm added a subscriber: Bstorm.May 31 2019, 9:05 PM