Page MenuHomePhabricator

Install a robots.txt file for
Open, Needs TriagePublic


It's my first bug report, so my apologises if I'm doing something wrong...

I saw map-tiles showing up in the Google Image search of my website. Which means that the Google Crawler has crawled and index your server. I checked your robots.txt file but you seem to not have one. In my mind you could reduce your server load significantly by installing a robots.txt file that disallows all crawlers.
Just like OSM does:
Or OpenTopoMap:

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 12 2019, 5:22 PM