Page MenuHomePhabricator

Increase page size limit for map-data on commons
Open, MediumPublic

Description

Page size limit for the map-data on commons is 2 MB. The commons-map-editor automatically completes tabs and blanks, so the real data-size for the GeoJSON is about 1.5 MB. That's really too small. Please increase this limit. Thank you.

Limit described here:
https://www.mediawiki.org/wiki/Help:Map_Data#Restrictions_and_gotchas

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 29 2017, 5:39 PM
Looniverse updated the task description. (Show Details)Oct 29 2017, 6:03 PM
debt triaged this task as Medium priority.Oct 30 2017, 9:04 PM
debt added a project: Discovery.
debt added a subscriber: debt.

Thanks for opening this ticket, @Looniverse, but currently the Map team doesn't have the engineering resources to take this on.

MaxSem added a subscriber: MaxSem.Oct 31 2017, 6:55 PM

This is a general restriction on page length in MediaWiki, and I don't think that raising this already quite generous limit would be a good thing. We aren't just storing this data, we are serving it to users in maps, and waiting for a huge chunk of data to be downloaded isn't nice. There are other ways to reduce the size of data, like limiting precision and simplifying geometries.

We wanted to store the geographic data of the constituencies in Germany. One of the 299 constituencies has a filesize of the GeoJSON with 2,5 MB. The official data came from the Federal Returning Officer; when reducing the precision or simplifying geometries, data aren't consistent any more.
In my opinion 2,5 MB is not a huge chunk of data.

Even Openstreetmap does not want to save the data in a relation, because they don't want historic data and they want to reduce the number of relations.

If nobody wants to save the data, that is sad.

Yurik added a subscriber: Yurik.Oct 31 2017, 8:48 PM

@Looniverse I am sure Wikipedia wants to store the data, it just doesn't have good means to work with it at this point. The .map pages are meant to be used as a "chunk" - all of the data is used at once. The larger datasets, like OSM, are meant to be pre-processed when displayed, e.g. broken up into tiles and simplified for the zoom level. I would recommend simplifying the geometry to make it storable in .map.

If I recall correctly the limit for the Data: namespace was set to match the $wgMaxArticleSize.

@Looniverse Could you break up the data into multiple GeoJSON files to incorporate into your map?

Looniverse added a comment.EditedNov 18 2017, 10:46 PM

I simplified the geometry with mapshaper. My GeoJSON was 480 KB. The WM-Commons-Map-Editor automatically inserts a lot of spaces: so map-size was 2,1 MB. Is it possible to add fewer spaces? In every line with coordinates are 36 blanks!
That means, more than 75% of the page size are used by spaces...

I believe that simplifying geometry is not a solution. When we talk about geographical information, in my case water basins, we cannot give wrong information. The line runs along the crest of the mountains, not near or more or less.
Finally, if there is no solution, this good tool in its intention is frustrated. We will have to continue with OSM and Wikidata. Regards