Page MenuHomePhabricator

Show KML files on maps
Closed, DeclinedPublic

Description

Maps should be able to show a specified KML file as an overlay/layer. From mw:Talk:Maps#KML:

Overlaying a KML layer would generally be done by the map library, (e.g. leaflet), and there would be no problems doing so with these tiles. Pnorman (talk) 23:38, 24 September 2015 (UTC)

Note that until T28059 is resolved, KML files have to stored as wikitext on template subpages. The actual file can be accessed using action=raw, and a page's related KML file can be found through Wikidata Property:P3096 (see Meta:KML files and d:Property talk:P3096 for further information).

Event Timeline

Evad37 created this task.Aug 29 2016, 7:28 AM
Restricted Application added a project: Discovery. · View Herald TranscriptAug 29 2016, 7:28 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Yurik added a subscriber: Yurik.Aug 29 2016, 1:57 PM

I think we should not support KML files, but instead offer a on-the-fly conversion to GeoJSON. This way we can store it using our JSON storage content handler (JsonConfig), optionally allow Lua to do on-the-fly modification (e.g. localization or filtering), and show directly in the <mapframe> and <maplink>.

As an editor, I want to start with "I have a KML file for this feature" and end up with "this feature is displayed on the map", so any sort of automated process in the middle (on-the-fly conversion to GeoJSON etc.) would be fine.

Would it also be possible to have on-the-fly conversion from stored GeoJSON back to KML? That way the Attached KML templates could work with the stored GeoJSON data, rather than having to redundantly store the same data in both KML format as well as GeoJSON format.

Yurik added a comment.Sep 9 2016, 5:04 AM

CCing from the Village pump:

@Evad37, we need to figure out how KML files are used, and if that information is an exact duplicate of what users contribute to the Open Street Map. I presume we don't want duplicates if we can avoid it, except possibly in some corner cases. It seems that most of the KML usage has been to draw a line representing some road, as well as some shapes to represent city limits. If so, OSM is a much much better location for such information - we have less than 10,000 shapes, they have millions. OSM community has much better ways to deal with the accuracy of such data, as well as some tools to annotate it, and it is automatically available to all the Wikipedia languages.

<maplink latitude="52.4580" longitude="13.4143" zoom="10" text="map of Berlin">{
"type":"ExternalData", "href":"geoshape:///?ids=Q64"
}</maplink>

This example does not require any data to be stored on the wiki, or any KML files. It simply references Berlin by its Wikidata ID Q64. That said, Wikidata IDs are very new to OSM community, and they need to be improved. Currently there are about 40,000 items in OSM db that have Wikidata ID.

The most massive KML usage - lines - is still a TODO. We might have all the needed data as OSM db, but we don't have it available to you yet. See T145175.

Lastly, geo data should be storable in one centralized location, e.g. Commons, and easily usable from all wikis. This is still a work in progress

Isn't this relying on OpenStreetMap for data though, outside of our control? I don't think this is the right way to go.

Evad37 added a comment.Sep 9 2016, 7:22 AM

we need to figure out how KML files are used

This SPARQL query shows the various types of items KML files are associated with (for items which have P31 specified on Wikidata)

and if that information is an exact duplicate of what users contribute to the Open Street Map

Much of it is probably duplicated, but not everything will be within the scope of OSM - e.g. 2013 Moore tornado, or the former M-31 (Michigan highway) (decommissioned in 1926). Or different source data being used (see below).

OSM community has much better ways to deal with the accuracy of such data

Not sure about this, doesn't OSM encourage users to do their own research rather than using (what Wikipedia would consider) reliable sources, so as to avoid copyright issues? - which could result in there being differences between OSM data and wikimedia KML files

some tools to annotate it, and it is automatically available to all the Wikipedia languages

Definitely an advantage, especially if the annotations are or could be translatable (but I will note that cross-wiki access to KML files is now possible through Module:Attached KML & Wikidata)

This example does not require any data to be stored on the wiki, or any KML files. It simply references Berlin by its Wikidata ID Q64.

Looks good! Close integration between OSM / Wikidata / other Wikimedia projects to avoid redundant information is probably beneficial for all concerned.

The most massive KML usage - lines - is still a TODO

As long as its on the agenda and hasn't been overlooked, that's fine

Lastly, geo data should be storable in one centralized location, e.g. Commons, and easily usable from all wikis. This is still a work in progress

Definitely agree with this :) . I having been thinking along similar lines, in regards to the existing KML files, but it seems like a lot of work for limited benefit, given that each wiki's community, including Commons, would need to have a discussion and then agree to move the KML files over.

Yurik added a comment.Sep 9 2016, 7:38 AM

@Rschen7754 i think "control" is the wrong word here - we don't have control over either dataset - anyone can join both projects and contribute there :) Yes, the rules are slightly different, but I doubt there will be many fundamental disagreements about the data.

@Evad37, thanks for the query, interesting! the M-31 and tornado are great examples of the non-OSM data, and would need to be stored on another platform (like Commons). My personal fav is the route Don Quixote took through spain.
Moreover, I think there is a huge trove of open data out there, like animal population densities, climate, forestation, ..., that should be usable by Wikipedia, just like we already use OSM data.
Lastly, KML itself - I am not very happy with that format - it is much harder to process, while GeoJSON seems much more intuitive and straight forward. Plus TopoJSON could significantly optimize its storage requirements.

@Yurik "Yes, the rules are slightly different, but I doubt there will be many fundamental disagreements about the data." How do you know this for sure? I can think of several cases off the top of my head where it would be different. Take California state highways: sometimes the road still exists, but is no longer a state highway. One of the fundamentals of good programming is accounting for corner cases, rather than trying to shoehorn everything into one paradigm/user story.

Also, for reference there are almost 11,000 KML files: https://www.wikidata.org/wiki/Wikidata:Database_reports/Constraint_violations/P3096

@Rschen7754, I was saying that in most of the cases and most of the time - the shapes of the highways and the shapes of cities or regions will be the same in both OSM DB and what wiki community will want to draw. On occasion, when this is not the case, you will be able to store it as geojson on commons.

Yurik moved this task from Wiki-related to Kartographer on the Maps board.Dec 6 2016, 3:18 PM
Yurik edited projects, added Maps (Kartographer); removed Maps.
mxn added a subscriber: mxn.Dec 7 2016, 8:01 PM

I think this is ready to be closed, because even though we don't support KML, we now support GeoJSON in the .map Commons Datasets, usable everywhere; as well as the geoshapes service directly from OSM database.

I agree this can be closed now, given that GeoJSON or OSM shapes can be used instead.

Yurik closed this task as Resolved.Dec 24 2016, 4:06 PM
Yurik claimed this task.

So can Maps now show a specified KML file as an overlay/layer, or not?

Yurik added a comment.Dec 26 2016, 2:20 PM

Maps can show geojson layer, and KML can be easily covered to geojson using many online tools, hence the ticket author agreed that the spirit of the ticket is done))

Aklapper changed the task status from Resolved to Declined.Dec 28 2016, 10:51 PM

If Maps cannot show KML files (see task summary) then this task is not resolved but declined (as a very simple workaround is available)...