Page MenuHomePhabricator

Add support for KML/KMZ filetype
Open, MediumPublic

Description

When I try to upload a valid KML file I get "File extension does not match MIME type."

See Also:

This card tracks a proposal from the 2015 Community Wishlist Survey: https://meta.wikimedia.org/wiki/2015_Community_Wishlist_Survey

This proposal received 27 support votes, and was ranked #34 out of 107 proposals. https://meta.wikimedia.org/wiki/Community_Tech_project_ideas#Support_KML_files_in_Commons

Details

Reference
bz26059

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Moved to Normal, because we do not view this as high priority at this time.

Well, Erik Moller has just sent out an email saying that JS on IE6 will be disabled completely with 1.24wm17.

Does this mean that something could be done with this bug? Or would it be better to shift efforts to Wikidata? (or both, and just have Wikidata link to files on Commons?)

Well something could have always been done, just noone has been willing to spend the time to do it.

The js announcement doesnt affect the security issues mentioned above.

The use cases for which I opened the bug are not helped in any way by Wikidata. They also have nothing to do with the WMF. So having Wikidata does not help.

Joeroen De Dauw: see bug 55549, which would solve the big-picture problem with Wikidata.

Wikimedia will apply to Google Summer of Code and Outreachy on Tuesday, February 17. If you want this task to become a featured project idea, please follow these instructions.

If there is no interest / critical mentoring mass to promote this project for GSoC / Outreachy, then maybe the current "Normal" priority should be lower?

It is hard to say, since this is (according to my understanding) a security issue, and because it is not clear how Wikidata will affect this.

@Qgil, I note this is not an easy task, since there are some security aspects to it... That's the reason why it stalled. The task itself is easy, but the background is complex and a solution will require cooperation with and validation by @csteipp

Thank you for the quick feedback. Ok, I will remove the Possible-Tech-Projects tag. Feel free to bring it back if/when this task is a good candidate for GSoC/Outreachy.

The security issues don't sound all the complicated. They basically involve building a parser, checking the file is valid according to some subset of the spec (For our usecase, it might not even have to be a very big subset).

Possibly maybe also adding 64kb of whitespace to the beginning of the file to trick the IE parser.

Not a walk in the park, but possibly within the realm of what a gsoc student could do. Of course it would still require a mentor (I am not volunteering)

Khushbuparakh removed Khushbuparakh as the assignee of this task.
Khushbuparakh subscribed.
Qgil added a project: Commons.

Not a walk in the park, but possibly within the realm of what a gsoc student could do. Of course it would still require a mentor (I am not volunteering)

I'm bringing back the Possible-Tech-Projects tag, also encouraged by the interest at https://meta.wikimedia.org/wiki/Community_Tech_project_ideas#Support_KML_files_in_Commons

This is a message posted to all tasks under "Backlog" at Possible-Tech-Projects. Outreachy-Round-11 is around the corner. If you want to propose this task as a featured project idea, we need a clear plan with community support, and two mentors willing to support it.

This is a message sent to all Possible-Tech-Projects. The new round of Wikimedia Individual Engagement Grants is open until 29 Sep. For the first time, technical projects are within scope, thanks to the feedback received at Wikimania 2015, before, and after (T105414). If someone is interested in obtaining funds to push this task, this might be a good way.

TasneemLo subscribed.

Adding this to the "Missing Mentors" column.
@Bawolff are you still interested in mentoring this project ?

This is the last call for Possible-Tech-Projects missing mentors. The application deadline for Outreachy-Round-11 is 2015-11-02. If this proposal doesn't have two mentors assigned by the end of Thursday, October 22, it will be moved as a candidate for the next round.

Interested in mentoring? Check the documentation for possible mentors.

As previously mentioned, this task is moved to 'Recheck in February 2016' as it doesn't have two mentors assigned to it as of today, October 23 - 2015. The project will be included in the discussion of next iteration of GSoC/Outreachy, and is excluded from #Outreachy-11. Potential candidates are discouraged from submitting proposals to this task for #Outreachy-11 as it lacks mentors in this round.

I am contributing in Outreachy11 for this Project whom I can contact

I am contributing in Outreachy11 for this Project whom I can contact

@Khushbuparakh Thank you for your interest. The project lacks mentors as of now. Feel free to apply to some other project in https://phabricator.wikimedia.org/tag/possible-tech-projects/ - 'Featured for GSoC and Outreachy' and start by submitting a proposal, following https://www.mediawiki.org/wiki/Outreach_programs/Life_of_a_successful_project#Submitting_your_proposal

IMPORTANT: If you are a community developer interested in working on this task: The Wikimedia Hackathon 2016 (Jerusalem, March 31 - April 3) focuses on #Community-Wishlist-Survey projects. There is some budget for sponsoring volunteer developers. THE DEADLINE TO REQUEST TRAVEL SPONSORSHIP IS TODAY, JANUARY 21. Exceptions can be made for developers focusing on Community Wishlist projects until the end of Sunday 24, but not beyond. If you or someone you know is interested, please REGISTER NOW.
NOTE: This task is a proposed project for Google-Summer-of-Code (2016) and Outreachy-Round-12 : GSoC 2016 and Outreachy round 12 is around the corner, and this task is listed as a Possible-Tech-Projects for the same. Projects listed for the internship programs should have a well-defined scope within the timeline of the event, minimum of two mentors, and should take about 2 weeks for a senior developer to complete. Interested in mentoring? Please add your details to the task description, if not done yet. Prospective interns should go through Life of a successful project doc to find out how to come up with a strong proposal for the same.

I started to work with this problem and hope to finish within this month, probably at Hackathon.

Hi @putnik, did you work on this task at the Hackathon?

@putnik: Any news to share? What's the status? :)

This is a community wishlist task. Regarding the progress on this, could this task use help from an Outreachy intern( Dec 6 to March 6 )? Please note that applications are open until Oct - 17.

Let us know if possible at the earliest.
Ideally it should take about 2-3 weeks for an experienced developer to complete the task, in order to qualify as an intern project.
If the scope is wide, it could be worked out as per the internship needs :)

In any case, it looks like this task is beyond "Needing discussion" and into "Missing mentors"?

I think we should consider a few things when implementing (or not implementing) this task. There are several reasons to store KML/KMZ data on wiki:

  • store a well known geographical object feature, such as a city/country/park/building outline, or a line (i.e. a road/river/...). I believe most of the existing KML data in wiki falls into this category.
  • store an article specific map information, such as the area of an animal habitat, or some transient feature like the army positions during an armed conflict.

I think wiki is not well suited to solve the first use case - OpenStreetMap has the vast number of tools and the needed community's process to keep this information accurate and up to date, and we already have the way to show that data in wiki articles - see examples. The only requirement is that OSM's objects are tagged with the Wikidata IDs.
The second usecase is much more valid for on-wiki storage since we don't support (yet) any external geo data sources. Yet, I think KML/KMZ format is a dead end - it is much harder to process, and the industry has moved towards formats like GeoJSON/TopoJSON, which are better for machine processing at the servers and browsers.
That said, I understand there are many instances where the legacy KML format is still in use. So I think we could implement 3 "levels of support":

  • require users to use numerous online KML to GeoJSON converters, and only store GeoJSON
  • same as above, but also allow a way to download data as KML - doing an on-the-fly conversion
  • same as first two, plus offer a way to upload KML, which will be dynamically converted and stored as GeoJSON.

Per above, I think native KML support (storing data as KML) is counterproductive, and will increase our tech debt without providing much benefit due to the first use case being solved via OSM, and the second one being solved with the native geojson storage.

Any updates on this task? Could one of the subscribers be interested in mentoring this task in our upcoming programs such as Outreachy, GSOC, and RGSOC (railsgirlssummerofcode.org) or if it's beyond your scope, could you help suggest some potential mentors? Thank you very much!

@srishakatux, what do you think about the comment I wrote above in T28059#2726773?

Simple KML and GPX to GeoJSON converter:
https://github.com/mapbox/togeojson

This could be used like similar to write a JS feature to add an 'import' feature to Tabular .map pages, like i did with CSV and Excel files with tabularImportExport.js

Removing the Possible-Tech-Projects tag as we are planning to kill it soon! This project does not seem to fit in the Outreach-Programs-Projects category in its current state, so I am not adding that tag right now!

Because IE6 support (and it's content-analyser) is now dropped (T232563), this ticket might now be implementable for those who want to have a go at it.

@Yurik What do you think of the current state of the field? Still useful or not?