Page MenuHomePhabricator

Allow download of Wikidata query results in KML & GPX
Closed, ResolvedPublic

Description

Following on from T216601: Allow download of Wikidata query results in GPS-friendly format(s), which added geoJSON, this ticket is to add KML and GPX as well.

Details

Other Assignee
Pigsonthewing
Related Changes in Gerrit:
Related Changes in GitLab:
TitleReferenceAuthorSource BranchDest Branch
Bug: T414376repos/wmde/wikidata-query-gui!46atomoilfeature/kml-and-gpxmain
Customize query in GitLab

Event Timeline

I've been looking into the options for encoding arbitrary key-value pairs into XML (both GPX and KML are XML formats). I think XHTML definition lists (aka http://www.w3.org/1999/xhtml or dl, dt and dd nodes) would work for KML as it's more visually led and apache properties (aka http://apache.org/xml/properties or entry node, with key as property and the value as the node content) would work for GPX as it feels more machine read.

I'll add an implementation for XHTML/Apache properties and see how people viewing this ticket/the code feel about them.

An alternative to Apache properties would be RDF (http://www.w3.org/1999/02/22-rdf-syntax-ns). I'm not sure if anyone reading this has a preference, but if you do then let me know. I could always add both if anyone thinks it's worth it.

After some reflection, I decided RDF was a better format than Apache properties, so I switched GPX to use RDF for the extra data. I'm also using the first column in the query as the node's "name" in both formats.

I've also created an MR here - https://gitlab.wikimedia.org/repos/wmde/wikidata-query-gui/-/merge_requests/46

FYI here are the KML and GPX exports from this query:

SELECT ?railway_stationLabel ?railway_station ?coordinate_location WHERE {
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],mul,en". }
  ?railway_station wdt:P31 wd:Q55488;
    wdt:P131 wd:Q2256.
  OPTIONAL { ?railway_station wdt:P625 ?coordinate_location. }
}

LIMIT 100

Hey @Lucas_Werkmeister_WMDE - you very kindly reviewed https://phabricator.wikimedia.org/T216601 in July last year. I've now added the 2 missing geo formats (KML & GPX) and wanted to flag this to you in case you are still the right person to review these additions, or might know who would be. This is the only code I've contributed to the Wikimedia project, so if there is a process to follow to get these tickets reviewed, please don't hesitate to let me know or point me to any docs I should follow.

Sure, if you upload a patch to Gerrit GitLab, I can take a look.

Of the two example files you posted, the KML looks alright to me, but I’m not sure about the GPX. If I understand correctly, the file contains a single track containing a single segment with each result as a point on that segment; as a result, GNOME Maps displays it like this:

image.png (1,920×1,080 px, 1 MB)

The points should probably be more independent from each other, though I don’t know if the GPX format can actually accommodate that. (I don’t know all that much about geographic map formats – I’ve only dabbled a little bit with GPX for cycling tours.)

The article https://en.wikipedia.org/wiki/GPS_Exchange_Format suggests it should be possible to encode waypoints in GPX without making them part of a route. or track.

There's an example of such a file on that page.

hey @Lucas_Werkmeister_WMDE - thanks for replying!

I've got an MR here: https://gitlab.wikimedia.org/repos/wmde/wikidata-query-gui/-/merge_requests/46 is that what you mean by Gerrit (sorry for not knowing the names of things - give me a link if that's not the right place).

Thanks both for the notes on the GPX. I will make some updates to ensure the points are encoded separately so they're not part of a route/track. More soon!

I've improved the GPX export, and now it looks like this:

@Lucas_Werkmeister_WMDE I've looked at https://gerrit.wikimedia.org/r/admin/repos but can't see the wikidata-query-gui repo there, so I'm not sure how to link it properly. I'm happy to do the linking, but I'm just not sure how. I've added a comment on the GitLab MR with your name. I hope that's ok, helpful and not annoying.

Sorry, yes, I forgot that the repo moved to GitLab in the meantime.

Change #1275439 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[operations/deployment-charts@master] wikidata-query-gui: Bump query-gui image version

https://gerrit.wikimedia.org/r/1275439

Change #1275439 merged by jenkins-bot:

[operations/deployment-charts@master] wikidata-query-gui: Bump query-gui image version

https://gerrit.wikimedia.org/r/1275439

I've just tried this with this query:

https://w.wiki/LxtS

If I try to download GeoJSON I get a download dialogue instantaneously, but I get nothing for GPX or KML.

Uncaught TypeError: can't access property "replace", e is undefined

_escapeValuesForXML https://query.wikidata.org/js/wdqs.min.2b73b4309e1722e36149.js:1
e https://query.wikidata.org/js/wdqs.min.2b73b4309e1722e36149.js:1
_processData https://query.wikidata.org/js/wdqs.min.2b73b4309e1722e36149.js:1
getResultAsGPX https://query.wikidata.org/js/wdqs.min.2b73b4309e1722e36149.js:1
i https://query.wikidata.org/js/vendor.min.5ea9dc5b0beb68f52988.js:1
_initHandlersDownloads https://query.wikidata.org/js/wdqs.min.2b73b4309e1722e36149.js:1
dispatch https://query.wikidata.org/js/vendor.min.5ea9dc5b0beb68f52988.js:1
handle https://query.wikidata.org/js/vendor.min.5ea9dc5b0beb68f52988.js:1

Or, less minified:

Uncaught TypeError: can't access property "replace", value is undefined

_escapeValuesForXML http://localhost:8081/wikibase/queryService/api/Sparql.js:681
output http://localhost:8081/wikibase/queryService/api/Sparql.js:595
_processData http://localhost:8081/wikibase/queryService/api/Sparql.js:308
getResultAsGPX http://localhost:8081/wikibase/queryService/api/Sparql.js:580

So this is the following line:

var contentValue = self._escapeValuesForXML( binding.value );

(Likewise on line 649, which is identical but in the KML method instead of the GPX one.) So I guess this needs a check whether the value exists, before we add it to the extensions (GPX) / description (KML)?

Nice edge case! I'll have a look at fixing it now 😎

I've fixed the issue and created a new MR 😎

Change #1276402 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[operations/deployment-charts@master] wikidata-query-gui: Bump query-gui image version

https://gerrit.wikimedia.org/r/1276402

Change #1276402 merged by jenkins-bot:

[operations/deployment-charts@master] wikidata-query-gui: Bump query-gui image version

https://gerrit.wikimedia.org/r/1276402

Seems to be; thanks, both.