Page MenuHomePhabricator
Create Task
Maniphest T319283

GeoPoint feature can't handle "no-value" and "some-value" properties
Closed, ResolvedPublic
Actions

Description

Originally reported here: T188291#8279218

The relevant code can be seen in https://github.com/nyurik/wd-type-parser/blob/master/src/wikidataTypeParser.js#L49. It returns undefined. This currently ends in the final GeoJSON object as { coordinates: undefined }, which will be stringyfied to "geometry": { "type": "Point" }, which is not valid GeoJSON. All this happens after the sanitize-mapdata API was called.

Example page with this issue: https://www.wikidata.org/w/index.php?title=User:Ayack/Draft&oldid=1741989429

Details

SubjectRepoBranchLines +/-
Filter points without coordinates as early as possiblemediawiki/services/kartotherianmaster+21 -22
Filter "unknown" Wikidata coordinates represented as URI stringmediawiki/services/kartotherianmaster+5 -2
Ignore SPARQL results with some-value/no-valuemediawiki/services/kartotherianmaster+17 -1
Rename some variables in geoshape service for readabilitymediawiki/services/kartotherianmaster+16 -16
Remove unused `delete` from geoshapes servicemediawiki/services/kartotherianmaster+1 -1
Customize query in gerrit

Related Objects
Search...

Event Timeline

thiemowmde created this task.Oct 4 2022, 11:51 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 4 2022, 11:51 AM
gerritbot added a comment.Oct 4 2022, 11:57 AM

Change 838124 had a related patch set uploaded (by Thiemo Kreuz (WMDE); author: Thiemo Kreuz (WMDE)):

[mediawiki/services/kartotherian@master] Ignore SPARQL results with some-value/no-value

https://gerrit.wikimedia.org/r/838124

gerritbot added a project: Patch-For-Review.Oct 4 2022, 11:57 AM
gerritbot added a comment.Oct 4 2022, 12:45 PM

Change 838138 had a related patch set uploaded (by Thiemo Kreuz (WMDE); author: Thiemo Kreuz (WMDE)):

[mediawiki/services/kartotherian@master] Rename some variables in geoshape service for readability

https://gerrit.wikimedia.org/r/838138

gerritbot added a comment.Oct 4 2022, 1:45 PM

Change 838157 had a related patch set uploaded (by Thiemo Kreuz (WMDE); author: Thiemo Kreuz (WMDE)):

[mediawiki/services/kartotherian@master] Remove unused `delete` from geoshapes service

https://gerrit.wikimedia.org/r/838157

thiemowmde moved this task from Sprint Backlog to Tech Review on the WMDE-TechWish-Sprint-2022-09-28 board.Oct 4 2022, 1:45 PM
thiemowmde added a project: Unplanned-Sprint-Work.Oct 4 2022, 1:48 PM
gerritbot added a comment.Oct 5 2022, 1:34 PM

Change 838157 merged by jenkins-bot:

[mediawiki/services/kartotherian@master] Remove unused `delete` from geoshapes service

https://gerrit.wikimedia.org/r/838157

thiemowmde mentioned this in rMSKA797b7422e966: Remove unused `delete` from geoshapes service.Oct 5 2022, 1:35 PM
lilients_WMDE moved this task from Backlog to In sprint on the WMDE-GeoInfo-FocusArea board.Oct 5 2022, 2:19 PM
gerritbot added a comment.Oct 6 2022, 9:34 AM

Change 838138 merged by jenkins-bot:

[mediawiki/services/kartotherian@master] Rename some variables in geoshape service for readability

https://gerrit.wikimedia.org/r/838138

thiemowmde mentioned this in rMSKA7265f3d0c024: Rename some variables in geoshape service for readability.Oct 6 2022, 9:35 AM
WMDE-Fisch added a project: WMDE-TechWish-Sprint-2022-10-12.Oct 12 2022, 9:56 AM
ECohen_WMDE moved this task from Sprint Backlog to Tech Review on the WMDE-TechWish-Sprint-2022-10-12 board.Oct 12 2022, 9:56 AM
ECohen_WMDE removed a project: Unplanned-Sprint-Work.Oct 12 2022, 10:40 AM
gerritbot added a comment.Oct 13 2022, 6:52 AM

Change 838124 merged by jenkins-bot:

[mediawiki/services/kartotherian@master] Ignore SPARQL results with some-value/no-value

https://gerrit.wikimedia.org/r/838124

WMDE-Fisch mentioned this in rMSKA7b4e1aad854c: Ignore SPARQL results with some-value/no-value.Oct 13 2022, 6:55 AM
WMDE-Fisch moved this task from Tech Review to Done on the WMDE-TechWish-Sprint-2022-10-12 board.Oct 13 2022, 7:08 AM
Maintenance_bot removed a project: Patch-For-Review.Oct 13 2022, 7:30 AM
TheDJ updated the task description. (Show Details)Oct 13 2022, 10:28 AM
thiemowmde moved this task from In sprint to Epics / Watching on the WMDE-GeoInfo-FocusArea board.Nov 1 2022, 3:01 PM

This is technically done but waiting for the next (manual) deployment of Maps (Kartotherian). Can be closed when the example page works.

WMDE-Fisch mentioned this in T323920: Deploy recent map server work to production ( Jan 2023 ).Jan 6 2023, 9:23 AM
WMDE-Fisch subscribed.Jan 13 2023, 2:57 PM

I tried to test the fix on the beta cluster where it should be deployed now. I used the query from the example page.

https://en.wikipedia.beta.wmflabs.org/w/index.php?title=Maptests/T319283

The static image is broken and when I try to retrieved directly I get the following error message:

"geojson_datasource: Failed parse GeoJSON file from in-memory string  encountered during parsing of layer 'layer' in Layer at line 22"
WMDE-Fisch added a comment.Jan 13 2023, 3:24 PM

FWIW

That's the query result from map server perspective:

https://maps-beta.wmflabs.org/geopoint?getgeojson=1&query=SELECT+%3Fid+%3Fgeo+WHERE+{+%3Fid+wdt%3AP625+%3Fgeo%3B+(wdt%3AP131%2Fwdt%3AP131)+wd%3AQ12620%3B+(wdt%3AP31%2F(wdt%3AP279*))+wd%3AQ16970.+}

That's the JS error on the console:

Error: Invalid LatLng object: (t, h)
    LatLng https://en.wikipedia.beta.wmflabs.org/w/load.php?lang=en&modules=mapbox&skin=vector-2022&version=3l3a6:56
    coordsToLatLng https://en.wikipedia.beta.wmflabs.org/w/load.php?lang=en&modules=mapbox&skin=vector-2022&version=3l3a6:155
    geometryToLayer https://en.wikipedia.beta.wmflabs.org/w/load.php?lang=en&modules=mapbox&skin=vector-2022&version=3l3a6:154
    _initialize https://en.wikipedia.beta.wmflabs.org/w/load.php?lang=en&modules=mapbox&skin=vector-2022&version=3l3a6:233
    _initialize https://en.wikipedia.beta.wmflabs.org/w/load.php?lang=en&modules=mapbox&skin=vector-2022&version=3l3a6:233
    setGeoJSON https://en.wikipedia.beta.wmflabs.org/w/load.php?lang=en&modules=mapbox&skin=vector-2022&version=3l3a6:232
    initialize https://en.wikipedia.beta.wmflabs.org/w/load.php?lang=en&modules=mapbox&skin=vector-2022&version=3l3a6:232
    NewClass https://en.wikipedia.beta.wmflabs.org/w/load.php?lang=en&modules=mapbox&skin=vector-2022&version=3l3a6:44
    featureLayer https://en.wikipedia.beta.wmflabs.org/w/load.php?lang=en&modules=mapbox&skin=vector-2022&version=3l3a6:233
    addGeoJSONLayer https://en.wikipedia.beta.wmflabs.org/wiki/Maptests/T319283#/map/0 line 10 > injectedScript:12
    addDataGroups https://en.wikipedia.beta.wmflabs.org/wiki/Maptests/T319283#/map/0 line 10 > injectedScript:12
    addDataGroups https://en.wikipedia.beta.wmflabs.org/wiki/Maptests/T319283#/map/0 line 10 > injectedScript:12
thiemowmde added a comment.EditedJan 13 2023, 3:30 PM

The raw API request for this is https://en.wikipedia.beta.wmflabs.org/wiki/Special:ApiSandbox#action=query&prop=mapdata&titles=Maptests%2FT319283&mpdgroups=_010eee0e48d74d313740293a44e9b68c4548f8ab. The JSON is valid. The GeoJSON string can be extracted and parsed as well. But it's not valid GeoJSON. It contains weird things like "coordinates": "http://www.wikidata.org/.well-known/genid/51370059e35df6afff396e90f00a7294" in several places.

The SPARQL query is https://query.wikidata.org/#SELECT%20%3Fid%20%3Fgeo%20WHERE%20%7B%20%3Fid%20wdt%3AP625%20%3Fgeo%3B%20%28wdt%3AP131%2Fwdt%3AP131%29%20wd%3AQ12620%3B%20%28wdt%3AP31%2F%28wdt%3AP279%2a%29%29%20wd%3AQ16970.%20%7D. The result does indeed contain these weird broken coordinates.

Following the links to e.g. https://www.wikidata.org/wiki/Q3585894#P625 shows these are "unknown" values.

Here is the code that generates these weird genids: https://codesearch.wmcloud.org/search/?q=generateWellKnownURI.

I think we need to filter this the same way we filter "no value".

gerritbot added a comment.Jan 13 2023, 3:35 PM

Change 879821 had a related patch set uploaded (by Thiemo Kreuz (WMDE); author: Thiemo Kreuz (WMDE)):

[mediawiki/services/kartotherian@master] Filter "unknown" Wikidata coordinates represented as URI string

https://gerrit.wikimedia.org/r/879821

gerritbot added a project: Patch-For-Review.Jan 13 2023, 3:35 PM
gerritbot added a comment.Jan 13 2023, 3:52 PM

Change 879821 merged by jenkins-bot:

[mediawiki/services/kartotherian@master] Filter "unknown" Wikidata coordinates represented as URI string

https://gerrit.wikimedia.org/r/879821

thiemowmde mentioned this in rMSKAab86fc173ca7: Filter "unknown" Wikidata coordinates represented as URI string.Jan 13 2023, 3:52 PM
WMDE-Fisch added a project: WMDE-TechWish-Sprint-2023-01-04.Jan 13 2023, 3:57 PM
WMDE-Fisch moved this task from Sprint Backlog to Demo on the WMDE-TechWish-Sprint-2023-01-04 board.
Maintenance_bot removed a project: Patch-For-Review.Jan 13 2023, 4:30 PM
Lena_WMDE added a parent task: T323920: Deploy recent map server work to production ( Jan 2023 ).Jan 17 2023, 9:12 AM
awight moved this task from Demo to Watching / Epic / Stalled on the WMDE-TechWish-Sprint-2023-01-04 board.Jan 17 2023, 9:26 AM
thiemowmde closed this task as Resolved.Jan 18 2023, 9:19 AM
gerritbot added a comment.Jan 18 2023, 9:28 AM

Change 881358 had a related patch set uploaded (by Thiemo Kreuz (WMDE); author: Thiemo Kreuz (WMDE)):

[mediawiki/services/kartotherian@master] Filter points without coordinates as early as possible

https://gerrit.wikimedia.org/r/881358

gerritbot added a project: Patch-For-Review.Jan 18 2023, 9:28 AM
gerritbot added a comment.Feb 1 2023, 10:13 AM

Change 881358 merged by jenkins-bot:

[mediawiki/services/kartotherian@master] Filter points without coordinates as early as possible

https://gerrit.wikimedia.org/r/881358

thiemowmde mentioned this in rMSKA3a92218488db: Filter points without coordinates as early as possible.Feb 1 2023, 10:13 AM
Maintenance_bot removed a project: Patch-For-Review.Feb 1 2023, 10:31 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL