Page MenuHomePhabricator

Investigate why some complex maps have been failing
Closed, ResolvedPublic

Description

Let's find out why complex maps like the following one, reported by @Naveenpf in a tweet, have began failing - https://www.mediawiki.org/wiki/Help:Extension:Kartographer#GeoShapes_via_Wikidata_Query. If we find a good answer, we can fix or at least mitigate the problem.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

The underlying error is coming from geoshapes: "SPARQL query result contains non-unique ID \"Q759\""

The wikidata query can be seen here. The problem seem to be that New Hampshire has 2 heads of government for different periods but the query doesn't consider that.

It would be interesting to fix the query (because it's an interesting map) but I think this ticket should be about error handling. Here are some options for dealing with such cases:

  1. Display nothing (current situation)
  2. Display the base map but omit the overlay that fails
  3. Ignore duplicate IDs in geoshapes, log a warning
SBisson lowered the priority of this task from Unbreak Now! to High.EditedMay 1 2018, 1:38 PM

The map at https://www.mediawiki.org/wiki/Help:Extension:Kartographer#GeoShapes_via_Wikidata_Query is fixed because the data in wikidata was changed to have the current head of gov to have the "preferred" rank.

This and many other queries can break when the data changes so we'll have to improve error handling.

The map at https://www.mediawiki.org/wiki/Help:Extension:Kartographer#GeoShapes_via_Wikidata_Query is fixed because the data in wikidata was changed to have the current head of gov to have the "preferred" rank.

@Naveenpf, can we close this ticket?

Awesome, thanks @SBisson for figuring out. At first I thought it was something much more substantial as it coincided with various deployments.

How to debug this failures ?

As a start, you could use the developer tools of your web browser. I went to https://en.wikipedia.org/wiki/List_of_National_Highways_in_India_by_highway_number?debug=true , clicked "Map", and after trying to zoom in, I got an error in the Console:

Error
fileName: "https://en.wikipedia.org/w/extensions/Kartographer/lib/mapbox/mapbox-lib.js?42bb3"
​message: "Invalid GeoJSON object."

The map in this link is failing now.
https://en.wikipedia.org/wiki/List_of_National_Highways_in_India_by_highway_number
How to debug this failures ?

The wikidata query seem to be ok but it returns a lot of results. It times out in pg trying to fetch the data for the geolines.

We see an increased number of timeouts in the log and it's something we want to look at but in the meantime, you can try limiting the number of results to see if it helps.

Two months back 200 NHs where is easily displaying via maplink .
Now only 50 are getting displayed.

https://en.wikipedia.org/wiki/User:Naveenpf/sandbox/limit_test

We discussed this with @Pnorman and our conclusions were:

  • We need better error messages for this case. "Invalid GeoJSON" is not what really happened if we had a timeout looking up the shapes.
  • Paul suggested looking at the GeoJSON that this lookup tries to generate. 200+ highways might correspond to megabytes of GeoJSON. That can be OK for static map generation, but won't be OK for in-browser dynamic maps where all that data needs to be downloaded to and rendered by the client
  • Paul also said that once the schema change associated with the new style (T153282: [epic] Migrate to a new vector tile structure) is done, Wikidata-based queries will be faster
jmatazzoni renamed this task from Complex maps began to fail to Investigate why some complex maps have been failing.May 30 2018, 8:17 PM
jmatazzoni updated the task description. (Show Details)

The "investigate" part is done. The map mentioned in the description had started failing after a change in the data caused the query to return duplicate IDs. This has been fixed in the data.

The work around reporting those errors better will happen in T196015: Create an error message to better report about failings of maps with complex GeoJSON

Vvjjkkii renamed this task from Investigate why some complex maps have been failing to hxdaaaaaaa.Jul 1 2018, 1:14 AM
Vvjjkkii reopened this task as Open.
Vvjjkkii removed SBisson as the assignee of this task.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.

Now 200 NHs are displaying. Analysis of this defect was wrong.
Issue was with qid mapping.
https://en.wikipedia.org/wiki/User:Naveenpf/sandbox/limit_test#limit_test_200

Not all NHs are displayed yet.
It is getting blocked at 206 national highways.