Investigate why some complex maps have been failing
Closed, ResolvedPublic

Description

Let's find out why complex maps like the following one, reported by @Naveenpf in a tweet, have began failing - https://www.mediawiki.org/wiki/Help:Extension:Kartographer#GeoShapes_via_Wikidata_Query. If we find a good answer, we can fix or at least mitigate the problem.

Yurik created this task.May 1 2018, 3:26 AM
Restricted Application added a project: Discovery. · View Herald TranscriptMay 1 2018, 3:26 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Yurik added a subscriber: Catrope.
Restricted Application added subscribers: Liuxinyu970226, TerraCodes. · View Herald TranscriptMay 1 2018, 3:47 AM

The underlying error is coming from geoshapes: "SPARQL query result contains non-unique ID \"Q759\""

The wikidata query can be seen here. The problem seem to be that New Hampshire has 2 heads of government for different periods but the query doesn't consider that.

It would be interesting to fix the query (because it's an interesting map) but I think this ticket should be about error handling. Here are some options for dealing with such cases:

  1. Display nothing (current situation)
  2. Display the base map but omit the overlay that fails
  3. Ignore duplicate IDs in geoshapes, log a warning
SBisson lowered the priority of this task from Unbreak Now! to High.EditedMay 1 2018, 1:38 PM

The map at https://www.mediawiki.org/wiki/Help:Extension:Kartographer#GeoShapes_via_Wikidata_Query is fixed because the data in wikidata was changed to have the current head of gov to have the "preferred" rank.

This and many other queries can break when the data changes so we'll have to improve error handling.

The map at https://www.mediawiki.org/wiki/Help:Extension:Kartographer#GeoShapes_via_Wikidata_Query is fixed because the data in wikidata was changed to have the current head of gov to have the "preferred" rank.

@Naveenpf, can we close this ticket?

Yurik added a comment.May 1 2018, 4:37 PM

Awesome, thanks @SBisson for figuring out. At first I thought it was something much more substantial as it coincided with various deployments.

How to debug this failures ?

As a start, you could use the developer tools of your web browser. I went to https://en.wikipedia.org/wiki/List_of_National_Highways_in_India_by_highway_number?debug=true , clicked "Map", and after trying to zoom in, I got an error in the Console:

Error
fileName: "https://en.wikipedia.org/w/extensions/Kartographer/lib/mapbox/mapbox-lib.js?42bb3"
​message: "Invalid GeoJSON object."

The map in this link is failing now.
https://en.wikipedia.org/wiki/List_of_National_Highways_in_India_by_highway_number
How to debug this failures ?

The wikidata query seem to be ok but it returns a lot of results. It times out in pg trying to fetch the data for the geolines.

We see an increased number of timeouts in the log and it's something we want to look at but in the meantime, you can try limiting the number of results to see if it helps.

Two months back 200 NHs where is easily displaying via maplink .
Now only 50 are getting displayed.

https://en.wikipedia.org/wiki/User:Naveenpf/sandbox/limit_test

Catrope added a subscriber: Pnorman.May 8 2018, 6:04 PM

We discussed this with @Pnorman and our conclusions were:

  • We need better error messages for this case. "Invalid GeoJSON" is not what really happened if we had a timeout looking up the shapes.
  • Paul suggested looking at the GeoJSON that this lookup tries to generate. 200+ highways might correspond to megabytes of GeoJSON. That can be OK for static map generation, but won't be OK for in-browser dynamic maps where all that data needs to be downloaded to and rendered by the client
  • Paul also said that once the schema change associated with the new style (T153282: [epic] Migrate to a new vector tile structure) is done, Wikidata-based queries will be faster
Haros added a subscriber: Haros.May 23 2018, 9:33 AM
jmatazzoni renamed this task from Complex maps began to fail to Investigate why some complex maps have been failing.May 30 2018, 8:17 PM
jmatazzoni updated the task description. (Show Details)
SBisson closed this task as Resolved.Jun 11 2018, 7:59 PM

The "investigate" part is done. The map mentioned in the description had started failing after a change in the data caused the query to return duplicate IDs. This has been fixed in the data.

The work around reporting those errors better will happen in T196015: Create an error message to better report about failings of maps with complex GeoJSON

Vvjjkkii renamed this task from Investigate why some complex maps have been failing to hxdaaaaaaa.Jul 1 2018, 1:14 AM
Vvjjkkii reopened this task as Open.
Vvjjkkii removed SBisson as the assignee of this task.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
AfroThundr3007730 renamed this task from hxdaaaaaaa to Investigate why some complex maps have been failing.Jul 1 2018, 6:12 AM
AfroThundr3007730 closed this task as Resolved.
AfroThundr3007730 assigned this task to SBisson.
AfroThundr3007730 updated the task description. (Show Details)
AfroThundr3007730 added a subscriber: Aklapper.

Now 200 NHs are displaying. Analysis of this defect was wrong.
Issue was with qid mapping.
https://en.wikipedia.org/wiki/User:Naveenpf/sandbox/limit_test#limit_test_200

Not all NHs are displayed yet.
It is getting blocked at 206 national highways.