Page MenuHomePhabricator

New imposm3 setup provides some broken links for geoshapes when requesting linestrings
Open, HighPublicBUG REPORT

Description

When using the Maplink template and using OSM data via wikidata, features should populate based on OSM's data. Whether this works has been spotty at best in recent months. I could find no instance of this feature working perfectly, so here are some examples where the functionality comes up short: https://en.wikipedia.org/w/index.php?title=User:BMACS1002/sandbox&oldid=1037627830
I saw one bug report that addressed an aspect of this problem but also said it might be fixed by defining the zoom parameter, which did slightly change one test case, but didn't fix it, as indicated below.

Exhibits:
New Jersey Route 23: Completely broke, no indication that there's any data at all
U.S. 6 (NY): Mostly broke, but when zoom is defined it centers on the object
New Jersey Route 7: No image when outside the viewer, but once within works as expected
New Jersey County Route 506: An example where data from Commons works and appears as expected, but data from OSM doesn't show at all
New Jersey Route 12: partially showing in all views
New Jersey Route 13: Feature shows in its entirety before clicking on the map, but doesn't show when in the viewer
New Jersey Route 18: Works in all circumstances when viewed on its own page, but the preview doesn't show on this page

When looking at the page source in Chrome's developer tools, the code showed exactly the amount of data that was displayed (that being in most cases none at all), so this is not a problem with the browser.

Event Timeline

Pikne added a subscriber: Pikne.

I also notice that a week or a bit more ago OSM overlay objects started to disappear and appear randomly, similar to T268927 in last November. Is this another side-effect of T269582? Earlier, in first half of the year, these overlay objects displayed more reliably, apart from defective static snapshot of new map getting cached in page preview and correct image appearing an hour or so after saving (an issue that surfaced in the end of last year – T269984).

MSantos renamed this task from Kartographer maplink (via Wikidata) provides several broken links to external map services to New imposm3 setup provides some broken links for geoshapes when requesting linestrings.Aug 10 2021, 1:30 PM
MSantos triaged this task as High priority.
MSantos added a subscriber: MSantos.

@BMACS1002 and @Pikne thanks for all the information. I was able to trace it back to a specific behavior caused by how imposm3 imports linestrings from OSM, e. g. relations are treated differently.

We need to fine-tune how the geoshapes service will query this information.

I appreciate your patience while we tweak the recent work to get maps back to a stable state.

It seems that a relation is not displayed with geoline service in particular, while if it's a polygon then it is displayed with geoshape service. E.g. Delaware article now displays blank boundary map as it's set to use geoline service (type=line in given wrapper template), but the relation is displayed (in page preview at least) if it's switched to use geoshape service (type=shape).

If it's also necessary to configure particular types of relations, like route relations, to work properly with imposm, then once at it, maybe in similar manner you can also configure waterway relations that previously weren't available with osm2pgsql (T156433)?

If it's also necessary to configure particular types of relations, like route relations, to work properly with imposm, then once at it, maybe in similar manner you can also configure waterway relations that previously weren't available with osm2pgsql (T156433)?

Yes, AFAICT we could make any OSM relation available with the strategy I'm looking for, which is preparing a table that is specific for geoshapes service to query any geometry that contains wikidata tag. I'm experimenting with this approach and will post here the results once I have them.

Change 713827 had a related patch set uploaded (by MSantos; author: MSantos):

[operations/puppet@production] maps: update imposm mapping

https://gerrit.wikimedia.org/r/713827

Change 713827 merged by Hnowlan:

[operations/puppet@production] maps: update imposm mapping

https://gerrit.wikimedia.org/r/713827

Change 720985 had a related patch set uploaded (by MSantos; author: MSantos):

[maps/kartotherian/deploy@imposm] geoshapes: update table names

https://gerrit.wikimedia.org/r/720985

Change 720985 merged by jenkins-bot:

[maps/kartotherian/deploy@imposm] geoshapes: update table names

https://gerrit.wikimedia.org/r/720985

Relations now appear again. However, it seems that now the problem is the other way around. Previously individual ways with wikidata tag weren't affected, but now I can't find any that'd be available via geoline service. For instance:

I realize this is somewhat unrelated, so maybe there's a better place to ask this, but is there an API for maps.wikimedia.org? I didn't know about this until @Pikne 's last comment

I realize this is somewhat unrelated, so maybe there's a better place to ask this, but is there an API for maps.wikimedia.org? I didn't know about this until @Pikne 's last comment

We are lacking a bit of an simple introduction to that indeed. I made a start here: https://www.mediawiki.org/wiki/Wikimedia_Maps/API

Not sure I understand exactly what the current status of this issue is, but I just wanted to say that the original problem I reported (https://phabricator.wikimedia.org/T286089) is still there: relations whose members are sub-relations and not directly objects with a geometry are not properly handled by geoshapes.

I suppose geoshapes still need to reach planet_osm_line table as well, or tables need further rearrangement?

...the original problem I reported (T286089) is still there: relations whose members are sub-relations and not directly objects with a geometry are not properly handled by geoshapes.

This is old issue. I think superroute relations are rather in scope of T156433.

Well, I'm not the one who decided to join the issue I filed to this one instead of T156433 :-)

So, can you simply readd planet_osm_line next to new wikidata_relation_members in this config file? Similar to polygons that appear to be queried from both wikidata_relation_polygon and planet_osm_polygon.

In Estonian Wikipedia this currently affects about 1200 articles where map is broken due to indivdual OSM way with wikidata tag not being reachable via geoline service.

So, can you simply readd planet_osm_line next to new wikidata_relation_members in this config file? Similar to polygons that appear to be queried from both wikidata_relation_polygon and planet_osm_polygon.

Querying from both tables requires a little extra work AFAIC. I'll investigate that.