Page MenuHomePhabricator

Remove pushpin markers in Kartographer maps for geoline objects
Open, Needs TriagePublic0 Estimated Story PointsBUG REPORT

Description

  • Look for any use cases where the points are being used intentionally. For example, subway station markers are being styled.
  • Review and merge the short-term fix to filter out nodes.
  • Create future task for filtering points out of the OSM import. The data might be useable, so instead I'm splitting out a future discussion about integrating under a new feature.

Since a few days, on Kartographer maps pushpin markers are shown in case of geoline objects. Pushpin markers should be shown only for single coordinates. The markers were placed by a Javascript script.

List of steps to reproduce (step by step, including full links if applicable):

What happens?:

  • There is a grey and unnumbered pushpin marker above the word "Ägypten" which belongs to the country's border line (a geoshape object). The is a similar behavior in case of geoline objects (for instance on public transport lines).

What should have happened instead?:

  • No pushpin marker should be shown (the behavior as in the past five years).

Software version (if not a Wikimedia wiki), browser information, screenshots, other information, etc:

  • Firefox 92.0.1

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

It seems to me that markers actually aren't included for geoshape objects (polygons). If I change "geoline" to "geoshape" in page preview of given example, then the unwanted marker goes away. In given example polygon representation (using transparent fill if necessary) also seems more adequate as referenced OSM object is for an area, not only its boundary.

But elsewhere where line representation is appropriate and OSM relation in addition to member ways has member nodes, it'd probably be better to omit these nodes indeed. With nodes the selection of data is less flexible and also we can't set what exactly these nodes stand for. I assume member nodes are currently included due to updated imposm mapping, see T288400.

TheDJ added a subscriber: TheDJ.

https://maps.wikimedia.org/geoline?getgeojson=1&ids=Q79
P17431

This shows that the geoline service returns these points. It seems to be the location of the label, and the location of the capital of the country/region...
If this is new then it's definitely a Karthoterian issue, not the Kartographer side.

K. it might be new... but i do note that OSM itself also shows this geometry with two points. So the data is probably correct, but it might be that we filtered this out in the past etc etc. Hard to say.

I found the bug described for geoline objects, too. See for instance: https://de.wikivoyage.org/wiki/Vorlage:Mapshapes . The map shows several subway lines of Vienna whereas the stations ar marked with a pushpin marker.

If you call https://maps.wikimedia.org/geoline?getgeojson=1&ids=Q873397 to get the geoline object of U2 subway you will detect a lot of point objects which should not be part of a line. You can see from the GeoJSON array returned:

points-in-geolines.jpg (742×668 px, 35 KB)

I think the problem can be solved by removing the point features and keeping the Linestring or similar objects.

By the way, the pushpin markers are not only needless and cover the geoshape of interest. The authors have no mean to modify the pushpin (colors, sizes etc).

I agree that it is suboptimal. On the other hand, it is data which is in the OSM database, I don't think it is unreasonable to have the geoshape service return it too.. Maybe we should filter them out when rendering ?

It's hard to decide without more people weighing in :(

Those pushpins should definitely be filtered out. I haven't seen a single dynamic map where those pins provide any added value. They only clutter the map.

This was discussed on Wikivoyage as well: https://en.wikivoyage.org/wiki/Template_talk:Mapshapes#Grey_dots_are_now_appearing_on_all_maps_with_railway_lines
This is, in my opinion, the most pressing problem related to dynamic maps in all Wikimedia projects.
Cannot be more than a few lines of code to fix it?

Cannot be more than a few lines of code to fix it?

Cool, contributions are welcome

Good idea. Somehow Kartographer doesn't show up in the list of software projects though (https://www.mediawiki.org/wiki/New_Developers#Choose_a_software_project)

Hi, see https://phabricator.wikimedia.org/project/profile/1829/ for the Source Repo (as it's not a project recommended for completely new developers). :) Thanks!

Hi, see https://phabricator.wikimedia.org/project/profile/1829/ for the Source Repo (as it's not a project recommended for completely new developers). :) Thanks!

Aklapper, thanks for the link. I found some set-up instructions. Is there a place where I can ask questions if I get stuck?

I agree that it is suboptimal. On the other hand, it is data which is in the OSM database, I don't think it is unreasonable to have the geoshape service return it too.. Maybe we should filter them out when rendering ?

It's hard to decide without more people weighing in :(

It's not just suboptimal, but it is leading to nearly unusable maps: https://de.wikivoyage.org/wiki/Emmerich_am_Rhein#Mobilit%C3%A4t

I hope the priority of this bug will be raised.

Maybe it makes sense to filter these markers while rendering. But the communities have no change to do it by themselves. At least, there should be a hook to add this filter to MediaWiki:Kartographer.js.

Meanwhile, I added a CSS rule to MediaWiki:Kartographer.css to suppress the grey markers. This rule can alternately be added to MediaWiki:Common.css/Mediawiki:Mobile.css.

/* Removing grey pushpin markers on mapframe maps */
img[src$="pin-m+7e7e7e@2x.png"] {
	display: none;
}

... it might be that we filtered this out in the past etc etc. Hard to say.

I'm fairly sure points were filtered out. This is affected by patches in T288400. Previously geolines were queried from planet_osm_line table which has linestring table type set in imposm_mapping.yml. Now geolines come from wikidata_relation_members table of relation_member table type (not limted to linestrings). @MSantos: maybe it wouldn't be too difficult to adjust the mapping so that this new table would also include only linestrings?

Pikne renamed this task from Remove pushpin markers in Kartographer maps for geoline and geoshape objects to Remove pushpin markers in Kartographer maps for geoline objects.May 7 2022, 7:58 AM
Pikne updated the task description. (Show Details)

Now geolines come from wikidata_relation_members table of relation_member table type (not limted to linestrings). @MSantos: maybe it wouldn't be too difficult to adjust the mapping so that this new table would also include only linestrings?

This looks like a promising approach! The OSM relationship corresponding to Q873397 above is https://www.openstreetmap.org/relation/27114 , which is composed of node and way members.

We can probably set up a second level of filtering in kartotherian, as an immediate workaround. For example, compare the results of:
select ST_AsGeoJson(geometry) from wikidata_relation_members where wikidata='Q168162' and GeometryType(geometry) != 'POINT'
select ST_AsGeoJson(geometry) from wikidata_relation_members where wikidata='Q168162' and GeometryType(geometry) == 'POINT'

Change 790260 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/services/kartotherian@master] [WIP] Filter out points from geoshape and geoline query

https://gerrit.wikimedia.org/r/790260

Part of this ticket should also be creating follow up tickets for a long term fix to filter points in the imposm import.

awight updated the task description. (Show Details)
awight set the point value for this task to 5.May 12 2022, 12:49 PM

I tried to dig a bit into the code that is ultimately responsible for these points.

  • https://gerrit.wikimedia.org/g/operations/puppet/+/production/modules/osm/files/imposm_mapping.yml appears to be the configuration we are looking for. Specifically the wikidata_relation_members part.
  • relation_member here means this is going to import all members of an OSM relation that have a wikidata item id assigned. This can include points.
  • According to the documentation at https://imposm.org/docs/imposm3/latest/relations.html it should be possible to add something like filters: ↲ reject: ↲ type: point at the end of the wikidata_relation_members section to exclude points.
  • It might be necessary to also add - name: type ↲ type: member_type to the list of columns: to make this work. (This is really confusing because type means two different things here.)
  • However. The name of the table wikidata_relation_members sounds like it's used for other things as well, not only for the geoline feature we are talking about here. To not accidentally destroy other features that want these points we would need to stick to https://gerrit.wikimedia.org/r/790260 then. Which might be ok.

Thank you @thiemowmde for digging into the imposm mapping format and finding some filters we can try. It should be possible to test this locally on a subset of the data.

I looked for any pages where geoline is used in combination with any of the marker- styles, which would indicate that users were relying on the points intentionally.

./search_insource 'insource:geoline insource:/geoline.+marker-/'

There were a few (40 or so) hits, and they all seem to be false positives, just a geoline coexisting on the page with other points.

Just for fun, I've demonstrated that the usage is possible but seems to not be done anywhere (and is undocumented). Given all these facts, it makes sense to disable support now and possibly have a later conversation about whether it's desirable to reintroduce a feature like this on purpose.

For reference, this is how the demo looks like:

Screenshot from 2022-05-13 16-23-19.png (470×1 px, 619 KB)

What's special about this is that the (accidental, not officially supported) markers are styled in red, and with an icon. Source code:

<mapframe width="full" height="500">
{
    "type": "ExternalData",
    "service": "geoline",
    "ids": "Q873397",
    "properties": {
        "marker-symbol": "rail-underground",
        "marker-color": "#f00"
    }
}
</mapframe>

In case we ever want to officially support something like this, it should be a separate feature, separated from geoline.

Change 790260 merged by jenkins-bot:

[mediawiki/services/kartotherian@master] Filter out points from geoshape and geoline query

https://gerrit.wikimedia.org/r/790260

awight updated the task description. (Show Details)
awight moved this task from Tech Review to Demo on the WMDE-TechWish-Sprint-2022-05-11 board.

Still needs to be deployed to the beta cluster. Locally demonstrated.

Here's a demonstration that we've fixed the bug with the latest code:
https://de.wikipedia.org/wiki/Benutzer:Adamw/Spielwiese/T292613 (note that the snapshot is broken, and the interactive map shows a marker near the center)
vs.
https://en.wikipedia.beta.wmflabs.org/wiki/User:Adamw/sandbox/T292613 (just a line, in snapshot and interactive maps)

Another example that should be fixed with this solution:
map of Central Tokyo

awight removed awight as the assignee of this task.Jul 6 2022, 8:41 AM

Several issues that are probably related to this bug are collected in this task: T310289

Here's a demonstration that we've fixed the bug with the latest code:
https://de.wikipedia.org/wiki/Benutzer:Adamw/Spielwiese/T292613 (note that the snapshot is broken, and the interactive map shows a marker near the center)
vs.
https://en.wikipedia.beta.wmflabs.org/wiki/User:Adamw/sandbox/T292613 (just a line, in snapshot and interactive maps)

Great work, Adam! The Tokyo article on Wikivoyage seems to be fine now (no more grey markers). If you go to the article MRT Circle Line, click on edit, uncomment the public transport lines for the dynamic map and preview it then all the grey markers are still there. Maybe the update needs some time to be applied to all Wikimedia sites?

The Tokyo article on Wikivoyage seems to be fine now (no more grey markers).

This is due to local CSS hack (see T292613#7911179), added to English Wikivoyage a few days ago.

Awight's patch for some reason hasn't been deployed in production yet, as far as I can see.

awight changed the point value for this task from 5 to 0.Wed, Jul 20, 10:44 AM