Page MenuHomePhabricator

Epic: Borders aren't always marked as disputed
Open, Stalled, HighPublic

Description

From https://www.mediawiki.org/wiki/Talk:Maps#Controversial_borders written by @Amire80:

I noticed some controversial borders at https://maps.wikimedia.org :

  • The border between Crimea and Kherson is shown as international, which is the Russian Federation POV.
  • South Ossetia and Abkhazia are shown as countries, which is the Russian Federation POV.
  • The border of Kosovo looks like a border of an independent country (Kosovan POV), but it doesn't have a country name (weird POV ;) ).
  • As a technically opposite example, there's no border in the middle of Cyprus (Greek POV), but there is a label for the Turkish Republic of Northern Cyprus (Turkish POV).
  • The border between Israel and Syria is shown according to the Israeli POV.

There are possibly more issues of this kind. Wikipedias usually show maps of disputed territories quite well. For example, maps of Russia and Ukraine usually show Crimea in a special color - even in the Russian Wikipedia. I understand that at the moment Wikimedia's maps service probably just takes whatever OSM provides, but ideally it should do something similar to what the Wikipedias do and be as neutral as possible.

Related Objects

StatusAssignedTask
DeclinedNone
DeclinedNone
DeclinedPnorman
DeclinedPnorman
ResolvedPnorman
InvalidNone
StalledPnorman
ResolvedPnorman
DeclinedNone
OpenPnorman
DeclinedNone
DeclinedNone
DeclinedNone
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedMholloway
ResolvedPnorman
DeclinedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
DeclinedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
DuplicateNone
ResolvedPnorman
ResolvedNone
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
Resolveddebt
ResolvedPnorman
ResolvedPnorman
ResolvedNone
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
DeclinedPnorman
ResolvedPnorman
DeclinedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
DeclinedNone
DeclinedGehel
ResolvedMholloway
ResolvedPnorman
ResolvedGehel
ResolvedPnorman
ResolvedGehel
ResolvedGehel
ResolvedGehel
ResolvedNone
DeclinedNone
ResolvedPnorman
ResolvedNone
ResolvedNone
Resolvedmobrovac
ResolvedGehel
ResolvedPnorman
DeclinedNone
DeclinedNone
DeclinedNone
DeclinedNone
DeclinedNone
DeclinedNone
DeclinedNone
ResolvedGehel
ResolvedSBisson
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedGehel
ResolvedGehel
ResolvedGehel
ResolvedPnorman
ResolvedMholloway
ResolvedGehel
ResolvedCKoerner_WMF
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
DeclinedCKoerner_WMF

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Yurik raised the priority of this task from to Needs Triage.Sep 18 2015, 1:13 AM
Yurik added projects: Maps, Discovery, Maps-Sprint.
Yurik added subscribers: Yurik, MaxSem, Tfinc, Amire80.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 18 2015, 1:13 AM
Yurik updated the task description. (Show Details)Sep 18 2015, 1:13 AM
Yurik set Security to None.

There are possibly more issues of this kind.

Oh yes, e.g. India vs Pakistan. I'd call all that "disputed" instead of "controversial" though.

Elitre added a subscriber: Elitre.Sep 18 2015, 12:18 PM
MaxSem renamed this task from Controversial Map Borders to Borders' aren't always marked as disputed.Sep 18 2015, 6:07 PM
MaxSem claimed this task.Sep 18 2015, 8:28 PM
MaxSem moved this task from Backlog to Needs review on the Maps-Sprint board.

Israel-(.*) borders are trickier because they're not disputed but marked with "border_type:armistice line".

Elitre renamed this task from Borders' aren't always marked as disputed to Borders aren't always marked as disputed.Sep 19 2015, 8:04 AM
AS added a subscriber: AS.Sep 21 2015, 7:23 PM
Deskana moved this task from Needs triage to Maps on the Discovery board.Sep 22 2015, 5:27 PM
MaxSem moved this task from Needs review to In progress on the Maps-Sprint board.Sep 22 2015, 9:17 PM
MaxSem added a comment.EditedSep 22 2015, 11:54 PM


Okay, so P2078 is showing what's going on on the above tile: lots of overlapping geometries from different rels but just 2 (?) underlying lines. Need to filter somehow the lines based on admin level and disputed flag.

ksmith moved this task from Maps to On Sprint Board on the Discovery board.Sep 24 2015, 5:13 PM
MaxSem moved this task from In progress to To-do on the Maps-Sprint board.Sep 24 2015, 6:19 PM

Using polygons for borders in inherently problematic for this, as you can't have part of the polygon contested and part not.

The polygons need to be turned into lines and tags from the ways moved to the lines and combined with tags from the relations

There are three obvious ways to do this

  • In SQL, at query time
  • Processing OSM data periodically, similar to how coastlines are periodically generated
  • With osm2pgsql

The first involves complex SQL
The second involves more complex deployments as now there's another set of data that needs updating.
The last involves Lua transforms, and should be theoretically possible

https://github.com/mapbox/osm-boundaries is a tool that Mapbox used to use for boundaries. It's not known if they still do.

Tfinc moved this task from To-do to Backlog on the Maps-Sprint board.Oct 6 2015, 9:59 PM
MaxSem removed MaxSem as the assignee of this task.Oct 6 2015, 9:59 PM
Restricted Application added a subscriber: StudiesWorld. · View Herald TranscriptNov 7 2015, 7:04 AM
Yurik moved this task from Styling & datasources to Tilerator on the Maps board.Nov 7 2015, 7:12 AM
Yurik moved this task from Tilerator to Styling & datasources on the Maps board.Feb 7 2016, 10:06 PM
Yurik moved this task from Backlog to Bugs on the Maps (Map-Styles) board.Mar 6 2016, 2:32 AM
debt added a subscriber: debt.Oct 12 2016, 8:28 PM
Pnorman claimed this task.Nov 7 2016, 5:26 PM

Looking at the borders and not administrative or place rendering and current OSM data

The border between Crimea and Kherson is shown as international, which is the Russian Federation POV.

Border ways have admin_level=2 disputed=yes and are members of relations with

  • admin_level=2 name:en=Russian Federation
  • admin_level=2 name:en=South federal district
  • admin_level=4 name:en=Autonomous Republic of Crimea
  • admin_level=4 name:en=Kherson Oblast
  • admin_level=4 name:en=Republic of Crimea
  • various higher level admin_levels

South Ossetia and Abkhazia are shown as countries, which is the Russian Federation POV.

Border ways have admin_level=3 disputed=yes and are members of relations with

  • admin_level=3 border_type=disputed name:en=Abkhazia
  • admin_level=4 name:en=Autonomous Republic of Abkhazia
  • admin_level=4 name:en=Samegrelo-Upper Svaneti

The border of Kosovo looks like a border of an independent country (Kosovan POV), but it doesn't have a country name (weird POV ;) ).

Too general an area to look at the specific tagging

As a technically opposite example, there's no border in the middle of Cyprus (Greek POV), but there is a label for the Turkish Republic of Northern Cyprus (Turkish POV).

Border ways have admin_level=3 and are members of relations with

  • boundary=administrative_fraction admin_level=3 name:en=Kuzey Kıbrıs
  • boundary=administrative_fraction admin_level=4 name:en=Cyprus

The border between Israel and Syria is shown according to the Israeli POV.

Too many borders are involved for me to be certain which ones are at issue.

In addition, osm-boundaries checks for disputed=* dispute=* border_status=dispute

debt triaged this task as High priority.Nov 7 2016, 6:25 PM
debt added a project: Maps-Sprint.
debt moved this task from Backlog to In progress on the Maps-Sprint board.
Yurik added a comment.Nov 7 2016, 8:27 PM

@Pnorman thanks for documenting it! We should eventually have a clear and concise guide on how borders are generated, and point everyone to that page (probably on OSM wiki). This way if anyone has a question on why a certain border is drawn a certain way, they will be able to either fix it themselves, or at least start a actionable discussion.

debt added a comment.Nov 9 2016, 7:24 PM

Just a friendly reminder - once we get the guidelines set, we'll need to document it on wiki, preferably somewhere off of this page: https://www.mediawiki.org/wiki/Maps.

@Pnorman - once it has been established what we want to do with disputed borders, I can add the guidelines to mediawiki, just let me know when it's ready! :)

Yurik renamed this task from Borders aren't always marked as disputed to Epic: Borders aren't always marked as disputed.Nov 9 2016, 7:33 PM
Yurik edited projects, added Epic; removed Maps-Sprint.

boundary=administrative relations and tags on ways seem to be enough for Crimea/Russia, China/South Asia, and Canada/US. Looking into what else I need to process for the other ex-USSR regions and Israel.

MaxSem added a comment.Dec 1 2016, 2:57 AM

The trick here is not to determine whether a border is disputed, that's very straightforward. The problem here is to filter several overlapping borders so that only the right one, disputed, is visible.

Yurik added a comment.Dec 1 2016, 3:12 AM

@MaxSem, I suspect that's what Paul's new code actually does - determines if multiple ways (lines) go on top of each other, and when they do, it only adds one line to the database with the "highest level" (lowest admin number) tag.

@MaxSem, I suspect that's what Paul's new code actually does - determines if multiple ways (lines) go on top of each other, and when they do, it only adds one line to the database with the "highest level" (lowest admin number) tag.

It does it based on way to parent relation information, but yes, there will only ever be one linestring generated for a way.

Yurik removed a project: Maps.Dec 15 2016, 4:35 AM
mxn added a subscriber: mxn.Jun 10 2017, 5:29 AM

An Indian user is dispmayed by our current handling of these areas:

While I appreciate the great effort [of internationalized maps], I regret that we may not be able to use it in India as POK and Aksai Chin are not shown as parts of India

@Pnomrnan, I assume those are on your list to handle as disputed? Can you confirm.

There isn't a list to show as disputed, except one for some testing in development. What we show as disputed is what is recorded as disputed in OpenStreetMap, which is fairly comprehensive.

For the region between India and China, we won't be showing it as part of one or the other, but as disputed. If the situation on the ground changes then when OpenStreetMap updates its data, we'll change what we display.

An Indian user is dispmayed by our current handling of these areas

As someone who has worked extensively on Indian border maps on Wikipedia and mapped them on OSM, i'd like to provide some more context:

  • The Indian Government criminalizes the publication of any map that does not show its version of the borders. Reference map.
  • The border has two large disputed regions, Kashmir (with Pakistan and China) and Arunachal Pradesh (with China). Reference map of all claims by every country.
  • OSM has the data for both the defacto boundaries that are currently dispute (already visible on the map) and extra claimed boundaries that extend into the neighboring country (not visible). Overpass query for complete Indian boundaries.
  • Satisfying the legal requirements for proper Indian maps will involve two steps
    1. Showing the defacto disputed boundaries as dotted or hiding them completely
    2. Showing the claimed areas into neighboring countries as part of India

Noting these complications, its impossible to have have a world map that can be a neutral world map as well as be legal in India. There are similar complications in making a map that would be legal in China due to identical map policies there which enforce a differing world view (where one state of India is to be shown as an undisputed territory of China).

My take is such cases of differing world views is best handled on the client side by overlaying the map tiles with custom data for the required end result. This can be fine tuned by each Wikipedia community as required to satisfy local laws.

Have made a quick demo of such an overlay to create a legal Indian map with the Wikimedia map tiles https://jsfiddle.net/planemad/oeeLmg1j/1/

The demo uses Mapbox GL, but it should be easy enough to replicate on leaflet and would not need to load any extra libraries.

stjn added a subscriber: stjn.EditedMay 3 2018, 11:25 AM

Satisfying any local laws apart from US laws is inherently wrong approach on going on with this. It is illegal to dispute that Crimea is part of Russia in Russia, too, but that doesn’t mean that Russian community should be non-neutral in approaching this subject and display Crimea as a part of Russia without any mention of disputed status on maps. Communities that reside in India should also defend the principle of neutrality, not bend over backwards to satisfy legal requirements of Indian authorities.

My take is such cases of differing world views is best handled on the client side by overlaying the map tiles with custom data for the required end result.

What do you mean by "client side"?

This can be fine tuned by each Wikipedia community as required to satisfy local laws.

Wikipedias are divided by language, not by country.

Yandex Maps shows Crimea as part of Russia no matter where you open it, probably because it's a Russian company (they certainly show them like this in Russia and in Israel; I'm not sure what do they show to users in Ukraine, if they work there at all). Google Maps show Crimea as part of Ukraine in Israel, but from what I heard, they show Crimea as part of Russia to users in Russia.

Wikipedia doesn't do it. Wikipedia shows the same thing in the whole world. The editions of Wikipedia in Russian and in English are widely read in both Ukraine and in Russia. They must show the same information about the borders of Crimea everywhere, and to represents all points of view. Wikipedia editors argue a lot, but they are surprisingly good at reaching an agreement with each other about representing maps, despite having very different political opinions.

The same goes for India and Pakistan: the English Wikipedia is widely read in both countries.

Satisfying any local laws apart from US laws is inherently wrong approach on going on with this. It is illegal to dispute that Crimea is part of Russia in Russia, too, but that doesn’t mean that Russian community should be non-neutral in approaching this subject and display Crimea as a part of Russia without any mention of disputed status on maps. Communities that reside in India should also defend the principle of neutrality, not bend over backwards to satisfy legal requirements of Indian authorities.

Indeed, and the same goes for all countries. India is far from being the only one that has a map controversy. See the task description for more examples, and that is still not the end of it.

If Wikipedia ever does anything like what Google do about this, it will be a major crisis. Let's not get there. Let's not get to having a Pakistani English Wikipedia, an Indian English Wikipedia, an American English Wikpiedia, and so on for other countries and languages. I'm not a lawyer. I don't know what can be done legally to keep being neutral. I do care about the principle of being a universal project.

stjn added a comment.EditedMay 3 2018, 1:25 PM

Yandex Maps shows Crimea as part of Russia no matter where you open it, probably because it's a Russian company (they certainly show them like this in Russia and in Israel; I'm not sure what do they show to users in Ukraine, if they work there at all).

[offtopic] On maps.yandex.ua Yandex follows Ukrainian point of view, yes. We are not a service that does this kind of geographical settings, and it would be a dangerous road if we went there (for one, because nothing after this stops users from saying that ‘if Wikimedia Maps cater to preferences of countries, then our local files should too’).

Wikipedia doesn't do it. Wikipedia shows the same thing in the whole world. The editions of Wikipedia in Russian and in English are widely read in both Ukraine and in Russia. They must show the same information about the borders of Crimea everywhere, and to represents all points of view. Wikipedia editors argue a lot, but they are surprisingly good at reaching an agreement with each other about representing maps, despite having very different political opinions.

Totally agree. Wikipedia should not be serving tiles that are non neutral, that goes against the fundamentals of the project.

There are valid reasons where local users might want to customize the map as a user setting, for eg. during a public presentation or to government officials where broadcasting such maps have the potential to cause trouble. This is a reality - the Wiki Conference in Mumbai in 2011 had political protests and negative press coverage because of this issue and dragged many community members into police purview.

As an Indian Wikimedian, the minimum that can be done to support the Indic community on this issue is to technically make it easy for a user to be able to customize the maps if they so wish on a personal level, maybe by using a gadget or plugin that achieves the same result as https://jsfiddle.net/planemad/oeeLmg1j/1/

I'll just add - neither am I the user who is dismayed at the current rendering (my history will show i have contributed hundreds of neutral Indian maps to Wikipedia), nor do I feel project should break its neutral core. My goal was to explain what the issue is and why the Indic community will continue to have problems with these maps. Hopefully those who don't live in a country where maps could actually result in physical threat can understand why this matters.

Thanks for your advice and experience @Planemad.

stjn added a comment.EditedMay 4 2018, 1:42 PM

Solutions for the problem you’ve talked about are: 1) hiding Wikimedia Maps in front of authorities, 2) not using a controversial map / a full map of country in your advertising, 3) not releasing personal information about yourself if you could be held liable for using a tool over which you have no direct control (Wikimedia Maps).

The solution you are proposing is, sorry, enabling sysops of some communities to push their POV on such conflicts. I know some administrators on Russian-speaking projects (≠ Russian Wikipedia) that would’ve loved to do what you are doing, just for Russia, not India. Ukrainian community, for example, is very patriotic, so they would go nuts for a possibility of redoing borders in their own favour (although I do not support the annexation of Crimea, having that status denied in Wikipedia would be strange). I don’t know about the situation in Hebrew/Yiddish communities, but there is a chance that some of their users would love such opportunity also. Same goes for any other disputed border. We’re better not to give away any possibilities of such things rather than to give them and then have to police what that would inevitably entail.

The situation where a government official might be upset is not unique to maps. It's true of any article that deals with disputed areas. It would not be legal to bring either Wikipedia or Wikimedia Maps into India, because they don't uniformly present the government's point of view as the truth.

Ghybu added a subscriber: Ghybu.May 9 2018, 8:18 PM
Pnorman closed this task as Declined.Jul 4 2018, 12:08 AM

It has been decided that because we don't have a dedicated user-focused product manager for maps changes we can't make material product changes for maps, so the new styles work will not be deployed. This includes the borders work. If the borders work were separated from the new style work, it would still be subject to the same restrictions, so we wouldn't deploy it by itself either.

Restricted Application removed a subscriber: Liuxinyu970226. · View Herald TranscriptJul 4 2018, 12:08 AM
stjn awarded a token.Jul 4 2018, 10:16 AM
stjn reopened this task as Open.Oct 2 2018, 7:08 PM

Reopening per https://lists.wikimedia.org/pipermail/wikitech-l/2018-October/090901.html

This is a valid ticket that needs solving even if the team doesn’t have necessary resources for it.

stjn rescinded a token.Oct 2 2018, 7:09 PM
Mholloway changed the task status from Open to Stalled.Oct 3 2018, 4:05 PM
Mholloway moved this task from Needs triage to Epics on the Reading-Infrastructure-Team-Backlog board.