Page MenuHomePhabricator

Support maps serving for affiliate sites via an allow list
Open, Needs TriagePublic

Description

Per the parent task, we will be limiting the maps API to only wikis and tools hosted on Wikimedia Foundation domains. However some affiliates or user groups may host volunteer or outreach tools on other domains.

If you are using the maps.wikimedia.org API for a Wikimedia affiliate or established wikiproject, please comment below with your domain and a link to an example of your usage of maps.

Event Timeline

JMinor renamed this task from Support maps serving for affiliate sites via an allow list. to Support maps serving for affiliate sites via an allow list.Aug 31 2020, 9:03 PM
JMinor created this task.
LGoto added a project: Epic.Sep 2 2020, 3:45 PM
LGoto moved this task from Needs triage to Epics on the Product-Infrastructure-Team-Backlog board.
Elitre updated the task description. (Show Details)Sep 3 2020, 8:47 AM
Elitre updated the task description. (Show Details)Sep 3 2020, 8:51 AM
seav added a subscriber: seav.Sep 3 2020, 10:03 AM

Hello, Wikimedia Czech Republic uses maps.wikimedia.org in our new website, which you can preview at https://test.wikimedia.cz. The map is at the very bottom, and it shows places where the events take place. Can we whitelist *.wikimedia.cz, please?

Thanks,

Martin Urbanec, WMCZ sysadmin

bd808 added a subscriber: bd808.Oct 6 2020, 4:43 PM

As a Toolforge and Cloud-VPS administrator I would like to request that *.tooforge.org, *.wmcloud.org, and *.wmflabs.org are included in the referrer allow list. These are Wikimedia Foundation domains, but they may have slipped other folks minds as such.

CDanis added a comment.Oct 6 2020, 4:45 PM

As a Toolforge and Cloud-VPS administrator I would like to request that *.tooforge.org, *.wmcloud.org, and *.wmflabs.org are included in the referrer allow list. These are Wikimedia Foundation domains, but they may have slipped other folks minds as such.

These domains are already allowed and even tested for :)

@CDanis based on the webserver logs we should know what domains give the most hits. Can you share a list of these domains which are not on the whitelist already? That would give a good indication of what might be missing before breaking it.

CDanis added a comment.Oct 6 2020, 7:11 PM

@CDanis based on the webserver logs we should know what domains give the most hits. Can you share a list of these domains which are not on the whitelist already? That would give a good indication of what might be missing before breaking it.

Yes, I have indeed thought to look at the webserver logs before, although they were not nearly as informative as one might be inclined to believe.

I took a quick look again just now, but there is still no real signal there.

In the past day, the top 20 Referer domains who were using Maps and encountering HTTP 429s were almost entirely commercial sites with no connection to the Movement. The two exceptions were personal blog sites of individuals who at a quick glance did not have an obvious connection to the Movement.

Change 632539 had a related patch set uploaded (by CDanis; owner: CDanis):
[operations/puppet@production] VCL: Maps Referer block: no-op!: comments & redo regex w/ comments

https://gerrit.wikimedia.org/r/632539

Change 632539 merged by CDanis:
[operations/puppet@production] VCL: Maps Referer block: no-op!: comments & redo regex w/ comments

https://gerrit.wikimedia.org/r/632539

Change 632544 had a related patch set uploaded (by CDanis; owner: CDanis):
[operations/puppet@production] VCL: Maps Referer block: allow wikimedia.cz & subdomains thereof

https://gerrit.wikimedia.org/r/632544

Change 632544 merged by CDanis:
[operations/puppet@production] VCL: Maps Referer block: allow wikimedia.cz & subdomains thereof

https://gerrit.wikimedia.org/r/632544

@CDanis and @Dzahn as per T261424#6538173, is there anything else to be done for the 3rd party block in the traffic layer?

@CDanis and @Dzahn as per T261424#6538173, is there anything else to be done for the 3rd party block in the traffic layer?

Sorry for missing this @MSantos but as of earlier today, 3rd party sites are fully blocked.

I'm going to leave this task open for a while and then close it if we haven't heard from any more affiliates etc.

Hello!
Have a problem with maps.
403 - Forbbiden
Domain - ukc.gov.ua
Example for use:
On the main page map of Ukraine with count appeals to the Goverment contact center.

Hi @Unlight123, what does ukc.gov.ua have to do with Wikimedia's maps servers?

Dzahn added a comment.Oct 24 2020, 9:29 PM

@Unlight123 I can confirm there is a broken map on the start page of ukc.gov.ua but that map appears to be using https://leafletjs.com/ which uses OSM tile servers (tile.openstreetmap.org) in the code examples.

So unless that is trying to use Wikimedia's tile servers the bug should be somewhere else and maybe better reported at https://github.com/Leaflet/Leaflet/blob/master/CONTRIBUTING.md#reporting-bugs?

Where exactly are you seeing the 403?

Hi @Unlight123, what does ukc.gov.ua have to do with Wikimedia's maps servers?

They apparently use wikimedia map servers.

@Unlight123 I can confirm there is a broken map on the start page of ukc.gov.ua but that map appears to be using https://leafletjs.com/ which uses OSM tile servers (tile.openstreetmap.org) in the code examples.

So unless that is trying to use Wikimedia's tile servers the bug should be somewhere else and maybe better reported at https://github.com/Leaflet/Leaflet/blob/master/CONTRIBUTING.md#reporting-bugs?

This isn't a bug in leaflet. See below.

Where exactly are you seeing the 403?

When I open the network tab in dev tools, I see that they attempt to load maps.wikimedia.org. The "bug" is definitely the fact Ukrainan government uses Wikimedia map services. See screenshot:

This is also visible in their sourcecode:

(or https://ukc.gov.ua/wp-content/themes/ukc/html/dist/static/js/all.js as an convenience link)

On the other hand, I agree that maps.wikimedia.org should only serve to purposes clearly affilated to the movement, which doesn't include a government website. However, I'm not a SRE, and I'm not a decision maker here.

@Aklapper
Thank you reply.
ukc.gov.ua use wiki maps server for display statistics of appeals from all teritories of Ukraine.
As you can see we got error and reply from wiki maps servers like that

@Unlight123: The Wikimedia Foundation recently made the decision to restrict the usage of maps.wikimedia.org to Wikimedia sites and closely-affiliated websites. The ukc.gov.ua website does not appear to be closely affiliated to the Wikimedia projects, and is accordingly no longer able to use the maps.wikimedia.org service. While this is unfortunate, access to maps.wikimedia.org from ukc.gov.ua is unlikely to be restored. The operator of that website should choose a different tile server.

@AntiCompositeNumber
Hello.
Thank you for your reply and explanation.

This comment was removed by Aklapper.
aapeli added a subscriber: aapeli.Nov 3 2020, 8:35 PM

The Wikimedia Maps TOS does not yet mention this change (it's linked in the attribution line on maps.wikimedia.org). It would be helpful for that to be updated, as it currently implies it's OK to use outside wikimedia projects.

@aapeli: Could you please file a separate task about updating https://foundation.wikimedia.org/wiki/Maps_Terms_of_Use ? Thanks!

aapeli added a comment.Nov 3 2020, 9:49 PM

OK, I've made a new task: T267170.

WMDE-leszek removed a subscriber: WMDE-leszek.

Is it possible to allow wiki.openstreetmap.org? OpenStreetMap seems affiliated, as it is the source for your maps and there had been multiple collaboration projects between OSM and Wikimedia before.

Wiki uses it to include static map images to document specific areas where the wiki pages apply to using this:
https://wiki.openstreetmap.org/wiki/Simple_image_MediaWiki_Extension

I mentioned it before, seems to be lost or no response.
https://wiki.openstreetmap.org/wiki/User_talk:Harry_Wood#Simple_image_MediaWiki_Extension_broken.3F
https://github.com/Firefishy/SimpleMap/issues/4

Template in the wiki tries to load URLs like:
https://maps.wikimedia.org/img/osm-intl,5,13.0,101.5,200x400.png
e.g. on https://wiki.openstreetmap.org/wiki/WikiProject_Thailand

Is it possible to allow wiki.openstreetmap.org? OpenStreetMap seems affiliated

Yes. Wikimedia Italia is also OpenStreetMap Italia and we rely on wiki.openstreetmap.org for certain things like https://wiki.openstreetmap.org/wiki/IT:Pagina_Principale

bd808 added a comment.Nov 30 2020, 4:29 PM

Is it possible to allow wiki.openstreetmap.org? OpenStreetMap seems affiliated

Yes. Wikimedia Italia is also OpenStreetMap Italia and we rely on wiki.openstreetmap.org for certain things like https://wiki.openstreetmap.org/wiki/IT:Pagina_Principale

Mostly a curiosity question, but why would OSM's own wiki not use OSM's own tile servers? https://operations.osmfoundation.org/policies/tiles/

The OSM tile servers are designed to support osm.org only, and do not support all features. The SimpleMap extension needs a tileserver that can stitch tiles together into one image. Kartographer uses this for thumbnails, so it's supported by Kartotherian/Wikimedia Maps. It is not required for the slippy map on osm.org though, so it isn't supported by the OSM tile servers.


The "bright-line" rule from T261424#6419744 is "affiliates and other community projects that are listed on metawiki". In Wikimedia-speak, "affiliate" is a specific term defined by a Board resolution. There are three currently-recognized models of affiliation: chapters, thematic organizations, and user groups, none of which include the OSMF or the OpenStreetMap project. OSMF might be considered a "movement partner", but that model has never been defined and is "not yet active". The list of affiliates is here. Because OSM/OSMF isn't listed there, and none of the active affiliation models apply, I don't think extending maps.wikimedia.org support would be consistent with the stated policy at this time. (I say that as an OSM contributor and as someone who isn't responsible for making the decision.)

Base added a subscriber: Base.EditedMon, Feb 22, 11:52 PM

Perhaps a known issue and not strictly related to this ticket, but it is referenced in the error, is that when I follow a link to maps.wikimedia.org in Facebook mobile app I stumble upon a 403 error page. It is not obvious that as simple action as opening it in the browser proper would remove the error. Rather it gives an impression that service is down until you read the small script explaining which error it is exactly.