| JMinor | |
| Aug 31 2020, 9:03 PM |
| F32412754: изображение.png | |
| Oct 24 2020, 10:53 PM |
| F32412751: изображение.png | |
| Oct 24 2020, 10:51 PM |
| F32412710: image.png | |
| Oct 24 2020, 9:37 PM |
| F32412716: image.png | |
| Oct 24 2020, 9:37 PM |
Per the parent task, we will be limiting the maps API to only wikis and tools hosted on Wikimedia Foundation domains. However some affiliates or user groups may host volunteer or outreach tools on other domains.
If you are using the maps.wikimedia.org API for a Wikimedia affiliate or established wikiproject, please follow the instructions at https://wikitech.wikimedia.org/wiki/Maps/External_usage to file a new Phabricator task.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | CDanis | T261424 Limit maps serving to Wikimedia hosted sites only | |||
| Resolved | JMinor | T261694 Support maps serving for affiliate sites via an allow list | |||
| Resolved | Dzahn | T261506 wikimedia.pl returns a HTTP 429 error (let it access varnish maps_domains) | |||
| Resolved | BUG REPORT | Dzahn | T260520 maps.wikilovesmonuments.org returns a HTTP 429 error (let it access varnish maps_domains) |
Hello, Wikimedia Czech Republic uses maps.wikimedia.org in our new website, which you can preview at https://test.wikimedia.cz. The map is at the very bottom, and it shows places where the events take place. Can we whitelist *.wikimedia.cz, please?
Thanks,
Martin Urbanec, WMCZ sysadmin
@CDanis based on the webserver logs we should know what domains give the most hits. Can you share a list of these domains which are not on the whitelist already? That would give a good indication of what might be missing before breaking it.
Yes, I have indeed thought to look at the webserver logs before, although they were not nearly as informative as one might be inclined to believe.
I took a quick look again just now, but there is still no real signal there.
In the past day, the top 20 Referer domains who were using Maps and encountering HTTP 429s were almost entirely commercial sites with no connection to the Movement. The two exceptions were personal blog sites of individuals who at a quick glance did not have an obvious connection to the Movement.
Change 632539 had a related patch set uploaded (by CDanis; owner: CDanis):
[operations/puppet@production] VCL: Maps Referer block: no-op!: comments & redo regex w/ comments
Change 632539 merged by CDanis:
[operations/puppet@production] VCL: Maps Referer block: no-op!: comments & redo regex w/ comments
Change 632544 had a related patch set uploaded (by CDanis; owner: CDanis):
[operations/puppet@production] VCL: Maps Referer block: allow wikimedia.cz & subdomains thereof
Change 632544 merged by CDanis:
[operations/puppet@production] VCL: Maps Referer block: allow wikimedia.cz & subdomains thereof
@CDanis and @Dzahn as per T261424#6538173, is there anything else to be done for the 3rd party block in the traffic layer?
Sorry for missing this @MSantos but as of earlier today, 3rd party sites are fully blocked.
I'm going to leave this task open for a while and then close it if we haven't heard from any more affiliates etc.
Hello!
Have a problem with maps.
403 - Forbbiden
Domain - ukc.gov.ua
Example for use:
On the main page map of Ukraine with count appeals to the Goverment contact center.
@Unlight123 I can confirm there is a broken map on the start page of ukc.gov.ua but that map appears to be using https://leafletjs.com/ which uses OSM tile servers (tile.openstreetmap.org) in the code examples.
So unless that is trying to use Wikimedia's tile servers the bug should be somewhere else and maybe better reported at https://github.com/Leaflet/Leaflet/blob/master/CONTRIBUTING.md#reporting-bugs?
Where exactly are you seeing the 403?
They apparently use wikimedia map servers.
This isn't a bug in leaflet. See below.
Where exactly are you seeing the 403?
When I open the network tab in dev tools, I see that they attempt to load maps.wikimedia.org. The "bug" is definitely the fact Ukrainan government uses Wikimedia map services. See screenshot:
This is also visible in their sourcecode:
(or https://ukc.gov.ua/wp-content/themes/ukc/html/dist/static/js/all.js as an convenience link)
On the other hand, I agree that maps.wikimedia.org should only serve to purposes clearly affilated to the movement, which doesn't include a government website. However, I'm not a SRE, and I'm not a decision maker here.
@Aklapper
Thank you reply.
ukc.gov.ua use wiki maps server for display statistics of appeals from all teritories of Ukraine.
As you can see we got error and reply from wiki maps servers like that
@Unlight123: The Wikimedia Foundation recently made the decision to restrict the usage of maps.wikimedia.org to Wikimedia sites and closely-affiliated websites. The ukc.gov.ua website does not appear to be closely affiliated to the Wikimedia projects, and is accordingly no longer able to use the maps.wikimedia.org service. While this is unfortunate, access to maps.wikimedia.org from ukc.gov.ua is unlikely to be restored. The operator of that website should choose a different tile server.
The Wikimedia Maps TOS does not yet mention this change (it's linked in the attribution line on maps.wikimedia.org). It would be helpful for that to be updated, as it currently implies it's OK to use outside wikimedia projects.
@aapeli: Could you please file a separate task about updating https://foundation.wikimedia.org/wiki/Maps_Terms_of_Use ? Thanks!
Is it possible to allow wiki.openstreetmap.org? OpenStreetMap seems affiliated, as it is the source for your maps and there had been multiple collaboration projects between OSM and Wikimedia before.
Wiki uses it to include static map images to document specific areas where the wiki pages apply to using this:
https://wiki.openstreetmap.org/wiki/Simple_image_MediaWiki_Extension
I mentioned it before, seems to be lost or no response.
https://wiki.openstreetmap.org/wiki/User_talk:Harry_Wood#Simple_image_MediaWiki_Extension_broken.3F
https://github.com/Firefishy/SimpleMap/issues/4
Template in the wiki tries to load URLs like:
https://maps.wikimedia.org/img/osm-intl,5,13.0,101.5,200x400.png
e.g. on https://wiki.openstreetmap.org/wiki/WikiProject_Thailand
Yes. Wikimedia Italia is also OpenStreetMap Italia and we rely on wiki.openstreetmap.org for certain things like https://wiki.openstreetmap.org/wiki/IT:Pagina_Principale
Mostly a curiosity question, but why would OSM's own wiki not use OSM's own tile servers? https://operations.osmfoundation.org/policies/tiles/
The OSM tile servers are designed to support osm.org only, and do not support all features. The SimpleMap extension needs a tileserver that can stitch tiles together into one image. Kartographer uses this for thumbnails, so it's supported by Kartotherian/Wikimedia Maps. It is not required for the slippy map on osm.org though, so it isn't supported by the OSM tile servers.
The "bright-line" rule from T261424#6419744 is "affiliates and other community projects that are listed on metawiki". In Wikimedia-speak, "affiliate" is a specific term defined by a Board resolution. There are three currently-recognized models of affiliation: chapters, thematic organizations, and user groups, none of which include the OSMF or the OpenStreetMap project. OSMF might be considered a "movement partner", but that model has never been defined and is "not yet active". The list of affiliates is here. Because OSM/OSMF isn't listed there, and none of the active affiliation models apply, I don't think extending maps.wikimedia.org support would be consistent with the stated policy at this time. (I say that as an OSM contributor and as someone who isn't responsible for making the decision.)
Perhaps a known issue and not strictly related to this ticket, but it is referenced in the error, is that when I follow a link to maps.wikimedia.org in Facebook mobile app I stumble upon a 403 error page. It is not obvious that as simple action as opening it in the browser proper would remove the error. Rather it gives an impression that service is down until you read the small script explaining which error it is exactly.
On April 5, 2020, we will update the Wikimedia Maps Terms of Use based on T261424. For more information, see: T267170#6935518 .
This change will not affect the policy for supporting Wikimedia Maps on the Wikimedia projects, tools hosted on Wikimedia Cloud Services, or other allowed Wikimedia-affiliated domains.
New user trying to setup a small website to track development. Installed Kartographer but think I need a URL for map tiles. Can anyone help me?
@McLeod919 I subscribed you to 2 tasks (T259868 and T271617) that has information about how to setup Kartographer configuration and its current gotchas after the 3rd party restriction.
I suggest you to write an e-mail to the Maps mailing list with more details about what you're trying to do, I think we can discuss it better there, what do you think?
Error: 403, Forbidden: Map tiles are restricted to Wikimedia & affiliated sites only. Please post on https://phabricator.wikimedia.org/T261694 if you believe your usage supports the Movement. at Wed, 23 Jun 2021 11:54:01 GMT
OK I have to report this domain owned by Wikimedia Italia:
https://barriere.wikimedia.it/
From that domain the application fails loading these resources:
https://maps.wikimedia.org/osm-intl/...
More about the project:
I think that barriere.wikimedia.it is the only domain needing to be whitelisted, from our side.
Edited: I think that these domains could be whitelisted:
Thank you so much!
Are WMF map tiles in use elsewhere on wikimedia.it, or just on barriere.wikimedia.it? https://osmit.it appears to just be a 301 redirect to https://wiki.openstreetmap.org/wiki/IT:Pagina_Principale, is it actually being used for something else?
Thank you. The reason is, it's in the short term WMIT's plans (OSM coordinator's plan, members' plans, ...) to integrate another cute slippy map in *.osmit.it and maybe some little more. Same reason for *.wikimedia.it. That's why I've suggested both, in the hope of saving your time, my time, subscribers ecc. in the next future.
I need a little help to understand this soft review. I think this whitelist is to prevent wasting resources by untrusted users, but the intentions of a recognized Wikimedia chapter should be somehow trusted in good faith, regardless and blindly if the traffic is low. Isn't it? If the real question is whether these websites have (or will have) high traffic, the answer is nope. No danger of spikes. If the opposite happens, feel free to use my head as a piñata.
Kind up to approve at least https://barriere.wikimedia.it/ :) It is a project from WMIT to help disabled people. Thank you again.
I'm just trying to make sure that the requested configuration change would do what you expect it to do. I see no problem with allowing all of wikimedia.it, as we have done for WMCZ. As long as osmit.it is just a redirect page, adding it to the allow list won't accomplish much of anything. As previously stated, I don't see the OSM Wiki as a good candidate for the allow list. Ultimately the decision comes down to SRE and the Wikimedia Maps maintainers, not me.
I'll prepare a patch to add wikimedia.it (including subdomains). If osmit.it is going to be expanded from a simple redirect to a site that includes maps on the osmit.it domain, it can also be added, but that's not clear to me at this point.
Change 703929 had a related patch set uploaded (by AntiCompositeNumber; author: AntiCompositeNumber):
[operations/puppet@production] VCL: Maps Referer block: allow wikimedia.it
Hi, I'm using the Wikimedia tiles for the Wiki Loves Monuments Italy app by Wikimedia Italy. The HTTP user-agent is it.wikimedia.wikilovesmonuments.
I think that it's suitable for an exception. 😄
It doesn't really fit the User-Agent policy format for a User-Agent though - https://meta.wikimedia.org/wiki/User-Agent_policy
@Reedy It seems descriptive enough anyway (and includes contact information, even if indirectly)
It's easier for you to actually properly follow the policy, than someone have to reverse the URL to end up on https://www.wikimedia.it/wiki-loves-monuments/ or looking for WMIT user details.
Also, you're not even reading the description.
If you are using the maps.wikimedia.org API for a Wikimedia affiliate or established wikiproject, please comment below with your domain and a link to an example of your usage of maps.
It doesn't say comment here with a User-Agent, which can be easily spoofed.
It's a mobile application, there isn't a domain. 😄 I forgot to provide a link to the example usage, which is the application itself (Play store - Github). @Reedy
The user-agent is unfortunately automatically set from the OSM library I'm using. Anyway I'll try to check if there's a way to customize it and follow the policy. If I had the possibility, I would have obviously followed the example in the policy straightforwardly. The map policy talks about an HTTP User-Agent or HTTP referer, even if they can be easily spoofed...
Ok, the library has been updated and I've set a more compliant user-agent. Now the app user-agent is: "Wiki Loves Monuments Italia app/2.1.2 (https://github.com/ferdi2005/monumenti; ferdinando.traversa@wikimedia.it) ti.osm/1.0.1"
I think that I'm compliant with the map policy (it's a Wikimedia projects tool, indeed it helps in uploading photo to Commons, it has a descriptive user-agent and I've also posted a notice about its usage).
Change 703929 merged by Legoktm:
[operations/puppet@production] varnish: Allow wikimedia.it to use maps tiles
I merged @AntiCompositeNumber's patch and tiles now work on https://barriere.wikimedia.it/ - sorry about the delay.
As a meta comment, having a long-running task like this does not work with the SRE clinic duty workflows that rely on new tasks being filed rather than long-running tasks being bumped. I propose we have a wiki page that documents what needs to be done (might already exist?), point the error message at that page, and have new requests be filed as individual tickets in SRE.
+1 to Legoktm's last comment. "Add a comment to this task" makes this a neverending open ticket, though tickets should be discrete and actionable. @JMinor?
Change 709511 had a related patch set uploaded (by Legoktm; author: Legoktm):
[operations/puppet@production] varnish: Improve comments around maps access, retire T261694
I created https://wikitech.wikimedia.org/wiki/Maps/External_usage just now (please edit/improve!) and submitted the above patch to update the Varnish error message. I also mentioned this in #wikimedia-sre last week (when I made the above comment), is there anyone else who should be consulted or OK to go ahead?
@Legoktm from Product-Infrastructure-Team-Backlog which are the official maintainers of maps, this looks great. Please go ahead and let us know if there is anything you need from us.
Change 709511 merged by Legoktm:
[operations/puppet@production] varnish: Improve comments around maps access, retire T261694
Awesome, thanks. I merged the change to update the error message, it'll rollout everywhere in ~30 min.
Just noting that the newly made page was pretty much "orphan" - most of the docs re: Maps live on mw.org, so
I went ahead and updated https://www.mediawiki.org/wiki/Wikimedia_Maps - please remember to keep that up to date in case of future changes! TY!
