https://lists.wikimedia.org/pipermail/maps-l/2020-August/001729.html

Note on https://wiki.openstreetmap.org/wiki/Tile_servers#Base_maps is updated with the deprecation notice. I'll remove it from the list entirely when external access is fully disabled.

In T261424#6416783, @AntiCompositeNumber wrote:

Note on https://wiki.openstreetmap.org/wiki/Tile_servers#Base_maps is updated with the deprecation notice. I'll remove it from the list entirely when external access is fully disabled.

@JMinor and I were wondering how to make that happen - super grateful that you are volunteering! Thanks on behalf of the entire team.

Aren't the referenced patch blocking access to all services on maps.wikimedia.org and not only osm-intl? This issue and the deprecation message sent to maps-l only addresses maps.wikimedia.org/osm-intl/.

There are some uses of maps.wikimedia.org by websites that are definitely movement-affiliated but may not be hosted on Wikimedia servers (T261506: wikimedia.pl returns a HTTP 429 error (let it access varnish maps_domains), T260520: maps.wikilovesmonuments.org returns a HTTP 429 error (let it access varnish maps_domains)). The deprecation notice implies that these uses will no longer be permitted. @Elitre, can you confirm that this is the case?

Isn't this already the case? I mean we just recently had to add wikilovesmonuments to the regex of allowed domains to let it use maps.wikimedia.org (T260520) and wikimedia.pl is asking to be added (T261506).

So if "limit maps serving to Wikimedia hosted sites only" wasn't already the status quo then why would we have to add to this:

https://gerrit.wikimedia.org/r/c/operations/puppet/+/621342/3/modules/varnish/templates/upload-frontend.inc.vcl.erb

It seems like this statement:

only support requests from the Wikimedia cluster

is in conflict with this statement:

Cloud Services hosted tools, and any internal development or testing tools should not be affected.

Per comments on Gerrit, this would only work if wmcloud.org etc would also be added.

@AntiCompositeNumber our intention was to create a clear, maintainable, line around what domains would be supported. We'd really like to avoid setting up policies and processes to build exception lists, so the limit to "hosted domains" came from that. However, we don't want to impede or remove support for something vital, and obviously an exception was already made for the existing "cache only" policy for wikilovesmonuments.org.

For the Polish affiliate, we could consider including affiliates domains, though I don't know, apart from the bug you link to, what scale of support or how many domains that would involve.

If people know Wikimedia affiliated groups potentially negatively impacted by this, please consider connecting us, either here or via email to @Elitre or me.

My two cents:

I suspect it's merely the case that the language in the announcement was somewhat imprecise. That's understandable; this isn't a simple situation.

It seems very reasonable to me that we'd support all WMCS usages, as well as usages by Wikimedia affiliates, as well as anything of direct value to the overall movement.

What we don't want to support are (for instance) Pokémon GO fan sites, which were once a double-digit percentage of traffic.

I am happy to +2 and merge any patches to implement the above into reality.

" from the Wikimedia cluster"

is a naive Product person's use of that term. I apologize if that imprecision is causing misunderstanding.

Cloud Services hosted tools, and any internal development or testing tools should not be affected.

Is the ultimate intention.

So I'd say changes to the patch to support that are welcome.

@CDanis captures the situation well!

[Removed my earlier comment because I had failed to refresh this ticket and catch up on all the other recent traffic, which changes everything]

If we're going to preserve access for community sites that request it, we're going to need some reasonable mechanism around that, or it will devolve into a case of infrastructure staff making educated guesses as to the legitimacy of requests from unrelated 3rd-party sites and domains that claim a community connection...

"So if "limit maps serving to Wikimedia hosted sites only" wasn't already the status quo then why would we have to add to this

My understanding is that, currently, cached tiles are returned to all requests, uncached tiles only to "hosted sites" (+the exception(s) being discussed). This change is to decline all requests, not just uncached requests, for "non Wikimedia hosted" domains. Additionally the status quo is a "temporary" response to an incident. This change is a deliberate and "permanent" (as any wiki thing can be) change in who we support with our maps.

@BBlack Totally agree. We tried to draw a bright line partly because of that. Not only should individual infrastructure staff not be in that position, I don't think we want anyone in that position!

For now I'd like to give the announcement time for people to be aware of it, and @Elitre and I will make an effort to reach out to anyone who might fall into these cracks and understand if there are alternatives or if we need to "grandfather" some folks in before this change goes into effect.

@JMinor @Elitre Although some of this is already mentioned upwards in this task, here's a summary of the community objections I'm aware of so far:

• T260520: maps.wikilovesmonuments.org returns a HTTP 429 error (let it access varnish maps_domains) for Commons's photo competition
• T261506: wikimedia.pl returns a HTTP 429 error (let it access varnish maps_domains) from Wikimedia Polska
• https://twitter.com/naveenpf/status/1299219712488779781 from a volunteer contributor: Apparently our tileserver is (one of the few? the only?) public server with many kinds of localized tiles.

One bright line that we could draw is "affiliates and other community projects that are listed on metawiki". I think that would be both reasonable and workable. (Although I think any additions to the list of allowed sites should be reactive to requests, not proactive based on everything already on meta.)

I noticed the announcement on the maps-l list and I also noticed https://twitter.com/krmaher/status/1299203640188690434 where @Katherine-WMF replied. Someone recently mention to me "the only way to prevent WMF people from doing making stupid mistakes these days is tagging Katherine on Twitter".

This is one of those mistakes. The maps service was always something the volunteers wanted more than the WMF. Finally after many years we have a production service, but the quality of managing the service is probably best described as "leaving it rot somewhere in the corner". Of course it gets more popular and any service with such poor maintenance will fall over at some point. That's what happened. Instead of severely limiting this service you should start maintaining it properly. The current state of the maps service is a disgrace.

FTR, in T261506 I added wikimedia.pl to our list of allowed domains.

• They're an affiliate, listed on metawiki for some time, which I think is the closest thing we have to a 'bright line' right now.
• It was a time-sensitive request
• It was similar in nature to the already-allowed wikilovesmonuments, which seemed uncontroversial.

Although I haven't had a chance to discuss with @Elitre @JMinor & @Gehel I think supporting affiliates makes some sense.

@CDanis

Yes, I will make a subtask for tracking these and any other affiliated domains that need an exemption.

We're also send an update to the mailing list letting people know they can reach out to me or @Elitre or comment on the subtask to request an exception for their affiliate site or recognized user group.

To this specifically:

Someone recently mention to me "the only way to prevent WMF people from doing making stupid mistakes these days is tagging Katherine on Twitter".

The person who tweeted at Katherine did not reply to the email announcement or comment on this ticket. So no, it wasn't the "only way" to get our attention or point to potential issues. In fact, because we generally don't consider Twitter a movement venue (like mailing lists or phab) it is actually a really inefficient way to get our attention.

Change 632539 had a related patch set uploaded (by CDanis; owner: CDanis):
[operations/puppet@production] VCL: Maps Referer block: no-op!: comments & redo regex w/ comments

https://gerrit.wikimedia.org/r/632539

Change 632539 merged by CDanis:
[operations/puppet@production] VCL: Maps Referer block: no-op!: comments & redo regex w/ comments

https://gerrit.wikimedia.org/r/632539

@JMinor @JKatzWMF please give your Go to get this done. Thanks!

Yes, I think everyone who requested to be added to the allow list has been added. There were a couple questions on the mailing list related to OSM chapters usage. I have reached out to those folks a couple times, but received no answer. So, I think we are clear to merge the change, as is.

@sdkim

In T261424#6539494, @JMinor wrote:

Yes, I think everyone who requested to be added to the allow list has been added. There were a couple questions on the mailing list related to OSM chapters usage. I have reached out to those folks a couple times, but received no answer. So, I think we are clear to merge the change, as is.

Thanks @JMinor. I should have time to write and deploy the patch later this week.

Change 634024 had a related patch set uploaded (by CDanis; owner: CDanis):
[operations/puppet@production] Map tiles for 3rd parties: block (403) all requests, not just misses

https://gerrit.wikimedia.org/r/634024

Change 634024 merged by CDanis:
[operations/puppet@production] Map tiles for 3rd parties: block (403) all requests, not just misses

https://gerrit.wikimedia.org/r/634024

I've removed Wikimedia Maps from https://wiki.openstreetmap.org/wiki/Tile_servers.

Thanks to everyone involved.

Is it an intended effect of this ticket or is it a bug that now when I access https://maps.wikimedia.org/ from browser by entering the URL into browser URL bar directly, I get an error saying

Request from 221.127.107.156 via cp5026 cp5026, Varnish XID 729746413
Upstream caches: cp5026 int
Error: 403, Forbidden: Map tiles are restricted to Wikimedia & affiliated sites only. See https://wikitech.wikimedia.org/wiki/Maps/External_usage if you believe your usage supports the Movement. at Sat, 04 Mar 2023 12:24:42 GMT

?

@C933103: This ticket has been closed for 29 months. Please file separate tickets for separate topics. Thanks. (Plus I cannot reproduce.)