Same origin policy access between tools deliberately broken by toolforge.org migration
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Kolossos
	Jun 7 2020, 3:56 PM

Description

See https://wikitech.wikimedia.org/wiki/News/Toolforge.org#Cross-Origin_Resource_Sharing_(CORS)_requests_broken for more information

As descripted in T234617 it is the plan to switch to toolforge.org and give each tool it's own subdomain.

I have in project WIWOSM a OpenStreetMap application (OpenLayers) , this project use data from project wp-world.
(The scripts are running in project osm,which seems in the moment not the problem.)

Original Script with Wikipedia data:
https://tools.wmflabs.org/wiwosm/osm-on-ol/kml-on-ol.php?lang=de&uselang=de&params=50.957777777778_N_13.94_E_region%3ADE-SN_type%3Acity(38320)&pagename=Pirna&language=de&title=Pirna&secure=1&zoom=11&lat=50.9604&lon=13.9484&layers=B00000000FTTTF

Modified script:
https://wiwosm.toolforge.org/osm-on-ol/kml-on-ol-test.php?lang=de&uselang=de&params=50.957777777778_N_13.94_E_region%3ADE-SN_type%3Acity(38320)&pagename=Pirna&language=de&title=Pirna&secure=1&zoom=11&lat=50.9604&lon=13.9484&layers=B00000000FTTTF

Error message from console:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://wp-world.toolforge.org/marks.php?.... (Reason: CORS request did not succeed)

Helppage from OpenLayers:
http://docs.openlayers.org/library/request.html

I my eyes the timeline for the hard end is much too short for volunteers. So I got the information about the move yesterday and have to work the next days.

What can I do?

Related Objects

Mentioned In: T258256: OOUI window management broken on pages with with additional frames, due to cross document access
T258254: Merge not working in Wikidata with Firefox
T258090: Kartographer map does not open on it.wikivoyage.org: "Blocked a frame from accessing a cross-origin frame"
Mentioned Here: T125589: Allow each tool to have its own subdomain for browser sandbox/cookie isolation
T189879: Set up a Nominatim instance to avoid geocode lookups with Google and/or OSM from tools
T234617: Toolforge. introduce new domain toolforge.org

Event Timeline

Kolossos created this task.Jun 7 2020, 3:56 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 7 2020, 3:56 PM

Also: Will the Nominatim Proxy (T189879) be any longer usable for the same reason?

Kolossos, try to add this to the .lighttpd.conf of the wp-world service:

setenv.add-response-header += ( "Access-Control-Allow-Origin" => "https://wiwosm.toolforge.org")

(https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web/Lighttpd#Header,_mimetype,_character_encoding,_error_handler)

ugh. this is a mess. I now made nominatim proxy accept both toolforge and wmflabs.org origins, but....

if i launch with --canonical, the old address starts redirecting and this breaks CORS.

GET /nominatim/search?format=json&q=sfas HTTP/2
> Host: tools.wmflabs.org
> Origin: https://wikimap.toolforge.org
> Accept: */*
> Referer: https://wikimap.toolforge.org/
> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1 Safari/605.1.15
> Cache-Control: no-cache
> Pragma: no-cache
> 
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 307 
< server: nginx/1.14.2
< date: Sun, 07 Jun 2020 19:44:18 GMT
< content-type: text/html
< content-length: 177
< location: https://nominatim.toolforge.org/search?format=json&q=sfas
< strict-transport-security: max-age=2592000
< x-clacks-overhead: GNU Terry Pratchett
< content-security-policy-report-only: default-src 'self' 'unsafe-eval' 'unsafe-inline' blob: data: filesystem: mediastream: *.toolforge.org wikibooks.org *.wikibooks.org wikidata.org *.wikidata.org wikimedia.org *.wikimedia.org wikinews.org *.wikinews.org wikipedia.org *.wikipedia.org wikiquote.org *.wikiquote.org wikisource.org *.wikisource.org wikiversity.org *.wikiversity.org wikivoyage.org *.wikivoyage.org wiktionary.org *.wiktionary.org *.wmcloud.org *.wmflabs.org wikimediafoundation.org mediawiki.org *.mediawiki.org wss://tools.wmflabs.org; report-uri https://csp-report.toolforge.org/collect;
< 
<html>
<head><title>307 Temporary Redirect</title></head>
<body>
<center><h1>307 Temporary Redirect</h1></center>
<hr><center>openresty/1.15.8.1</center>
</body>
</html>

Browser:
[Error] Origin https://locator.toolforge.org is not allowed by Access-Control-Allow-Origin.
[Error] Fetch API cannot load https://nominatim.toolforge.org/search?format=json&q=stads due to access control checks.

Cross-origin redirection to https://nominatim.toolforge.org/search?format=json&q=stads denied by Cross-Origin Resource Sharing policy: Origin https://wikimap.toolforge.org is not allowed by Access-Control-Allow-Origin.

I switched back to running without --canonical for now, as i can't fix all the tools that use nominatim proxy right now

I try some changes in .lighttpd.conf but without luck. It seems OpenLayers change the request from GET to OPTIONS methode. Perhaps this is a reason .

Just to be sure: Did you restart the service after your changes to become effective? I had luck with only the "Access-Control-Allow-Origin" line, no further lines for methods or so required.

But looks good, no JS errors in console & KML fetched:

https://www.test-cors.org/#?client_method=GET&client_credentials=false&server_url=https%3A%2F%2Fwp-world.toolforge.org%2Fmarks.php&server_enable=true&server_status=200&server_credentials=false&server_tabs=remote

Thank you @TheDJ! I tested all four combinations, I can call now your old & new URL from my old & new URL, looks good!

@Kolossos did you try adding access-control-allow-methods: "OPTIONS,GET" at the same time ?

bd808 renamed this task from toolforge.org destroy same origin policy in JS to Same origin policy access between tools deliberately broken by toolforge.org migration.Jun 8 2020, 5:34 PM

I renamed this task because in reality the *.toolforge.org migration is deliberately breaking this "feature" of the old path based routing. This is much more clear if you read the T125589: Allow each tool to have its own subdomain for browser sandbox/cookie isolation task which is the parent of T234617: Toolforge. introduce new domain toolforge.org.

I my eyes the timeline for the hard end is much too short for volunteers. So I got the information about the move yesterday and have to work the next days.

The initial announcement to the cloud-announce mailing list was made on 2020-04-13. Follow up announcements were made on 2020-05-04 and 2020-06-02. The 60 day window from the initial announcement to the planned forced migration feels like a reasonable timeline to me, but I am certainly willing to hear rational arguments as to why more time would typically be needed for a tool maintainer to address such issues.

bd808 updated the task description. (Show Details)Jun 8 2020, 5:59 PM

Answers to the questions above:

Yes I restarted the server each time.
I tested "OPTIONS,GET" at the same time,but today also one by one or alone.
Thanks for renaming this task.

My question is, if I see the outgoing request from OpenLayers are changing from GET to OPTIONS if I switch to an external subdomain, then it means I will get no content back. Right?

Simple solution would be now to move the little php-script from wp-world to wiwosm and solve this problem.

The OPTIONS request would be the CORS preflight request. Teh browser would be expecting a response including an Access-Control-Allow-Origin header and possibly an Access-Control-Allow-Methods header.

@Kolossos: Now it works, a "Access-Control-Allow-Headers" was also necessary.
I disabled the CORS header output in your php scripts and added this to the .lighttpd.conf

setenv.add-response-header += ( "Access-Control-Allow-Origin" => "*", "Access-Control-Allow-Methods" => "GET, OPTIONS", "Access-Control-Allow-Headers" => "*" )

Thanks a lot to all for the help.

@bd808 : To your schedule question, you have right from a professional standpoint that 2 months should be enough for a change. From a standpoint of a volunteer that maitains scripts that are many years old it's maybe different. I only notice the problem of this script this week by an other popular script (geohack), which is still broken. So maybe a week more could be in my eyes helpful.

In T254693#6207453, @Kolossos wrote:

So maybe a week more could be in my eyes helpful.

I think you will probably get that week. Many humans are motivated by deadlines, so it is useful to set them to help create a sense of urgency or things would just keep being put off forever (see my own bug backlog for signs of that). We do not want to break tools unnecessarily, and we have found a couple of new small bugs in the process just this week that are not all properly fixed yet.

Access-Control-Allow-Headers" => "*"

I suspect this is because of the always added response header: x-clacks-overhead: GNU Terry Pratchett
I suggest that if toolforge wants to add global headers, it should probably also 'globally' add that header to Access-Control-Allow-Headers. (Production wmf does this for the analytics headers I believe)

I didn't trigger that condition in nominatim, because my GET is a 'simple' json cors request, whereas this marks.php returns xml which causes it to be upgraded to preflight mode, which triggers the need for Access-Control-Allow-Headers/Methods in the options response to list all none-safe cors headers.

In T254693#6209865, @TheDJ wrote:

Access-Control-Allow-Headers" => "*"

I suspect this is because of the always added response header: x-clacks-overhead: GNU Terry Pratchett
I suggest that if toolforge wants to add global headers, it should probably also 'globally' add that header to Access-Control-Allow-Headers. (Production wmf does this for the analytics headers I believe)

I didn't trigger that condition in nominatim, because my GET is a 'simple' json cors request, whereas this marks.php returns xml which causes it to be upgraded to preflight mode, which triggers the need for Access-Control-Allow-Headers/Methods in the options response to list all none-safe cors headers.

My understanding is that "Access-Control-Allow-Headers" is about the headers that a user-agent is allowed to send to the server along with a CORS request. The "x-clacks-overhead" header is outbound from the server, not inbound. I believe that "Access-Control-Expose-Headers" would be the directive to instruct a user-agent to retain a custom header in the content it processed when receiving a CORS response. Am I misunderstanding what "Access-Control-Allow-Headers" controls?

My understanding is that "Access-Control-Allow-Headers" is about the headers that a user-agent is allowed to send to the server along with a CORS request. The "x-clacks-overhead" header is outbound from the server, not inbound. I believe that "Access-Control-Expose-Headers" would be the directive to instruct a user-agent to retain a custom header in the content it processed when receiving a CORS response. Am I misunderstanding what "Access-Control-Allow-Headers" controls?

You are right, it's about allowed headers along with the request. In this case: "X-Requested-With: XMLHttpRequest".

DB111 closed this task as Resolved.Jun 27 2020, 10:39 AM

DB111 claimed this task.

TheDJ mentioned this in T258090: Kartographer map does not open on it.wikivoyage.org: "Blocked a frame from accessing a cross-origin frame".Jul 16 2020, 12:00 PM

The_RedBurn mentioned this in T258254: Merge not working in Wikidata with Firefox.Jul 17 2020, 10:12 AM

The_RedBurn mentioned this in T258256: OOUI window management broken on pages with with additional frames, due to cross document access.Jul 17 2020, 10:22 AM

Same origin policy access between tools deliberately broken by toolforge.org migrationClosed, ResolvedPublicActions

Description

Related Objects

Event Timeline

Same origin policy access between tools deliberately broken by toolforge.org migration
Closed, ResolvedPublic
Actions