Page MenuHomePhabricator

Same origin policy access between tools deliberately broken by toolforge.org migration
Closed, ResolvedPublic

Description

See https://wikitech.wikimedia.org/wiki/News/Toolforge.org#Cross-Origin_Resource_Sharing_(CORS)_requests_broken for more information


As descripted in T234617 it is the plan to switch to toolforge.org and give each tool it's own subdomain.

I have in project WIWOSM a OpenStreetMap application (OpenLayers) , this project use data from project wp-world.
(The scripts are running in project osm,which seems in the moment not the problem.)

Original Script with Wikipedia data:
https://tools.wmflabs.org/wiwosm/osm-on-ol/kml-on-ol.php?lang=de&uselang=de&params=50.957777777778_N_13.94_E_region%3ADE-SN_type%3Acity(38320)&pagename=Pirna&language=de&title=Pirna&secure=1&zoom=11&lat=50.9604&lon=13.9484&layers=B00000000FTTTF

Modified script:
https://wiwosm.toolforge.org/osm-on-ol/kml-on-ol-test.php?lang=de&uselang=de&params=50.957777777778_N_13.94_E_region%3ADE-SN_type%3Acity(38320)&pagename=Pirna&language=de&title=Pirna&secure=1&zoom=11&lat=50.9604&lon=13.9484&layers=B00000000FTTTF

Error message from console:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://wp-world.toolforge.org/marks.php?.... (Reason: CORS request did not succeed)

Helppage from OpenLayers:
http://docs.openlayers.org/library/request.html

I my eyes the timeline for the hard end is much too short for volunteers. So I got the information about the move yesterday and have to work the next days.

What can I do?

Event Timeline

Kolossos created this task.Jun 7 2020, 3:56 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 7 2020, 3:56 PM
DB111 added a subscriber: DB111.Jun 7 2020, 5:05 PM

Also: Will the Nominatim Proxy (T189879) be any longer usable for the same reason?

DB111 added a comment.Jun 7 2020, 6:11 PM

Kolossos, try to add this to the .lighttpd.conf of the wp-world service:

setenv.add-response-header += ( "Access-Control-Allow-Origin" => "https://wiwosm.toolforge.org")

(https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web/Lighttpd#Header,_mimetype,_character_encoding,_error_handler)

TheDJ added a subscriber: TheDJ.EditedJun 7 2020, 7:49 PM

ugh. this is a mess. I now made nominatim proxy accept both toolforge and wmflabs.org origins, but....

if i launch with --canonical, the old address starts redirecting and this breaks CORS.

GET /nominatim/search?format=json&q=sfas HTTP/2
> Host: tools.wmflabs.org
> Origin: https://wikimap.toolforge.org
> Accept: */*
> Referer: https://wikimap.toolforge.org/
> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1 Safari/605.1.15
> Cache-Control: no-cache
> Pragma: no-cache
> 
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 307 
< server: nginx/1.14.2
< date: Sun, 07 Jun 2020 19:44:18 GMT
< content-type: text/html
< content-length: 177
< location: https://nominatim.toolforge.org/search?format=json&q=sfas
< strict-transport-security: max-age=2592000
< x-clacks-overhead: GNU Terry Pratchett
< content-security-policy-report-only: default-src 'self' 'unsafe-eval' 'unsafe-inline' blob: data: filesystem: mediastream: *.toolforge.org wikibooks.org *.wikibooks.org wikidata.org *.wikidata.org wikimedia.org *.wikimedia.org wikinews.org *.wikinews.org wikipedia.org *.wikipedia.org wikiquote.org *.wikiquote.org wikisource.org *.wikisource.org wikiversity.org *.wikiversity.org wikivoyage.org *.wikivoyage.org wiktionary.org *.wiktionary.org *.wmcloud.org *.wmflabs.org wikimediafoundation.org mediawiki.org *.mediawiki.org wss://tools.wmflabs.org; report-uri https://csp-report.toolforge.org/collect;
< 
<html>
<head><title>307 Temporary Redirect</title></head>
<body>
<center><h1>307 Temporary Redirect</h1></center>
<hr><center>openresty/1.15.8.1</center>
</body>
</html>

Browser:
[Error] Origin https://locator.toolforge.org is not allowed by Access-Control-Allow-Origin.
[Error] Fetch API cannot load https://nominatim.toolforge.org/search?format=json&q=stads due to access control checks.

Cross-origin redirection to https://nominatim.toolforge.org/search?format=json&q=stads denied by Cross-Origin Resource Sharing policy: Origin https://wikimap.toolforge.org is not allowed by Access-Control-Allow-Origin.

I switched back to running without --canonical for now, as i can't fix all the tools that use nominatim proxy right now

I try some changes in .lighttpd.conf but without luck. It seems OpenLayers change the request from GET to OPTIONS methode. Perhaps this is a reason .

DB111 added a comment.EditedJun 7 2020, 9:39 PM

Just to be sure: Did you restart the service after your changes to become effective? I had luck with only the "Access-Control-Allow-Origin" line, no further lines for methods or so required.

DB111 added a comment.EditedJun 7 2020, 10:39 PM

Thank you @TheDJ! I tested all four combinations, I can call now your old & new URL from my old & new URL, looks good!

TheDJ added a comment.Jun 8 2020, 7:32 AM

@Kolossos did you try adding access-control-allow-methods: "OPTIONS,GET" at the same time ?

bd808 renamed this task from toolforge.org destroy same origin policy in JS to Same origin policy access between tools deliberately broken by toolforge.org migration.Jun 8 2020, 5:34 PM
bd808 added a comment.Jun 8 2020, 5:37 PM

I renamed this task because in reality the *.toolforge.org migration is deliberately breaking this "feature" of the old path based routing. This is much more clear if you read the T125589: Allow each tool to have its own subdomain for browser sandbox/cookie isolation task which is the parent of T234617: Toolforge. introduce new domain toolforge.org.

bd808 added a comment.Jun 8 2020, 5:48 PM

I my eyes the timeline for the hard end is much too short for volunteers. So I got the information about the move yesterday and have to work the next days.

The initial announcement to the cloud-announce mailing list was made on 2020-04-13. Follow up announcements were made on 2020-05-04 and 2020-06-02. The 60 day window from the initial announcement to the planned forced migration feels like a reasonable timeline to me, but I am certainly willing to hear rational arguments as to why more time would typically be needed for a tool maintainer to address such issues.

bd808 updated the task description. (Show Details)Jun 8 2020, 5:59 PM

Answers to the questions above:

  • Yes I restarted the server each time.
  • I tested "OPTIONS,GET" at the same time,but today also one by one or alone.
  • Thanks for renaming this task.

My question is, if I see the outgoing request from OpenLayers are changing from GET to OPTIONS if I switch to an external subdomain, then it means I will get no content back. Right?

Simple solution would be now to move the little php-script from wp-world to wiwosm and solve this problem.

bd808 added a comment.Jun 8 2020, 8:59 PM

The OPTIONS request would be the CORS preflight request. Teh browser would be expecting a response including an Access-Control-Allow-Origin header and possibly an Access-Control-Allow-Methods header.

DB111 added a comment.Jun 8 2020, 9:47 PM

@Kolossos: Now it works, a "Access-Control-Allow-Headers" was also necessary.
I disabled the CORS header output in your php scripts and added this to the .lighttpd.conf

setenv.add-response-header += ( "Access-Control-Allow-Origin" => "*", "Access-Control-Allow-Methods" => "GET, OPTIONS", "Access-Control-Allow-Headers" => "*" )

Thanks a lot to all for the help.

@bd808 : To your schedule question, you have right from a professional standpoint that 2 months should be enough for a change. From a standpoint of a volunteer that maitains scripts that are many years old it's maybe different. I only notice the problem of this script this week by an other popular script (geohack), which is still broken. So maybe a week more could be in my eyes helpful.

bd808 added a comment.Jun 9 2020, 10:31 PM

So maybe a week more could be in my eyes helpful.

I think you will probably get that week. Many humans are motivated by deadlines, so it is useful to set them to help create a sense of urgency or things would just keep being put off forever (see my own bug backlog for signs of that). We do not want to break tools unnecessarily, and we have found a couple of new small bugs in the process just this week that are not all properly fixed yet.

TheDJ added a comment.EditedJun 10 2020, 8:56 AM

Access-Control-Allow-Headers" => "*"

I suspect this is because of the always added response header: x-clacks-overhead: GNU Terry Pratchett
I suggest that if toolforge wants to add global headers, it should probably also 'globally' add that header to Access-Control-Allow-Headers. (Production wmf does this for the analytics headers I believe)

I didn't trigger that condition in nominatim, because my GET is a 'simple' json cors request, whereas this marks.php returns xml which causes it to be upgraded to preflight mode, which triggers the need for Access-Control-Allow-Headers/Methods in the options response to list all none-safe cors headers.

bd808 added a comment.Jun 10 2020, 8:10 PM

Access-Control-Allow-Headers" => "*"

I suspect this is because of the always added response header: x-clacks-overhead: GNU Terry Pratchett
I suggest that if toolforge wants to add global headers, it should probably also 'globally' add that header to Access-Control-Allow-Headers. (Production wmf does this for the analytics headers I believe)

I didn't trigger that condition in nominatim, because my GET is a 'simple' json cors request, whereas this marks.php returns xml which causes it to be upgraded to preflight mode, which triggers the need for Access-Control-Allow-Headers/Methods in the options response to list all none-safe cors headers.

My understanding is that "Access-Control-Allow-Headers" is about the headers that a user-agent is allowed to send to the server along with a CORS request. The "x-clacks-overhead" header is outbound from the server, not inbound. I believe that "Access-Control-Expose-Headers" would be the directive to instruct a user-agent to retain a custom header in the content it processed when receiving a CORS response. Am I misunderstanding what "Access-Control-Allow-Headers" controls?

DB111 added a comment.Jun 10 2020, 8:52 PM

My understanding is that "Access-Control-Allow-Headers" is about the headers that a user-agent is allowed to send to the server along with a CORS request. The "x-clacks-overhead" header is outbound from the server, not inbound. I believe that "Access-Control-Expose-Headers" would be the directive to instruct a user-agent to retain a custom header in the content it processed when receiving a CORS response. Am I misunderstanding what "Access-Control-Allow-Headers" controls?

You are right, it's about allowed headers along with the request. In this case: "X-Requested-With: XMLHttpRequest".

DB111 closed this task as Resolved.Jun 27 2020, 10:39 AM
DB111 claimed this task.