Page MenuHomePhabricator
Create Task
Maniphest T258748

Client Developer has a cookie-free API call
Closed, ResolvedPublic
Actions

Description

"As a Client Developer, I want to avoid getting Set-Cookie headers, or having to provide Cookie headers, in my API requests, so that I can concentrate on OAuth 2.0 as the sole way to authorize my app."

RESTful API servers are usually cookie-free; client developers don't have to keep track of cookies in their clients, and can use other authentication mechanisms, like OAuth 2.0, for their authorization.

MediaWiki and other parts of our stack can add cookies even for API endpoints, so it would be beneficial to enforce this cookie-free discipline at the gateway level.

See sub-tasks for done criteria.

Related Objects
Search...

Event Timeline

• eprodromou created this task.Jul 23 2020, 8:13 PM
• eprodromou added a comment.Jul 23 2020, 8:21 PM

I noticed recently when using the Core REST API that my client received these cookies:

curl -v https://en.wikipedia.org/w/rest.php/v1/page/Sandbox
# ... other output ...
set-cookie: WMF-Last-Access=23-Jul-2020;Path=/;HttpOnly;secure;Expires=Mon, 24 Aug 2020 12:00:00 GMT
set-cookie: WMF-Last-Access-Global=23-Jul-2020;Path=/;Domain=.wikipedia.org;HttpOnly;secure;Expires=Mon, 24 Aug 2020 12:00:00 GMT
set-cookie: GeoIP=CA:QC:Montreal:45.53:-73.59:v4; Path=/; secure; Domain=.wikipedia.org

I'm not actually sure where these come from. If they're upstream of the gateway, it would be cool to filter them out if we can. If they're downstream of the gateway, it would be interesting to know how important they are and if they could be left out of requests that are specifically for the API.

• eprodromou moved this task from Backlog to Ready on the Platform Team Workboards (Green) board.Jul 27 2020, 11:50 AM
• eprodromou edited parent tasks, added: T255032: Wikimedia API Gateway Public Launch; removed: T255030: Wikimedia API Gateway MVP.Jul 27 2020, 12:06 PM
• eprodromou reassigned this task from • eprodromou to hnowlan.Jul 27 2020, 1:00 PM
• eprodromou triaged this task as Medium priority.
hnowlan added a comment.Jul 30 2020, 1:57 PM

Should Envoy be concerned with managing cookies at all? If cookies are being set then that seems like a problem with Mediawiki or other upstream services. I think Envoy should not bother managing cookies at all, it's up to the upstream services to ignore cookies if cookies are for some reason being set.

• WDoranWMF claimed this task.Jul 30 2020, 2:52 PM
• WDoranWMF moved this task from Ready to Doing on the Platform Team Workboards (Green) board.
• WDoranWMF added a subscriber: hnowlan.
• eprodromou moved this task from Doing to Backlog on the Platform Team Workboards (Green) board.Jul 30 2020, 7:11 PM
• eprodromou claimed this task.Jul 30 2020, 7:50 PM
• eprodromou updated the task description. (Show Details)
• eprodromou added a subscriber: • WDoranWMF.
• eprodromou added a comment.Jul 30 2020, 7:55 PM

I split this into two tasks, T259294 and T259296.

I'm going to figure out how important the cookie headers that are coming from the MediaWiki REST API are; if they're not important, we can do the second ticket, but that shouldn't block the first.

• eprodromou added a comment.Jul 30 2020, 8:08 PM

OK, I'm learning important things about cookies! The WMF-Last-Access and WMF-Last-Access-Global cookies are set by Varnish and/or ATS for unique device tracking, documentation here:

https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Traffic/Unique_Devices/Last_access_solution

The GeoIP cookie is for Analytics; documented here.

https://wikitech.wikimedia.org/wiki/Geolocation

All of them are set by Varnish or the traffic layer, so cookie-filtering in Envoy won't affect them. I'm going to check with Traffic team if they make sense for API traffic (seems like no to me, but I could be wrong), and if not if it's possible to leave them off for API Gateway requests.

• WDoranWMF added subscribers: • Nuria, Ottomata.EditedJul 31 2020, 12:38 PM
In T258748#6349930, @eprodromou wrote:

OK, I'm learning important things about cookies! The WMF-Last-Access and WMF-Last-Access-Global cookies are set by Varnish and/or ATS for unique device tracking, documentation here:

https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Traffic/Unique_Devices/Last_access_solution

The GeoIP cookie is for Analytics; documented here.

https://wikitech.wikimedia.org/wiki/Geolocation

All of them are set by Varnish or the traffic layer, so cookie-filtering in Envoy won't affect them. I'm going to check with Traffic team if they make sense for API traffic (seems like no to me, but I could be wrong), and if not if it's possible to leave them off for API Gateway requests.

Can we also check with Analytics? @Nuria and @Ottomata are these cookies useful for request to the MW REST API and/or would they be in the future?

Ottomata added subscribers: mforns, Milimetric.Jul 31 2020, 1:12 PM

The GeoIP cookie is for Analytics; documented here.

I don't think the GeoIP cookie is for analytics; that documentation just says that we geolocate IPs in a geocoded_data field on some Hive tables. I didn't know there was a GeoIP cookie! I think Traffic might use that for GeoDNS routing, but I'm not sure.

[is WMF-Last-Access] useful for request to the MW REST API and/or would they be in the future?

I'm going to guess not, these are used for counting unique devices, and I don't know if we include API requests in those counts. @Nuria and @mforns or maybe @Milimetric would know better.

• eprodromou moved this task from Backlog to Next Sprint on the Platform Team Workboards (Green) board.Aug 5 2020, 1:34 PM
• WDoranWMF moved this task from Next Sprint to Ready on the Platform Team Workboards (Green) board.Aug 6 2020, 6:08 PM
Naike added a project: Platform Team Sprints Board (Sprint 1).Aug 7 2020, 11:04 AM
• eprodromou mentioned this in T260943: Don't set cookies for api.wikimedia.org at the caching layer.Aug 20 2020, 8:54 PM
• eprodromou added a subscriber: Brandon.

I've pinged @Brandon and he thought it was OK to remove these cookies for API Gateway calls. I've started the ticket T260943 to track progress.

Brandon added a comment.Aug 21 2020, 12:44 AM
In T258748#6401164, @eprodromou wrote:

I've pinged @Brandon and he thought it was OK to remove these cookies for API Gateway calls. I've started the ticket T260943 to track progress.

well you pinged the wrong brandon though...

Brandon unsubscribed.Aug 21 2020, 12:45 AM
• Nuria added a project: Traffic.Aug 21 2020, 4:38 AM
Restricted Application added a project: SRE. · View Herald TranscriptAug 21 2020, 4:38 AM
CDanis added a subscriber: BBlack.Aug 21 2020, 4:49 AM
• eprodromou added a subscriber: Brandon.Aug 24 2020, 2:32 PM

Ha! Sorry about that, @Brandon!

• eprodromou removed a subscriber: Brandon.Aug 24 2020, 2:32 PM
• eprodromou added a subtask: T260943: Don't set cookies for api.wikimedia.org at the caching layer.Aug 24 2020, 2:35 PM
• Nuria added a comment.Aug 24 2020, 8:57 PM

@eprodromou In order to know how/if removal of cookies will affect metrics we would need to run some tests on our unique devices calculations. The GeoIP cookie is not used for analytics at all but it is used * i think* for routing of requests. Traffic can speak to that

Naike closed subtask T259294: Configure API Portal to only send session cookie to its own routes as Resolved.Aug 26 2020, 2:24 PM
• eprodromou moved this task from Ready to Doing on the Platform Team Workboards (Green) board.Aug 26 2020, 3:02 PM
• WDoranWMF moved this task from Doing to User Story Review on the Platform Team Workboards (Green) board.Sep 2 2020, 3:35 PM
• eprodromou moved this task from User Story Review to Blocked on the Platform Team Workboards (Green) board.Sep 16 2020, 1:37 PM

I'm moving this to blocked until we work out how to get the cookies out of the API requests in varnish.

• eprodromou moved this task from Blocked to Backlog on the Platform Team Workboards (Green) board.Sep 16 2020, 3:32 PM
• eprodromou moved this task from Backlog to Doing on the Platform Team Workboards (Green) board.
• WDoranWMF moved this task from Sprint 1 to Sprint 4 on the Platform Team Sprints Board board.Sep 25 2020, 5:58 PM
• WDoranWMF edited projects, added Platform Team Sprints Board (Sprint 4); removed Platform Team Sprints Board (Sprint 1).
• WDoranWMF moved this task from Doing to Ready on the Platform Team Workboards (Green) board.Sep 30 2020, 3:30 PM
• WDoranWMF removed projects: Platform Team Sprints Board (Sprint 4), Platform Team Workboards (Green).
Joe subscribed.Sep 30 2020, 3:37 PM

Those cookies are harmless cookies we set at the edge cache for all requests.

We could add an exception for the api gateway, but it would add more complexity to our already complex varnish business logic, and AIUI this is an optimization rather than a necessity.

I'll let Traffic folks comment further, but I'd be inclined to avoid special-casing the api gateway here.

• eprodromou closed subtask T259296: Filter cookie headers for all API route requests as Resolved.Oct 5 2020, 4:56 PM
• eprodromou edited parent tasks, added: T255034: Wikimedia API Gateway Long-term Use; removed: T255032: Wikimedia API Gateway Public Launch.Oct 8 2020, 10:21 PM
Aklapper removed • eprodromou as the assignee of this task.Apr 1 2021, 5:52 AM
hnowlan added a comment.Apr 1 2021, 9:37 AM

I don't think this is a necessary task, I am closing it - we can reopen if this becomes a major issue.

hnowlan closed this task as Resolved.Apr 1 2021, 9:38 AM
RhinosF1 subscribed.Aug 15 2022, 4:46 PM
Vgutierrez closed subtask T260943: Don't set cookies for api.wikimedia.org at the caching layer as Resolved.Sep 7 2022, 8:42 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits