Page MenuHomePhabricator

Define specification for retrieving API usage data
Open, MediumPublic

Description

The API Portal prototype includes an API usage view where a user can see their historical API usage data.

Note: Feel free to adjust the list of tasks below. These are just approximations based on my understanding. Possible duplicate of T246275

Mock data format:

{
  "limit": {
       "count": 10000,
       "period": 86400,
  },
  "usage": {
    "2020-04-20": 7819,
    "2020-04-21": 8090,
    "2020-04-22": 7401,
    "2020-04-23": 6991,
    "2020-04-24": 9118,
    "2020-04-25": 4251,
    "2020-04-26": 5823
   }
}

To do:

  • Create policy for data collection and retention. Review with relevant parties.
  • Create a specification for how this data can be retrieved by the API Portal (Requires T246269)

Event Timeline

apaskulin created this task.Apr 9 2020, 4:18 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 9 2020, 4:18 PM
apaskulin triaged this task as Medium priority.Apr 9 2020, 4:23 PM
apaskulin updated the task description. (Show Details)Apr 9 2020, 4:49 PM
eprodromou updated the task description. (Show Details)Apr 14 2020, 2:42 PM

I spoke to Jacob Rogers, who deals with our privacy policy for WMF. Here's the policy I outlined:

  • We'd like to give the app developers statistical information about how many calls their app has been making. "Your app EvanTool made 50,000 out of your allowed 100,000 API calls per day."
  • We'd also like to have some historical data, so that app developers can see if their API usage is going up or down with time.
  • We won't share individual API call information, like date and time, inputs, outputs, etc. with the app developer. This is an aggregate count of total API calls for an app, across all users of that app, for a given time period (a day).
  • The app usage information would only be shown to the app developer, not to the public.
  • It's sufficient for our requirements to only keep historical API usage information on a rolling 90-day window, and discarding it after 90 days, if needed.

Here was Jacob's reply:

I've taken a look through the info you provided as well as double checking with Security. You're approved to share this information as it's currently presented, we don't see any privacy policy issues and it may even be valuable for you to request a retention period exception to keep the data longer if there's ever a need to prove to someone that they did receive their allotted API calls.

If in future you intend to make the data more granular in some way (e.g., reporting out details of each individual API call in a way that could be used to identify the requester) please send us another request to review that.

I'm going to see what it takes to get a retention period extension, but for now I think we should stick with the 90-day retention.

eprodromou updated the task description. (Show Details)Apr 14 2020, 2:56 PM
eprodromou updated the task description. (Show Details)Apr 14 2020, 2:59 PM

I dropped the checkbox for collecting and storing the data; that's covered in the API Rate Limiting epic T246269.

apaskulin updated the task description. (Show Details)Apr 14 2020, 3:11 PM

@eprodromou Great!

Will we need to communicate this policy to users of the Portal?

apaskulin renamed this task from API usage to Define specification for retrieving API usage data.Apr 17 2020, 10:49 PM
apaskulin updated the task description. (Show Details)May 4 2020, 6:17 PM