Background

Technical contributors is an essential audience, but one that is generally not well understood. API users are a slice of that broader community, and one which we would like to better understand, so that we can better inform success metrics for the ongoing work in this space, as well as informing the appropriate level of investment and impact for future projects.

The purpose of this project is to better understand API developer growth and attrition rates. While we can see that Wikimedia developer account creation is growing steadily over time, account creation is not necessarily directly correlated to successful contribution and participation in the technical community. API utilization can be an indicator of success (or loss) within the developer audience.

Scope

• Begin to measure organic growth (or attrition/churn) for API adoption
  • Differentiate unique developers from unique applications where possible
• Include organic growth monitoring to the API analytics dashboard (month over month? Ideally YoY would also be possible, but unclear if we can keep data long enough for that)
• [Stretch] Monitor changes to adoption of existing applications or users (ie: a deepening relationship based on more frequent and/or broader API requests)

Expected Outcomes

The goal of this work is to better understand the API landscape and how it is evolving over time. Ideally, the investments that we are making in improving developer experiences (eg: vNext APIs, developer portal) will result in an increase in new developer adoption and depth of utilization.

Known limitations

The biggest limitation that we have in this work is that not all requests provide a reliable identifier. Specifically, user-agents may change over time, independently of it actually being associated with a new application or user.

Data retention policies may also affect the accuracy of this work. For example, if a script or automated process runs on a quarterly basis, it may show up as a new application every quarter.

Related work

This work builds on the API analytics dashboard and data pipeline that exists today.

1. 1. Dependencies
2. Token data will be required in the webrequest table (if not already present from T417879: Add OAuth information to X-Analytics data to obtain better API usage metrics) to more reliably differentiate and attribute API traffic.
3. We will likely need additional data from Toolforge to identify/differentiate tools, as they are largely exempt from user-agent and authentication policies.
4. Superset performance remains a challenge; the performance issues likely need to be addressed before we can meaningfully expand the scope of API analytics.

1. 1. Next steps
2. Determine a methodology for how to reliably attribute traffic.

Timeline

Measuring organic growth will be tired to a Q4 hypothesis (or possibly a new KR) to measure the long term impact and efficacy of the new developer portal. However, to establish a baseline, this work may need to be completed prior to the portal launch in February (Q3)