User Story: “As one of many (potentially millions) of users of the ChatGPT plugin, I want to be able to access the plugin without downtime or lags, so that I can maintain a positive impression of the Foundation and its services”

Acceptance criteria

What is the expected result of this story?

We have made changes to our infrastructure to support our ChatGPT plugin getting hit significantly more often than it is currently being hit.

ToDo

• Discuss traffic implications and expected volume
• Update terraform infrastructure to support this expected volume
• Update the app if necessary to accommodate these infra changes and impose any necessary rate limiting not implemented at the ALB or other level
• Setup necessary monitoring and alerting to ensure we're on top of any changes to overall traffic
• Understand and plan for the financial implications of these changes

Test Strategy

How I'm planning to test this? Can I do integration testing or this should be tested manually? Where this testing should be happening (what environment, local or dev)? What this can influence (what can be broken by this feature)?

• We will need to think through how to load test this plugin - I would defer to any existing mechanisms in place here

Checklist for testing

• We can hit the plugin many times without DDOS'ing it
• We have a way to monitor traffic in the future

Things to consider:

• We will need new monitoring and alerting
• We will need to document this infrastructure

Description (optional)

We are currently in a limited alpha phase for our Wikipedia ChatGPT plugin - we have about 50 WMF employees who have access to the plugin, but we'd like to roll it out to more users, particularly those outside of the Foundation. If we were to open this up to the full set of users who have access to plugins (which we will very likely want to do, though likely not for several weeks, if this happens at all), we're looking at many orders of magnitude above this, namely several million users. We will need to think through how to update and harden our infrastructure to accommodate this.