When our infrastructure is being blocked by 429s (Too Many Requests), I want to be able to see alarm being triggered on PagerDuty, so that I can be a able to timely react to the such event.
**Acceptance criteria**
When we are blocked by 429s support engineer gets a notification.
**ToDo**
- [ ] add an alarm to IaC
- [ ] trigger PagerDuty alarm based on that
- [ ] test it
**Notes**
We can monitor the logs of `structured-data` service to do this, all of the logs that we need to catch will contain `429 Too Many Requests` within the log entry. Here's an example:
```
/app/submodules/wmf/wmf.go:589: 429 Too Many Requests:
{
"error": {
"code": "too_many_requests",
"message": "AlimitednumberofparallelconnectionsperIPisallowed."
}
```
==Related task
https://phabricator.wikimedia.org/T342171