We have had several reports from users using various API endpoints that Anubis is blocking them. We suspect this is due to them (or the tool they are using) having a browser-like user-agent (or missing a user agent). We would like to try excluding the API endpoints from having to complete Anubis challenges and monitor the traffic.
List of endpoints to exclude:
- /w/api.php
- /w/rest.php
- /query/sparql
Dev Notes:
- we looked that the rate limits in ingress-nginx, and we are happy that as part of this ticket we don't need to adjust them.
- we acknowledge that if this results in an unsustainable amount of traffic, we will have to revert this change.
Option 1: continue running all traffic through Anubis and edit the Anubis config to ignore the above endpoints
Option 2: have the above endpoints skip Anubis
- This is likely difficult due to all traffic that hits the platform-nginx always going through Anubis currently. It is likely not worth investigating further.
Merged solution for option 1
