I made a patch for core and for the two relevant maintenance scripts that I could find.

In T411502#11431294, @brennen wrote:

I created the repo repos/mediawiki/services/smoke-alarm but I can't do anything to it. Apparently I can't even create a main branch. Please make me the owner of that repo.

@jeena did this a bit ago.

This is familiar. I it's probably something like "developer access to a group allows a project to be created, but developer access to a project granted through the group doesn't allow a main branch to be created".

Although the A category within x-trusted-request flag includes WMCS (as well as other wikimedia network traffic), it would indeed be valuable to better differentiate internal processes from true WMCS tools and services.

In T411623#11428668, @colewhite wrote:

Done! Let us know if something is amiss!

Still happening at a low rate, maybe one per day. The ones that involve InfoAction seem to all come from job runner. There are also instances from web requests, from PageTriage's handler for the ArticleViewFooter hook.

Bumping priority to high, since filtering access logs by x-trusted-request is on the critical path for work on api rate limiting.

In T411503#11428004, @taavi wrote:

Sure, done at https://tools-static.wmflabs.org/admin/meta/worker-ips.json and documented at the same place.

In T411503#11427653, @taavi wrote:

Ok, we now publish https://meta.wmcloud.org/cloudvps-ips-all.json (which is now documented at https://wikitech.wikimedia.org/wiki/Help:Cloud_VPS_IP_space#Machine-readable_data). I can't quite tell from the comments here - would a similar file for the Toolforge workers be helpful as well?

@Mooeypoo looks like this is related to the API stats you added recently.

In T411503#11426657, @CDanis wrote:

We already support the Googlebot JSON format, which has become something of a de facto standard.

In T411503#11424246, @taavi wrote:

I don't think we currently have any places outside of https://wikitech.wikimedia.org/wiki/Help:Cloud_VPS_IP_space that publish our IP space. Would it be helpful if we published the same information in some machine-readable format?

For the record, I asked @taavi about including information about the tool or user in requests coming from WMCS. He said it's not possible because the tools use HTTPS, we can't mess with the encrypted traffic.

In T411411#11421223, @RLazarus wrote:

<rzl> mmm I agree with you about the problem but I'm not sure about the solution
<rzl> I think you can still get into confusing versioning semantics with a race condition if 0.0.1 is live, I write 0.0.2-rzl, and you write 0.0.2-duesen

In T410198#11416038, @akosiaris wrote:

It depends on your end goal. Do you want to remove them so that they don't mess with your data in a statistical manner? Or to be absolutely certain ?

I 'll also point out that part of the task is about having internal workloads talk to API Gateway (via the mesh or directly, doesn't matter). I 've always been against such a change. The API Gateway should be an entry point from the outside, not to be used internally. That's what the mesh is for (and what Clement is pointing out above for some workloads).

In T365977#11405395, @Alex44019 wrote:

Doesn't feel like my call really haha

In T410198#11415578, @akosiaris wrote:

None of these IP belong in either 10.192.72.0/24 or 10.194.128.0/17, so they are unrelated.

However, the IP address listed previous are indeed wikikube-worker IPs, so node IPs. That is consistent indeed with health checks (which are originating on purpose from the IP address of the host of the pods).

In T410198#11378640, @hnowlan wrote:

All of the 10.192 addresses are wikikube workers, which would correspond to either health checks or something else in wikikube making direct calls (which we actively discourage). The 172.16.* hosts are from WMCS. We need to investigate what kind of traffic we're seeing from these hosts further most likely

Deployed and tested

As for the disallow fix, might be as simple as $argv[1] === $script?

Replacing Router and Module with more narrow interfaces in Handler would be nice, yes. Doesn't even have to be new/separate helper classes, the first step can be narrow interfaces implemented by Router and Module.

Deployed and tested. We still only use per-hour limits in production though.

@Alex44019 Indeed, the problem is that we don't know which options have parameters, and which don't. I am preparing a patch that allows options before the script path, but only if they don't need a parameter or they use --foo=bar syntax. I think that would be a useful compromise, though it's a little awkward that --foo bar is supported in some places but not in others.

Do we need a factory for handlers, or for modules? It seems to me like we need an easy way to get a module, because modules haven an OpenApi spec. The handlers only generates bits of that... also, once we have a module, we can easily get a hanlder from it.

Hi @Alex44019, thank you for the analysis! This is indeed unfortunate and not intended. I don't remember if passing the raw argv down to the script implementation is necessary. if so, we should prevent options to be passed before the scipt name. But I supsect that it was simply an oversight on my part - I was probably just assuming that the sscript name is always $argv[1].

In T400119#11395040, @Guycn2 wrote:

This seems to be blocking legitimate web captures by the Internet Archive (see, for example, here). Is this actually the intended outcome?

It may be possible to implement this using route metadata: the route sets the metadata, and the ratelimit action uses it. That way, the route can control the policy. A special "no-limit" policy could be used to disable rate limiting entirely (like the "no-limit" class).

Let's do T410273: api rate limiting: Assign ratelimit class based on IP range instead.

In T410198#11378496, @hnowlan wrote:

Could you supply some of these IP addresses for investigation? My gut feeling is that these are going to be health checks. Is this the api-gateway or the rest-gateway?

We currently have hourly limits in place that would potentially cause load spikes at the beginning of each hour:

This is done, @Clement_Goubert deployed API rate limiting for all users of all APIs today (in "shadow mode", so nothing is enforced).

According to turnilo we are getting a couple of hundred requests per day with the xslt= parameter set.

Envoy's ratelimit service configuration supports catch-all rules, but unfortunately they don't play well with metrics keys. relevant upstream ticket: https://github.com/envoyproxy/ratelimit/pull/996. Looks like someone is working on it.