Integrating LogEntries with DomainEvents is something I have explored before, but it proved rather tricky to get right. I don't recall the details, but T389602 has some discussion.

In T427491#11967267, @CDanis wrote:

How sharp of a corner case is this for you?

Oh, I think I know what happened. There are log entries for category_redirect (commons:commons; User:RussBot) Pywikibot/10.2.0 (g19732) requests/2.32.3 Python/3.11.2.final.0 with no x-ua-contact set, but they are flagged as known-network. I guess we don't extract the contact info for requests from known networks then?

In T427491#11964711, @Joe wrote:

So I'm not sure what this task is about.

Care to provide examples?

In T427491#11964475, @BCornwall wrote:

While spoofing isn't new, this allows operators to spoof users, not programs. It's another thing we'd have to sift through.

In T427491#11963857, @ssingh wrote:

Per https://www.mediawiki.org/wiki/Manual:Pywikibot/User-agent, users can set a custom UA easily by setting user_agent_format. Should we perhaps not be encouraging that vs blanket allow-listing the default UA? That then falls into line with our other UA policy for library defaults -- each bot should set a custom UA.

This has been declined before, see T36778: Deploy extension Memento on Wikipedia sites

In T424846#11950008, @matmarex wrote:

Effect of these changes looks nice:

In T424824#11940675, @JMeybohm wrote:

In T424824#11939704, @hashar wrote:

An easy path is to add Lua/Make to the image and have the deployment-charts Rakefile to invoke the suite. That is quite easy to add and we would pair on it next week.

The main concern with this was that it could easily "blow up" if/when people start adding more tests that require additional dependencies - and there is no automatic way to satisfy them. But I'd be okay with biting that bullet and dealing with it when the time comes.

In T426665#11936145, @jnuche wrote:

The three pieces are:

• Access to Trusted Runners: Requires entry in https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/blob/main/projects.json
• Job must be tagged (GitLab job tag, no relation to git) with trusted. You already had that in your .gitlab-ci.yml
• Your (git) tags must be marked as protected by GitLab. You can do that at https://gitlab.wikimedia.org/repos/mediawiki/services/smokepy/-/settings/repository#js-protected-tags-settings. The usual approach is to specify the pattern v* so all your tags, present and future, get protected

In T426665#11935859, @jnuche wrote:

Fixed now, public-candidate job ran successfully: https://gitlab.wikimedia.org/repos/mediawiki/services/smokepy/-/jobs/828344

Re-opening, because this still doesn't seem to be working. I do see the trusted runner, but the publish-canidate job still gets stuck with the message "This job is stuck because of one of the following problems. There are no active runners online, no runners for the protected branch , or no runners that match all of the job's tags: trusted".

In T426665#11935147, @jnuche wrote:

Project added and new config deployed

In T426665#11932626, @taavi wrote:

This repository seems to be missing a software license.

Thank you for looking into this @Ottomata!

In T424824#11915027, @SLong-WMF wrote:

Hey Daniel! It's a little unclear to me what exactly is being tested here. I'm familiar with helm charts and orchestration tooling, are you looking for help figuring out what should be the test running architecture that runs existing tests (the foundation has a huge amount of existing unit, integration and UI layer tests) or are you looking at spinning up a new set of automated tests to validate some new functionality (or functionality that is currently uncovered)?

@SLong-WMF hi, Halley suggested I ping you on this ticket, because you might have ideas on how to approach this. I am trying to figure out how to best implement per-chart CI tests on the deployment-charts repo. The main challenge is that the nature of the services is very heterogeneous, so there may be a potentially large number of tools and dependencies needed to run the tests. On the other hand, right now there only two or three charts that have such tests.

I have been thinking that we could extend the Rake file we are already using in production to check helm charts to also run chart-specific tests. I have been discussing this with @Joe and @Blake on Slack. They both seem to like the idea, but the main problem is the dependencies that these tests introduce.

In T413570#11897097, @Nylki wrote:

Right now, it is treated like a bot.

Thank you for confirming. Is there a form or another formal process to apply for non-bot status? The bot-category does not fit very well for an mobile app whose traffic is user-driven (like a browser, as you said), and not centrally orchestrated.

In T424823#11895414, @Ahoelzl wrote:

@daniel can you provide background information how this supports KR or metrics work?

In T413448#11897478, @Blake wrote:

Hi @daniel, just wanted to check in - do you happen to know when I might be able to start migrating traffic? Thanks!

In T413570#11895915, @Bbbub wrote:

Thank you for the update, I'll be happy to report back if I observe any changes after tomorrow. What do you mean by

This will however not do anything for limits on media requests.

? If I am not mistaken this entire issue is about image / media requests. For us, the problem we encounter is definitely focused on the media requests from end-user clients ie apps. Are the request limits for media documented somewhere else? If not what are they if they're not counted like normal API requests?

In general I think the spec of an API module should be the same no matter where that module is installed. Extensions changing APIs provided by other components is something that proved problematic in the action API. It also makes versioning less meaningful. Ideally, the spec should tell a client what it can expect from an API module, no matter on which domain it is called.

In T413570#11892868, @Bbbub wrote:

I am sorry to say that this issue continues to be unresolved for our iOS / Android app. There was an improvement after the initial issue was fixed but still nearly every user has 429-issues with request volumes far below (<10 / min) the rate limits (should be 200 / min) that are talked about here: https://www.mediawiki.org/wiki/Wikimedia_APIs/Rate_limits for clients with compliant user agent. I've send an e-mail to bot-traffic@wikimedia.com as well to see whether I was on any blacklist.

Any progress on this? We ran into a situation with requests from the Wikipedia App getting blocked, and having the rate limit class in Turnilo would really help with investigating the problem...

In T414805#11875160, @A_smart_kitten wrote:

One potential difference that comes to mind is that -- potentially nobody (or few people) may have reasonably expected the thumbnail/image URLs to stop working at some point in the future (as long as the images in question are not deleted on the wikis themselves), as may be partially evidenced by the fact that these thumbnail URLs were/are in use within WMF-authored deployed code; whereas APIs/API methods could perhaps be reasonably expected to have parts that are deprecated/removed from time to time.

In T424768#11871173, @matmarex wrote:

Does MWHttpRequest automatically support cookies, and if yes, where does it persist them? This is necessary to get better rate limits when using owner-only tokens. (https://www.mediawiki.org/wiki/Wikimedia_APIs/Rate_limits/FAQ#Why_do_I_need_to_send_cookies_with_owner-only_consumers_and_OAuth_1.0?)