Page MenuHomePhabricator

WikibaseQualityConstraints include extension name in UA for query service requests
Closed, ResolvedPublic5 Estimated Story Points

Description

In T204267 the extension flooded the query service with requests and ended up getting all mediawiki requests banned.
In the future other less floody SPARQL requests may come from mediawiki and thus have the same UA & get banned (unless the banning is done by IP only?)
In order to avoid these blankety bans each sub product / service within mediawiki making the requests should add its own product to the UA in addition to the default mediawiki one.

We should also include the name of the wiki that the request is coming from. This is planning for a future where QualityConstraints may also be run on another wikibase instance within the WMF cluster (for example commons).

Current example UA:

MediaWiki/1.32.0-wmf.20

Proposed UA:

MediaWiki/1.32.0-wmf.20 WikibaseQualityConstraints

User-Agent RFC: https://tools.ietf.org/html/rfc7231#section-5.5.3

Pointers

The option probably needs to be added to https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/WikibaseQualityConstraints/+/refs/heads/master/src/ConstraintCheck/Helper/SparqlHelper.php#583 however other usage should be checked..

Event Timeline

Addshore created this task.Sep 17 2018, 7:52 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 17 2018, 7:52 AM
Addshore triaged this task as Low priority.Sep 17 2018, 7:52 AM
Addshore added a subscriber: Smalyshev.

@Smalyshev In order to decide if there is any point in this one....

Does the query service actually differentiate requests from a single IP with different UAs for rate limiting / banning?

Addshore moved this task from Backlog to Questions on the wdwb-tech-focus board.

TBD / TBA does the query service actually differentiate requests from a single IP with different UAs?

That sounds like a really easy way for any spammer to circumvent a ban, so I would be very surprised if we actually did this tbh.

Proposed UA:

MediaWiki/1.32.0-wmf.20 WikibaseQualityConstraints

I would think that WikibaseQualityConstraints should come first, since MediaWiki acts more like the “library” component here:

WikibaseQualityConstraints MediaWiki/1.32.0-wmf.20

Rate limiting is bucketing based on IP+User agent, so having distinctive user agent for WBQC is certainly a good idea.

Addshore updated the task description. (Show Details)Sep 18 2018, 9:08 AM
Addshore moved this task from Questions to Ready on the wdwb-tech-focus board.
Addshore moved this task from needs discussion or investigation to ready to go on the Wikidata board.
Addshore added a project: Wikidata-Campsite.
Addshore moved this task from Incoming to Ready to estimate on the Wikidata-Campsite board.
Addshore moved this task from ready to go to in progress on the Wikidata board.Sep 18 2018, 2:23 PM

Change 463923 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[mediawiki/extensions/WikibaseQualityConstraints@master] Set custom user agent in SparqlHelper

https://gerrit.wikimedia.org/r/463923

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptOct 2 2018, 10:11 AM

Change 463923 merged by jenkins-bot:
[mediawiki/extensions/WikibaseQualityConstraints@master] Set custom user agent in SparqlHelper

https://gerrit.wikimedia.org/r/463923

This should be verified once deployed.
This should be deployed on the evening of the 17th October.
We can verify by making sure the query service gets requests with the new UA