Page MenuHomePhabricator

It would be cool if cargo had configuration to reduce risk of denial of service attacks
Open, Needs TriagePublicFeature

Description

DB queries can be expensive. It would be cool if cargo had configuration options to limit risk around expensive queries.

The options available (Note, these may differ between mariadb and mysql):

  • MAX_EXECUTION_TIME - MediaWiki already has support for this as an item in the $options array. It kills queries that take too long. I think it might make sense to have a default of 90 seconds (this option is in ms), and have a config option for users who want to be more paranoid. Queries taking longer than 90 seconds probably aren't likely to work that well anyways due to php max execution time.
  • max_join_size - will prevent running queries that mariadb thinks will look at a lot of rows (mariadb could be wrong of course). This has the benefit of not even starting the query if it looks like it would be bad. That said I don't know how accurate it really is.
  • LIMIT .. ROWS EXAMINED - if you have an SQL statement like LIMIT XXX ROWS EXAMINED 10000000; The sql query will stop after looking at 10000000 rows. It will possibly give partial results with a warning if that happens, or it might give an error. I think this is similar to max_join_size except that instead of guessing before the query begins, it runs the query until it hits the limit. This is different than the normal LIMIT since it is rows looked at not rows returned. (I think this is mariadb only and not supported by mysql. Unfortunately the MW abstraction layer doesn't support this afaik)

Anyways, i think it would be useful to have these as config options in Cargo. Especially the MAX_EXECUTION_TIME one, which might make sense to be on by default with a high limit.

Event Timeline

I agree that supporting MAX_EXECUTION_TIME, especially, would be great. (The other options would probably be less effective, and harder for admins to figure out.) But it seems like it was only added to MySQL in version 5.7, which means that it can only work with MW 1.39 and higher... which is better than nothing, of course.

I think MediaWiki already has some code where it will detect mysql version to figure out if it can use the MAX_EXECUTION_TIME option.

The nice things about the other two is that they'd be more consistent - queries would probably either run or they wouldn't [well sort of, guess it depends on query plan], where time based limits can vary depending on server load, which can be frustrating to users. I guess they can also be useful as a guard against mistakes (If the limit is something like 100000000 rows, you'd probably never hit that in a valid query no matter what you do, but might hit it if you typo a cartisian join, although i don't know if that type of typo is even possible in Cargo queries)

I don't know if queries automatically stop when php hits the php.ini max execution time. If they don't, the SQL MAX_EXECUTION_TIME might be useful even at a really high value, to stop run away queries from just going on and on.

AIUI PHP's max_execution_time is a CPU-time limit, rather than a wall time limit. It also cannot interrupt pending queries.

Change 1008844 had a related patch set uploaded (by TK-999; author: TK-999):

[mediawiki/extensions/Cargo@master] CargoQueryAPI: Add ratelimit support

https://gerrit.wikimedia.org/r/1008844

Something that came up recently (in the context of lol.fandom.com) was the ability to add rate limits to the Cargo query API, since there have been a few cases of people accidentally issuing too many queries in too short a timeframe. I made a patch for that.

Change 1008844 merged by jenkins-bot:

[mediawiki/extensions/Cargo@master] CargoQueryAPI: Add ratelimit support

https://gerrit.wikimedia.org/r/1008844

@TK-999 - thank you for this patch! I didn't know about that pingLimiter() method. This looks like a great improvement!

Could pingLimiter() also be used to limit queries run within the Special:CargoQuery interface?

@Yaron_Koren Yeah, this could also be integrated with the special page, good idea!

Change 1008967 had a related patch set uploaded (by TK-999; author: TK-999):

[mediawiki/extensions/Cargo@master] Add ratelimiting support to Special:CargoQuery

https://gerrit.wikimedia.org/r/1008967

Change 1008967 merged by jenkins-bot:

[mediawiki/extensions/Cargo@master] Add ratelimiting support to Special:CargoQuery

https://gerrit.wikimedia.org/r/1008967