Page MenuHomePhabricator

Quarry is a tested way to perform a DOS against labsdb
Closed, InvalidPublic

Description

How to replicate:

  • Login to quarry- it does not require a Labs/wikitech account, only a regular wiki login
  • You can execute as many "SELECT * FROM enwiki_p.revision a JOIN enwiki_p.revision b JOIN enwiki_p.revision c JOIN enwiki_p.revision d JOIN enwiki_p.revision e JOIN enwiki_p.revision f JOIN enwiki_p.revision g JOIN enwiki_p.revision h JOIN enwiki_p.revision i" as you want. Add more JOINS for more fun!
  • You now have successfully maximized CPU usage and performed a successful DOS against labsdb

How to fix:

  • Some limitations should be set for queries executed

Event Timeline

jcrespo assigned this task to yuvipanda.
jcrespo raised the priority of this task from to Needs Triage.
jcrespo updated the task description. (Show Details)
jcrespo added a project: Toolforge.
jcrespo subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
valhallasw set Security to None.

Disable querry, as it goes against labs usage policy (allowing arbitrary queries to non lab users)

Note that this is a Tool Labs policy, not a generic Labs policy.

This does, of course, not mean that allowing every user to DOS the database server is a good idea.

So the measures that Quarry has in place to prevent abuse are:

  1. Tied to user accounts on SUL, and the queries themselves have the username attached. We can trout people who abuse it.
  2. Kills queries after a timeout (currently at 20min)

There's currently nobody who is abusing labsdb through Quarry - in fact, it is much easier to do so by getting a tools account (since you get programmatic access to run as many queries as you want).

If you think more constraints should be put in place, let's discuss and work on that - I'll be happy to implement more.

I'll note that the only person who seems to have run toxic queries has been you :) http://quarry.wmflabs.org/JCrespo%20(WMF) has broken Quarry as well.

('broken' in the sense that it is reporting non-running queries (they were killed too) as running, not that Quarry itself isn't working anymore)

My suggestions:

  • queries should be limited to 1 per account
  • There should be a throttling on new account creation

I've filed T104316 for #1. Can you explain what you mean by #2? Quarry's logins are just Wikimedia SUL accounts...

I think '1' is too stringent a limit - Quarry has never caused labsdb problems before. I do agree a limit is required, however - I was going to implement it the first time it was necessary because someone DoS'd labsdb, and that is now :)

Accounts probably should be autoconfirmed, or have a verified email to get in contact with them.

Or maybe have a default small query limit (1 min), make it longer for older accounts.

I think we should be as permissible as possible and do restrictions based only on actual abuse than imagined. Autoconfirmed does not work - autoconfirmed on which wiki? meta? enwiki? I also do not know if there's a verified way to get in contact with them.

My current setup for dealing with abuse is:

  1. Ban user from Quarry
  2. Trout them on their talk page.

And I've so far not had to use it.