I'm thinking of extending the scoring_system config with a block for matching user-agents.

queue_maxsize: 100 # pending tasks
queue_thresholds:
  everyone:
    n: 25
    user-agent: ''
  email_in_user-agent:
    n: 25
    user-agent: '\S+@\S+'
  white-list:
    n: 100
    user-agent: 'precached|ChangePropagation|MW_api.php'

I don't like the idea that a scoring system should be able to take a user-agent. I think we'll want an external prioritization system that passes priorities into the score() method.

OK. New idea. I think that ORES should independently flag the priority of requests and that a scoring system could then use those priorities to decide when it is too overloaded. E.g.:

ores:
  priority:
    everyone:
      user-agent: ''
    email_in_user-agent:
      user-agent: '\S+@\S+'
    white-list:
      user-agent: 'precached|ChangePropagation|MW_api.php'

...

scoring_systems:
  celery_queue:
    queue_maxsize: 100
    queue_thresholds:
      everyone: 10
      email_in_user-agent: 25
      white-list: 100

As a fun side-project, I'm studying distributed systems theory and it seems, in services using several queues is highly encouraged rather than using prioritization in one queue. I think that's why celery has virtually no support of prioritization and everywhere people say "use several queues instead". An example is Out-of-band data.

I hereby suggest declining this. We don't have capacity problems anymore and if needed we should dedicate fast lanes (queues) for important jobs and not prioritize requests. If there's no objection by next Monday, I will close this.