Page MenuHomePhabricator

Request increased quota for editgroups Toolforge tool
Closed, ResolvedPublic

Description

Tool Name: editgroups
Quota increase requested: +1 continuous jobs
Reason: This tool is a Django+Celery web app which has experienced issues with the shared redis database. I would like to try migrating it to use its own redis container, following the brand new approach offered by @bd808 at https://wikitech.wikimedia.org/wiki/Tool:Containers#Redis_container. This involves running one more service (the redis container), on top of the three existing ones:

  • the web service
  • the Celery instance
  • the edits ingester (listens to the Wikimedia eventstream to watch for any relevant edits to be indexed)

The redis service itself would not need a lot of resources at all (it's primarily used for communication between the web service and Celery, and only marginally for caching small things).

$ toolforge jobs quota
Running jobs                                  Used    Limit
--------------------------------------------  ------  -------
Total running jobs at once (Kubernetes pods)  3       16
Running one-off and cron jobs                 0       15
CPU                                           1500m   8
Memory                                        1536Mi  8Gi

Per-job limits    Limit
----------------  -------
CPU               3
Memory            6Gi

Job definitions                             Used    Limit
----------------------------------------  ------  -------
Cron jobs                                      0       50
Continuous jobs (including web services)       3        3

Event Timeline

+1

This request might be a good excuse to bump the deployments.apps quota to 16 for all tools similar to the changes made for T362520: [maintain-kubeusers] Increment default services quota.

dcaro changed the task status from Open to In Progress.Jun 11 2024, 8:43 AM
dcaro claimed this task.

project_1317_bot_df3177307bed93c3f34e421e26c86e38 opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/323

maintain-kubeusers: bump to 0.0.147-20240611090228-a9acf2f7

This has been deployed :)

tools.editgroups@tools-bastion-13:~$ toolforge jobs quota
Running jobs                                  Used    Limit
--------------------------------------------  ------  -------
Total running jobs at once (Kubernetes pods)  3       16
Running one-off and cron jobs                 0       15
CPU                                           1500m   8
Memory                                        1536Mi  8Gi

Per-job limits    Limit
----------------  -------
CPU               3
Memory            6Gi

Job definitions                             Used    Limit
----------------------------------------  ------  -------
Cron jobs                                      0       50
Continuous jobs (including web services)       3       16

Enjoy!