Page MenuHomePhabricator

Quota-requests - Increase glamwikidashboard disk sizes
Closed, ResolvedPublic

Description

Project Name:
glamwikidashboard

Type of quota increase requested:
Disk

Amount of quota increase:
3 instances of 200GB Disk and 8 Cpus per Instance

Reason:
To be able to fulfil T321702

Event Timeline

Hello. First of all, I think for the 3 instances you mention in the task description, you would also need an increase in CPU and RAM quota in addition to the disk quota. Second, could you describe what those will be used for? The linked task is very vague and the original project scope in project creation task doesn't mention any analytics related instances.

Hey Taavi. Thanks for the response. As outlined in the original request, Wikimedia Israel, which currently hosts the https://glamwikidashboard.org on AWS would like to move it to our servers to reduce costs (they are red lining it at the moment). After an investigation with with the Product Platform teams and the Tech lead from Wikimedia Israel, it was determined that the best course of action was to move it to horizon in the short term. It currently is using two 200gb AWS postgres instances, but they cannot expand/add additional GLAMs due to costs. These instances will be used to host the the databases needed to power the dashboard. Ill let @Yonatan.Bendahan (Wikimedia Isreal Tech Lead) decide if he needs more CPU and RAM.

@taavi: for a little more context, the dashboard basically keeps track of statistics about access to content provided by GLAMs. The long term plan is to make better APIs that can directly serve their needs, but for now they need to parse mediarequest dumps and store the results. This would be easier in more ways than one on our cloud infrastructure.

Are y'all happy standing up your own postgres DBs? We have a basic database-as-a-service option available as an alternative, but postgres support there is pretty spotty -- it /might/ be easier but you'd be early adopters. Some docs here: https://wikitech.wikimedia.org/wiki/Help:Adding_a_Database_to_a_Cloud_VPS_Project#Trove:_Database_as_a_Service_for_Cloud_VPS

@Andrew I will try to use the new service you shared, if it won't work I will try to spin up postgres instances my self.
does horizon support any kind of infrastructure as code?

@Yonatan.Bendahan Terraform works (mostly) fine, though it's not officially supported.

@Yonatan.Bendahan do you still need a quota adjustment here or are you still experimenting?

@Andrew I still need the quota specified, especially the disk space for the db instances. I can start with 2 db instance of 200GB each (one main and one a read replica), if it makes it easier

Confusingly, quotas for database-as-a-service are different from the quotas for block storage. I've set your dbaas quota for now, feel free to re-open this ticket if you turn out to need block storage instead.

@Andrew I created a db instance with a replica, but I don't see the replica created. what am I missing?
attaching a screenshot

Screenshot 2023-01-05 at 12.14.28.png (316×3 px, 79 KB)

@Andrew any known issue with db replication? or maybe I missing were it should appear?

Sorry for the delayed response, @Yonatan.Bendahan . Replication is not currently supported in our stack as it requires an object storage backend. We're in the process of building that but it will be a few months before the feature is supported.

thanks @Andrew. I added a contributor to the openstack project through Project Access as member, reader (added their wikimedia tech username), but they can't create new instances, how can I provide permissions for them for the project?

The permissions look right to me. I'd start by having them log out of horizon and back in (or just switch projects and switch back.) I'm hoping this is just a caching issue.

So logging out and in did the work, but now the right sidebar is not working (the lists are empty), but we can access the links by putting them in the url
is it a known bug? we were able to reproduce it with two users on Ubunutu with chrome
UPDATE - we fixed it by restarting the computer

@Andrew can we get a floating ip for our project?

can we get a floating ip for our project?

Please open a new quota request for this and provide a specfic explanation of why your project needs to have a public IPv4 address. Cloud VPS provides a reverse proxy service with TLS security to support the most common reason that projects want IPv4 connectivity.

hi everyone, I'm happy to share we finally moved the dashboard to Cloud VPS
https://glamwikidashboard.wmcloud.org/
a quick question, how we can set our own DNS record to route to that instance?