Page MenuHomePhabricator

DISCUSS: Relocate Generated Data Platform (neé AQS) test/dev tables?
Open, MediumPublic

Description

The cassandra-dev cluster (and the restbase-dev cluster before it), are primarily a way of staging changes to Cassandra clusters (new versions of Cassandra, JVMs, configuration changes, etc). Over time, we've also begun using it for staging services as well. It has been useful to have a staging database for services, but it is also useful for staging Cassandra changes, since we have actual workloads to test with.

The staging versions of sessionstore and echostore use cassandra-dev, as does PCS. Perhaps we could also move aqs_testing and test_spark3_loading there as well?

It is worth noting that cassandra-dev is in codfw, and that it's significantly smaller than the production cluster (less compute and storage).

Event Timeline

Eevans triaged this task as Medium priority.Jan 26 2024, 1:20 AM

I think this is a good idea :)
The smaller size shouldn't be an issue as we should not test scalability but functions.

I think this is a good idea :)
The smaller size shouldn't be an issue as we should not test scalability but functions.

Does your tooling let you control size and throughput? Assuming we have the capacity on the link(s) connecting datacenters, and others do not object (we should probably loop in networking at least), it would great to be able to send a commensurate request throughput.

Does your tooling let you control size and throughput?

The tooling allows us to control throughput, and sizing depends on user query, therefore should be made small.
Since we don't have automated ways to test yet, manual errors could happen though.