Page MenuHomePhabricator

AQS 2.0: Aqsassist and test envs. Make changes corresponding to mediawiki history reduced snapshot automation
Open, HighPublic5 Estimated Story Points

Description

Description
In order to make MW history reduced snapshot updates automatically, we need to:

  • Create an airflow job that updates the snapshot value in Cassandra's aqs.config table when the new data is ready. Done as part of T352948.
  • Modify aqsassist to include a DynamicConfig object that is able to read config properties (the snapshot) from Cassandra's aqs.config table.
  • Modify aqsassist to factor out Druid and Cassandra configuration, since now some services will need to access both of them.
  • Add the aqs.config table to Cassandra's test environment, so integration tests can use the new code.

Event Timeline

@SGupta-WMF Does this include changes to the Cassandra dev environment, so that AQS can get the snapshot value, or do we do it in another task?
Also, I imagine some changes will be needed in aqs-assist, apart from dev envs, no? Like a util to query Cassandra and get the snapshot? Is this included in this task?

WDoranWMF set the point value for this task to 5.Feb 5 2024, 1:26 PM
mforns renamed this task from AQS 2.0 : Druid test environment Make changes corresponding to mediawiki history reduced snapshot automation to AQS 2.0: Aqsassist and test envs. Make changes corresponding to mediawiki history reduced snapshot automation.Feb 26 2024, 8:21 PM
mforns updated the task description. (Show Details)