Analytics Query Service (AQS) is the software behind `/metrics` family of endpoints in RESTBase. Essentially it's a read-only HTTP proxy to Cassandra and Druid backends. It's currently based on a very old version of RESTBase, the codebase was initially forked from RESTBase codebase and received little updates over the years.
As a part of the goal to sunset RESTBase, AQS has to be migrated from RESTBase codebase ~~to service-template-node~~ to be in line with the rest of the services and exposed in API Gateway via envoy.
Plan:
1. ~~(Optional) Add support in service-template-node for talking to Cassandra. Since we are planning to move storage from RESTBase down to individual services, we might benefit from some shared library support. This step however is optional - perhaps we could just use Cassandra driver directly without additional abstractions. To be investigated.~~
2. ~~Finish support for Cassandra schema distribution, leftover from sessionstore project. Currently in RESTBase the schema is stored in code and can be created upon software startup. This pattern has been proven to be unsuitable for production, so the schema is actually created manually. In Kask, we've moved away from this pattern, and the schema is created only manually. Automation for schema/options distribution has never been finished for Kask, but now if we are to start migrating more service off RESTBase, we need better ways for schema distribution T220246~~
3. Rewrite AWS service ~~using service-template-node. Because AQS codebase is quite simple, this should not be too problematic. Some of the RESTBase built-in features, for example request parameter validation, will have to be reimplemented since there's no magic support for it in service-template. Upgrade to node10 only in the meantime.~~
4. Deploy AQS 2.0 on k8s. ~~Since AQS will be based on service-template-node, existing patterns for k8s deployments of node services can be reused. Cassandra connection setup could be borrowed from either of the Kask deployments.~~
5. Expose /metrics hierarchy in API Gateway. Deprecate /api/rest_v1/metrics hierarchy. Switch RESTBase to proxying requests from old AQS cluster to the new, k8s AQS cluster.
6. Eventually phase out RESTBase /metrics hierarchy.
Solving this will make us progress on multiple fronts: T198901 T210704 T262315
----
(NOTE) EDIT: The normative (non-RESTBase) bits of the AQS code are pretty trivial (and reuse isn't quite copy-paste) so let's leave open the decision of implementation language.
NOTE: This will be picked up by Platform Engineering, with support from Analytics.