Page MenuHomePhabricator

[AQS 2.0] New Service Request device_analytics
Open, HighPublic

Description

point people @Atieno @FGoodwin @hnowlan

The repo is here:
Gerrit

Acceptance Criteria

  • Docker container is built
  • Service supports TLS and authenticator configuration for Cassandra (T328765)
  • Integrate container creation into the pipeline
  • Write a Helm chart
  • Deploy
  • Provide metrics gathered via Prometheus and dashboards to monitor and assess
  • Documentation updated

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Atieno updated the task description. (Show Details)
hnowlan updated the task description. (Show Details)

I'll need access to a sample configuration and any requirements for the service operation for this work to proceed fully - things like external services (other than the aqs*.eqiad.wmnet hosts for the Cassandra datastores)

Hi @hnowlan where are said sample configuration located?

Hi @hnowlan where are said sample configuration located?

I'm not sure, the developers will probably know. Most likely in the source repo, something representative of what production will look like would be most useful.

Also I've edited the repo url from Gitlab to Gerrit

Adding the blubber file should be pretty straightforward and can be done by the team if there's capacity - if not I can do it. This Blubber file will more or less suffice with some changes.

Frankie and I will do the Blubber file

Atieno renamed this task from New Service Request uniqueDevices Endpoint: AQS 2.0 to New Service Request device_analytics Endpoint: AQS 2.0.Jan 30 2023, 3:15 PM
Atieno renamed this task from New Service Request device_analytics Endpoint: AQS 2.0 to [AQS 2.0] New Service Request device_analytics.

Note: Include in deployment plan - the naming standard for phab tickets

Change 885295 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[generated-data-platform/aqs/device_analytics@main] Add pipeline configuration for Blubber

https://gerrit.wikimedia.org/r/885295

Which tests should be run as part of the image build process for unique_devices? Currently the tests run by make test require a second container to run and are more akin to integration tests, which is fine, but this won't fit into our pipeline currently.

Change 885295 merged by Hnowlan:

[generated-data-platform/aqs/device_analytics@main] Configuration for Blubber, type/format fixes

https://gerrit.wikimedia.org/r/885295

Change 886353 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[generated-data-platform/aqs/device_analytics@main] Fix docker image used

https://gerrit.wikimedia.org/r/886353

Change 886358 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] WIP: helmfile: add device-analytics configuration

https://gerrit.wikimedia.org/r/886358

Change 886353 merged by BPirkle:

[generated-data-platform/aqs/device_analytics@main] Fix docker image used

https://gerrit.wikimedia.org/r/886353

Change 889960 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] service, k8s: add service configuration for AQS2 service device-analytics

https://gerrit.wikimedia.org/r/889960

Change 890398 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/dns@master] Add service records for device-analytics using ingress.

https://gerrit.wikimedia.org/r/890398

Change 893416 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[integration/config@master] Add pipeline configuration for device-analytics

https://gerrit.wikimedia.org/r/893416

Changes pending once that is done:
✅ Integration into pipeline https://gerrit.wikimedia.org/r/893416

Change 893416 merged by jenkins-bot:

[integration/config@master] Add pipeline configuration for device-analytics

https://gerrit.wikimedia.org/r/893416

Change 895716 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] conftool-data: add device-analytics service

https://gerrit.wikimedia.org/r/895716

Change 890398 merged by Hnowlan:

[operations/dns@master] Add service records for device-analytics.

https://gerrit.wikimedia.org/r/890398

Change 895716 merged by Hnowlan:

[operations/puppet@production] conftool-data: add device-analytics service

https://gerrit.wikimedia.org/r/895716

Change 895824 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[labs/private@master] kubernetes: add stub values for device-analytics

https://gerrit.wikimedia.org/r/895824

Change 895824 merged by Hnowlan:

[labs/private@master] kubernetes: add stub values for device-analytics

https://gerrit.wikimedia.org/r/895824

Change 898810 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[labs/private@master] cassandra: add stub secret for device_analytics

https://gerrit.wikimedia.org/r/898810

Change 898810 merged by Hnowlan:

[labs/private@master] cassandra: add stub secret for device_analytics

https://gerrit.wikimedia.org/r/898810

Change 889960 merged by Hnowlan:

[operations/puppet@production] service, k8s: add service configuration for AQS2 service device-analytics

https://gerrit.wikimedia.org/r/889960

Change 886358 merged by jenkins-bot:

[operations/deployment-charts@master] helmfile: add device-analytics configuration, namespace

https://gerrit.wikimedia.org/r/886358

Change 898820 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] device-analytics: add missing mesh port

https://gerrit.wikimedia.org/r/898820

Change 898824 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] cassandra: fix device_analytics creation syntax

https://gerrit.wikimedia.org/r/898824

Change 898820 merged by jenkins-bot:

[operations/deployment-charts@master] device-analytics: add missing mesh port

https://gerrit.wikimedia.org/r/898820

Change 898824 merged by Hnowlan:

[operations/puppet@production] cassandra: fix device_analytics creation syntax

https://gerrit.wikimedia.org/r/898824

Change 899607 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] service: move device-analytics to lvs_setup

https://gerrit.wikimedia.org/r/899607

Change 899608 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] service: move device-analytics to production

https://gerrit.wikimedia.org/r/899608