Page MenuHomePhabricator

[AQS 2.0] New Service Request device_analytics
Closed, ResolvedPublic

Description

point people @Atieno @FGoodwin @hnowlan

The repo is here:
Gerrit

Acceptance Criteria

  • Docker container is built
  • Service supports TLS and authenticator configuration for Cassandra (T328765)
  • Integrate container creation into the pipeline
  • Write a Helm chart
  • Deploy
  • Provide metrics gathered via Prometheus and dashboards to monitor and assess
  • Documentation updated

Details

SubjectRepoBranchLines +/-
operations/deployment-chartsmaster+8 -67
operations/puppetproduction+172 -0
operations/puppetproduction+584 -0
operations/puppetproduction+172 -0
operations/deployment-chartsmaster+1 -1
operations/puppetproduction+1 -1
operations/puppetproduction+1 -1
operations/puppetproduction+2 -2
operations/deployment-chartsmaster+1 -0
operations/deployment-chartsmaster+181 -0
operations/puppetproduction+52 -0
labs/privatemaster+1 -0
labs/privatemaster+4 -0
operations/puppetproduction+1 -0
operations/dnsmaster+4 -0
integration/configmaster+26 -0
generated-data-platform/aqs/device_analyticsmain+3 -2
generated-data-platform/aqs/device_analyticsmain+96 -55
Show related patches Customize query in gerrit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Which tests should be run as part of the image build process for unique_devices? Currently the tests run by make test require a second container to run and are more akin to integration tests, which is fine, but this won't fit into our pipeline currently.

Change 885295 merged by Hnowlan:

[generated-data-platform/aqs/device_analytics@main] Configuration for Blubber, type/format fixes

https://gerrit.wikimedia.org/r/885295

Change 886353 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[generated-data-platform/aqs/device_analytics@main] Fix docker image used

https://gerrit.wikimedia.org/r/886353

Change 886358 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] WIP: helmfile: add device-analytics configuration

https://gerrit.wikimedia.org/r/886358

Change 886353 merged by BPirkle:

[generated-data-platform/aqs/device_analytics@main] Fix docker image used

https://gerrit.wikimedia.org/r/886353

Change 889960 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] service, k8s: add service configuration for AQS2 service device-analytics

https://gerrit.wikimedia.org/r/889960

Change 890398 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/dns@master] Add service records for device-analytics using ingress.

https://gerrit.wikimedia.org/r/890398

Change 893416 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[integration/config@master] Add pipeline configuration for device-analytics

https://gerrit.wikimedia.org/r/893416

Changes pending once that is done:
✅ Integration into pipeline https://gerrit.wikimedia.org/r/893416

Change 893416 merged by jenkins-bot:

[integration/config@master] Add pipeline configuration for device-analytics

https://gerrit.wikimedia.org/r/893416

Change 895716 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] conftool-data: add device-analytics service

https://gerrit.wikimedia.org/r/895716

Change 890398 merged by Hnowlan:

[operations/dns@master] Add service records for device-analytics.

https://gerrit.wikimedia.org/r/890398

Change 895716 merged by Hnowlan:

[operations/puppet@production] conftool-data: add device-analytics service

https://gerrit.wikimedia.org/r/895716

Change 895824 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[labs/private@master] kubernetes: add stub values for device-analytics

https://gerrit.wikimedia.org/r/895824

Change 895824 merged by Hnowlan:

[labs/private@master] kubernetes: add stub values for device-analytics

https://gerrit.wikimedia.org/r/895824

Change 898810 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[labs/private@master] cassandra: add stub secret for device_analytics

https://gerrit.wikimedia.org/r/898810

Change 898810 merged by Hnowlan:

[labs/private@master] cassandra: add stub secret for device_analytics

https://gerrit.wikimedia.org/r/898810

Change 889960 merged by Hnowlan:

[operations/puppet@production] service, k8s: add service configuration for AQS2 service device-analytics

https://gerrit.wikimedia.org/r/889960

Change 886358 merged by jenkins-bot:

[operations/deployment-charts@master] helmfile: add device-analytics configuration, namespace

https://gerrit.wikimedia.org/r/886358

Change 898820 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] device-analytics: add missing mesh port

https://gerrit.wikimedia.org/r/898820

Change 898824 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] cassandra: fix device_analytics creation syntax

https://gerrit.wikimedia.org/r/898824

Change 898820 merged by jenkins-bot:

[operations/deployment-charts@master] device-analytics: add missing mesh port

https://gerrit.wikimedia.org/r/898820

Change 898824 merged by Hnowlan:

[operations/puppet@production] cassandra: fix device_analytics creation syntax

https://gerrit.wikimedia.org/r/898824

Change 899607 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] service: move device-analytics to lvs_setup

https://gerrit.wikimedia.org/r/899607

Change 899608 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] service: move device-analytics to production

https://gerrit.wikimedia.org/r/899608

Change 899607 merged by Hnowlan:

[operations/puppet@production] service: move device-analytics to lvs_setup

https://gerrit.wikimedia.org/r/899607

Mentioned in SAL (#wikimedia-operations) [2023-04-06T14:30:55Z] <hnowlan@cumin1001> START - Cookbook sre.loadbalancer.restart-pybal rolling-restart of pybal on P{lvs1020*,lvs2010*} and A:lvs (T320967)

Mentioned in SAL (#wikimedia-operations) [2023-04-06T14:32:14Z] <hnowlan@cumin1001> END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on P{lvs1020*,lvs2010*} and A:lvs (T320967)

Mentioned in SAL (#wikimedia-operations) [2023-04-06T14:37:56Z] <hnowlan@cumin1001> START - Cookbook sre.loadbalancer.restart-pybal rolling-restart of pybal on P{lvs1019*,lvs2009*} and A:lvs (T320967)

Mentioned in SAL (#wikimedia-operations) [2023-04-06T14:39:14Z] <hnowlan@cumin1001> END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on P{lvs1019*,lvs2009*} and A:lvs (T320967)

Change 899608 merged by Hnowlan:

[operations/puppet@production] service: move device-analytics to production

https://gerrit.wikimedia.org/r/899608

Mentioned in SAL (#wikimedia-operations) [2023-05-08T10:45:55Z] <hnowlan@cumin1001> START - Cookbook sre.loadbalancer.restart-pybal rolling-restart of pybal on P{lvs1020*,lvs2010*} and A:lvs (T320967)

Mentioned in SAL (#wikimedia-operations) [2023-05-08T10:47:03Z] <hnowlan@cumin1001> END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on P{lvs1020*,lvs2010*} and A:lvs (T320967)

Mentioned in SAL (#wikimedia-operations) [2023-05-08T10:54:55Z] <hnowlan@cumin1001> START - Cookbook sre.loadbalancer.restart-pybal rolling-restart of pybal on P{lvs1019*,lvs2009*} and A:lvs (T320967)

Mentioned in SAL (#wikimedia-operations) [2023-05-08T10:56:13Z] <hnowlan@cumin1001> END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on P{lvs1019*,lvs2009*} and A:lvs (T320967)

hnowlan updated the task description. (Show Details)

This is now deployed to the production cluster. Given the overlap we can do the final work in T335505.

Change 925859 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] device-analyics: deploy new version

https://gerrit.wikimedia.org/r/925859

Change 925859 merged by jenkins-bot:

[operations/deployment-charts@master] device-analyics: deploy new version

https://gerrit.wikimedia.org/r/925859

Change 935464 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] trafficserver: add gateway routing script, route device-analytics

https://gerrit.wikimedia.org/r/935464

Change 935464 merged by Hnowlan:

[operations/puppet@production] trafficserver: add gateway routing script, route device-analytics

https://gerrit.wikimedia.org/r/935464

Change 936509 had a related patch set uploaded (by Vgutierrez; author: Hnowlan):

[operations/puppet@production] trafficserver: add gateway routing script, route device-analytics on cp2037

https://gerrit.wikimedia.org/r/936509

Mentioned in SAL (#wikimedia-operations) [2023-07-10T09:14:12Z] <vgutierrez> depool cp2037 (debugging ATS cacheability issues) - T320967

Change 936509 merged by Vgutierrez:

[operations/puppet@production] trafficserver: add gateway routing script, route device-analytics on cp2037

https://gerrit.wikimedia.org/r/936509

Mentioned in SAL (#wikimedia-operations) [2023-07-10T11:52:12Z] <vgutierrez> repool cp2037 (debugging finished) - T320967

Change 941405 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] trafficserver: add gateway routing script, route device-analytics

https://gerrit.wikimedia.org/r/941405

Change 941405 merged by Hnowlan:

[operations/puppet@production] trafficserver: add gateway routing script, route device-analytics

https://gerrit.wikimedia.org/r/941405

Change 953982 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] device-analytics: use global AQS configuration files

https://gerrit.wikimedia.org/r/953982

Change 953982 merged by jenkins-bot:

[operations/deployment-charts@master] device-analytics: use global AQS configuration files

https://gerrit.wikimedia.org/r/953982