Page MenuHomePhabricator
Create Task
Maniphest T320967

[AQS 2.0] New Service Request device_analytics
Open, HighPublic
Actions

Description

point people @Atieno @FGoodwin @hnowlan

The repo is here:
Gerrit

Acceptance Criteria

  • Docker container is built
  • Service supports TLS and authenticator configuration for Cassandra (T328765)
  • Integrate container creation into the pipeline
  • Write a Helm chart
  • Deploy
  • Provide metrics gathered via Prometheus and dashboards to monitor and assess
  • Documentation updated

Details

Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 17 2022, 2:33 PM
Atieno updated the task description. (Show Details)Oct 17 2022, 2:34 PM
Atieno updated the task description. (Show Details)
Atieno claimed this task.Oct 17 2022, 2:39 PM
Atieno added a project: Platform Team Workboards (Platform Engineering Reliability).
Atieno moved this task from Backlog to Doing on the Platform Team Workboards (Platform Engineering Reliability) board.
Atieno updated the task description. (Show Details)Oct 17 2022, 3:07 PM
hnowlan claimed this task.Oct 17 2022, 3:36 PM
hnowlan updated the task description. (Show Details)
hnowlan added a subscriber: BPirkle.Oct 17 2022, 4:03 PM
hnowlan added a comment.Oct 17 2022, 4:13 PM

I'll need access to a sample configuration and any requirements for the service operation for this work to proceed fully - things like external services (other than the aqs*.eqiad.wmnet hosts for the Cassandra datastores)

Atieno added a comment.Oct 21 2022, 5:03 PM

Hi @hnowlan where are said sample configuration located?

hnowlan added a comment.Oct 24 2022, 9:48 AM
In T320967#8335921, @Atieno wrote:

Hi @hnowlan where are said sample configuration located?

I'm not sure, the developers will probably know. Most likely in the source repo, something representative of what production will look like would be most useful.

hnowlan added a project: serviceops.Oct 24 2022, 3:51 PM
hnowlan moved this task from Doing to Blocked on the Platform Team Workboards (Platform Engineering Reliability) board.Oct 26 2022, 11:19 AM
VirginiaPoundstone moved this task from Backlog to QA/Review on the API Platform board.Oct 26 2022, 7:40 PM
VirginiaPoundstone edited projects, added API Platform (Sprint 01); removed API Platform.Oct 26 2022, 7:43 PM
akosiaris added a subscriber: akosiaris.Oct 31 2022, 3:25 PM
BPirkle mentioned this in T322152: Route external requests to RESTbase via API gateway.Nov 1 2022, 9:54 PM
Atieno closed subtask T320983: Move uniqueDevices service repo from Gitlab to Gerrit as Resolved.Nov 23 2022, 2:31 PM
JArguello-WMF edited projects, added API Platform (Sprint 02); removed API Platform (Sprint 01).Dec 1 2022, 3:11 PM
VirginiaPoundstone triaged this task as High priority.Dec 3 2022, 7:55 AM
akosiaris edited projects, added serviceops-radar; removed serviceops.Dec 5 2022, 4:40 PM
JArguello-WMF moved this task from Sprint 02 to Sprint 03 on the API Platform board.Jan 10 2023, 6:22 PM
JArguello-WMF edited projects, added API Platform (Sprint 03); removed API Platform (Sprint 02).
Atieno closed subtask T321575: Add uniqueDevices configuration support for multiple cassandra hosts as Resolved.Jan 30 2023, 12:08 PM
Atieno added a comment.Jan 30 2023, 12:11 PM

Hi @hnowlan this is merged

Atieno updated the task description. (Show Details)Jan 30 2023, 12:16 PM

Also I've edited the repo url from Gitlab to Gerrit

hnowlan added a comment.Jan 30 2023, 1:00 PM

Adding the blubber file should be pretty straightforward and can be done by the team if there's capacity - if not I can do it. This Blubber file will more or less suffice with some changes.

Atieno added a comment.Jan 30 2023, 3:10 PM

Frankie and I will do the Blubber file

Atieno renamed this task from New Service Request uniqueDevices Endpoint: AQS 2.0 to New Service Request device_analytics Endpoint: AQS 2.0.Jan 30 2023, 3:15 PM
Atieno renamed this task from New Service Request device_analytics Endpoint: AQS 2.0 to [AQS 2.0] New Service Request device_analytics.

Note: Include in deployment plan - the naming standard for phab tickets

gerritbot added a comment.Jan 31 2023, 11:31 AM

Change 885295 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[generated-data-platform/aqs/device_analytics@main] Add pipeline configuration for Blubber

https://gerrit.wikimedia.org/r/885295

gerritbot added a project: Patch-For-Review.Jan 31 2023, 11:31 AM
hnowlan added a comment.Jan 31 2023, 11:50 AM

Which tests should be run as part of the image build process for unique_devices? Currently the tests run by make test require a second container to run and are more akin to integration tests, which is fine, but this won't fit into our pipeline currently.

JArguello-WMF moved this task from Next Up to In Progress on the API Platform (Sprint 03) board.Jan 31 2023, 2:53 PM
gerritbot added a comment.Jan 31 2023, 5:58 PM

Change 885295 merged by Hnowlan:

[generated-data-platform/aqs/device_analytics@main] Configuration for Blubber, type/format fixes

https://gerrit.wikimedia.org/r/885295

hnowlan mentioned this in rGADAf7675d4d0d9e: Configuration for Blubber, type/format fixes.Jan 31 2023, 5:59 PM
Maintenance_bot removed a project: Patch-For-Review.Jan 31 2023, 6:31 PM
JArguello-WMF edited projects, added API Platform (Sprint 05); removed API Platform (Sprint 03).Feb 1 2023, 7:28 PM
JArguello-WMF moved this task from Next Up to In Progress on the API Platform (Sprint 05) board.
gerritbot added a comment.Feb 3 2023, 12:33 PM

Change 886353 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[generated-data-platform/aqs/device_analytics@main] Fix docker image used

https://gerrit.wikimedia.org/r/886353

gerritbot added a project: Patch-For-Review.Feb 3 2023, 12:33 PM
hnowlan updated the task description. (Show Details)Feb 3 2023, 12:51 PM
gerritbot added a comment.Feb 3 2023, 1:29 PM

Change 886358 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] WIP: helmfile: add device-analytics configuration

https://gerrit.wikimedia.org/r/886358

gerritbot added a comment.Feb 9 2023, 4:36 PM

Change 886353 merged by BPirkle:

[generated-data-platform/aqs/device_analytics@main] Fix docker image used

https://gerrit.wikimedia.org/r/886353

hnowlan mentioned this in rGADAdc36bb9b83da: Fix docker image used.Feb 9 2023, 4:37 PM
gerritbot added a comment.Feb 17 2023, 11:04 AM

Change 889960 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] service, k8s: add service configuration for AQS2 service device-analytics

https://gerrit.wikimedia.org/r/889960

gerritbot added a comment.Feb 20 2023, 11:09 AM

Change 890398 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/dns@master] Add service records for device-analytics using ingress.

https://gerrit.wikimedia.org/r/890398

gerritbot added a comment.Mar 1 2023, 11:12 AM

Change 893416 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[integration/config@master] Add pipeline configuration for device-analytics

https://gerrit.wikimedia.org/r/893416

hnowlan added a comment.EditedMar 1 2023, 12:18 PM

Changes pending once that is done:
✅ Integration into pipeline https://gerrit.wikimedia.org/r/893416

gerritbot added a comment.Mar 1 2023, 3:55 PM

Change 893416 merged by jenkins-bot:

[integration/config@master] Add pipeline configuration for device-analytics

https://gerrit.wikimedia.org/r/893416

Stashbot added a comment.Mar 1 2023, 3:55 PM

Mentioned in SAL (#wikimedia-releng) [2023-03-01T15:55:52Z] <hashar> Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/893416 | T320967

Stashbot added a comment.Mar 1 2023, 5:37 PM

Mentioned in SAL (#wikimedia-releng) [2023-03-01T17:37:36Z] <hashar> Deploying device-analytics-* jobs for for https://gerrit.wikimedia.org/r/c/integration/config/+/893416 | T320967

hnowlan updated the task description. (Show Details)Mar 1 2023, 6:03 PM
hnowlan moved this task from Blocked to In review on the Platform Team Workboards (Platform Engineering Reliability) board.Thu, Mar 2, 4:37 PM
gerritbot added a comment.Wed, Mar 8, 11:20 AM

Change 895716 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] conftool-data: add device-analytics service

https://gerrit.wikimedia.org/r/895716

gerritbot added a comment.Wed, Mar 8, 12:07 PM

Change 890398 merged by Hnowlan:

[operations/dns@master] Add service records for device-analytics.

https://gerrit.wikimedia.org/r/890398

gerritbot added a comment.Wed, Mar 8, 12:26 PM

Change 895716 merged by Hnowlan:

[operations/puppet@production] conftool-data: add device-analytics service

https://gerrit.wikimedia.org/r/895716

gerritbot added a comment.Wed, Mar 8, 4:00 PM

Change 895824 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[labs/private@master] kubernetes: add stub values for device-analytics

https://gerrit.wikimedia.org/r/895824

gerritbot added a comment.Wed, Mar 8, 4:06 PM

Change 895824 merged by Hnowlan:

[labs/private@master] kubernetes: add stub values for device-analytics

https://gerrit.wikimedia.org/r/895824

hnowlan mentioned this in rLPRIc5ee2220a094: kubernetes: add stub values for device-analytics.Wed, Mar 8, 4:07 PM
JArguello-WMF moved this task from Sprint 05 to Sprint 06 on the API Platform board.Tue, Mar 14, 3:02 PM
JArguello-WMF edited projects, added API Platform (Sprint 06); removed API Platform (Sprint 05).
JArguello-WMF moved this task from Next Up to In Progress on the API Platform (Sprint 06) board.
gerritbot added a comment.Tue, Mar 14, 5:14 PM

Change 898810 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[labs/private@master] cassandra: add stub secret for device_analytics

https://gerrit.wikimedia.org/r/898810

gerritbot added a comment.Tue, Mar 14, 5:17 PM

Change 898810 merged by Hnowlan:

[labs/private@master] cassandra: add stub secret for device_analytics

https://gerrit.wikimedia.org/r/898810

gerritbot added a comment.Tue, Mar 14, 5:35 PM

Change 889960 merged by Hnowlan:

[operations/puppet@production] service, k8s: add service configuration for AQS2 service device-analytics

https://gerrit.wikimedia.org/r/889960

hnowlan mentioned this in rLPRIf0f95c7bd43f: cassandra: add stub secret for device_analytics.Tue, Mar 14, 5:35 PM
gerritbot added a comment.Tue, Mar 14, 5:53 PM

Change 886358 merged by jenkins-bot:

[operations/deployment-charts@master] helmfile: add device-analytics configuration, namespace

https://gerrit.wikimedia.org/r/886358

Maintenance_bot removed a project: Patch-For-Review.Tue, Mar 14, 6:11 PM
gerritbot added a comment.Tue, Mar 14, 6:13 PM

Change 898820 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] device-analytics: add missing mesh port

https://gerrit.wikimedia.org/r/898820

gerritbot added a project: Patch-For-Review.Tue, Mar 14, 6:13 PM
gerritbot added a comment.Tue, Mar 14, 6:28 PM

Change 898824 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] cassandra: fix device_analytics creation syntax

https://gerrit.wikimedia.org/r/898824

gerritbot added a comment.Wed, Mar 15, 10:27 AM

Change 898820 merged by jenkins-bot:

[operations/deployment-charts@master] device-analytics: add missing mesh port

https://gerrit.wikimedia.org/r/898820

gerritbot added a comment.Wed, Mar 15, 11:17 AM

Change 898824 merged by Hnowlan:

[operations/puppet@production] cassandra: fix device_analytics creation syntax

https://gerrit.wikimedia.org/r/898824

Maintenance_bot removed a project: Patch-For-Review.Wed, Mar 15, 11:30 AM
hnowlan updated the task description. (Show Details)Wed, Mar 15, 12:35 PM
gerritbot added a comment.Wed, Mar 15, 12:43 PM

Change 899607 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] service: move device-analytics to lvs_setup

https://gerrit.wikimedia.org/r/899607

gerritbot added a project: Patch-For-Review.Wed, Mar 15, 12:43 PM

Change 899608 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] service: move device-analytics to production

https://gerrit.wikimedia.org/r/899608

JArguello-WMF moved this task from In Progress to Blocked/Paused on the API Platform (Sprint 06) board.Thu, Mar 23, 6:46 PM
hnowlan added a subscriber: JArguello-WMF.Wed, Mar 29, 4:31 PM

I plan on moving this along this week @JArguello-WMF

Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL