Page MenuHomePhabricator

AQS 2.0 Cassandra-based services: Explore how to run QA test suite using Docker
Closed, ResolvedPublic

Description

Goals

To run the QA test suite using docker before deploying any cassandra service

Background

Our entry point is the blubber file we use to deploy the service. Using that file we should be able to create a docker image to run the service as a docker container in our laptop. Once done that, we'd need run a docker-compose project in such a way we run a project with two containers in the same network: service and cassandra. If we export the service port, we will be to run the QA test against the dockerized service that will connect to the cassandra one to fetch the data.

Tasks
  • Blubber file
    • Prepare the blubber file
  • Dockerize the service
    • Create a docker image with the service from the blubber file
  • Customize Cassandra test env to run in a docker-compose project with the service
    • Prepare a docker-compose project to be able to run the service and test env container
  • Document all the needed steps
  • Test it first in an specific service (geo-analytics in this case) and prepare the rest of them according to the steps that are explained in the documentation
Related tasks

T344374: AQS 2.0 Druid-based services: Explore how to run QA test suite using Docker

Event Timeline

Sfaci triaged this task as Medium priority.
Sfaci updated the task description. (Show Details)
Sfaci updated the task description. (Show Details)

Change 949560 had a related patch set uploaded (by Santiago Faci; author: Santiago Faci):

[generated-data-platform/aqs/geo-analytics@main] Deployment and QA test suite using docker: Added the needed blubber file for deployment and some changes to be able to create a dockerized service and run it with the cassandra test-env to be able to run QA test suite using docker

https://gerrit.wikimedia.org/r/949560

A new docker-compose config file has been added to the aqs-docker-test-env project (cassandra test env) to be able to run a new docker compose project with both containers: service and cassandra. It's currently in the following branch: https://gitlab.wikimedia.org/frankie/aqs-docker-test-env/-/tree/data-scripts-update

Sfaci updated the task description. (Show Details)

Regarding aqs-docker-test-env and geo-analytics, pending changes have been merged

Change 952273 had a related patch set uploaded (by Santiago Faci; author: Santiago Faci):

[generated-data-platform/aqs/media-analytics@main] Configured to run it as a docker container

https://gerrit.wikimedia.org/r/952273

Change 952273 merged by Santiago Faci:

[generated-data-platform/aqs/media-analytics@main] Configured to run it as a docker container

https://gerrit.wikimedia.org/r/952273

Sfaci updated the task description. (Show Details)
VirginiaPoundstone edited projects, added AQS2.0; removed AQS2.0 (Sprint 10).
VirginiaPoundstone moved this task from AQS 2.0 Backlog to Sprint 10 on the AQS2.0 board.
VirginiaPoundstone edited projects, added AQS2.0 (Sprint 10); removed AQS2.0.
Sfaci moved this task from Paused to In Process on the Data Products (Sprint 01) board.
Sfaci added a subscriber: SGupta-WMF.

Change 956912 had a related patch set uploaded (by Santiago Faci; author: Santiago Faci):

[generated-data-platform/aqs/page-analytics@main] Ready to be dockerized for deployment and QA purposes

https://gerrit.wikimedia.org/r/956912

Change 956912 merged by Santiago Faci:

[generated-data-platform/aqs/page-analytics@main] Ready to be dockerized for deployment and QA purposes

https://gerrit.wikimedia.org/r/956912

Sfaci removed a project: Patch-For-Review.
Sfaci moved this task from In Process to Paused on the Data Products (Sprint 01) board.

Just mention that device-analytics had already been deployed before we started this task and it hasn't been prepared to be dockerized yet (we don't need it right now). That task have been added to T342018: compile list of known issues for triage post AQS 2.0 launch to be addressed post AQS 2.0 launch