Page MenuHomePhabricator

[QA task] Community Configuration 2.0 - QA workflows in local env
Closed, ResolvedPublic

Description

Docker-based local env (GrowthExperiments extension) gives more flexibility to test Community Configuration 2.0 features.

The advantages: using local env allow
(1) to test before changes are merged to beta cluster
(2) to change config settings to explore various test cases in a more efficient way
(3) to reduce the potential risks of disrupting normal beta cluster functionality, e.g. if a Community Configuration page is deleted, blanked or some config settings are deleted or significantly changed.

As a QA, I should be able to

  • given a patch id, I easily create a docker container, so I can test features before they are merged to beta cluster
  • if during testing, my local env becomes unusable, I should be able to discard it and easily create a previosly working copy of local env

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Since the nature of CC2.0 is to be consumed by other extensions we might want to use CommunityConfigurationExample to accomplish purpose.

As a QA, I should be able to

  • given a patch id from CommunityConfiguration repository and another patch id (sha-commit) from CommunityConfigurationExample test a user story described in a Phabricator issue, eg: T354390. If additional steps, like adding some configuration, are required to test the feature, I should be able to find them in the commit message or dedicated issue comment with a mention.

Edit: given the QA step happens after code review and merging of code into master/main branches of both repositories, it should be possible to test any CC2.0 task pre-beta deployment without checking out specific commits.

I'd say that if one of both extensions makes the local environment unusable we should rise a flag and investigate the reasons. Maybe a message in Slack for shared diagnosis and then a new issue.

Etonkovidova closed this task as Resolved.EditedMar 15 2024, 5:31 PM
Etonkovidova added a subscriber: dom_walden.

Since the nature of CC2.0 is to be consumed by other extensions we might want to use CommunityConfigurationExample to accomplish purpose.

As a QA, I should be able to

  • given a patch id from CommunityConfiguration repository and another patch id (sha-commit) from CommunityConfigurationExample test a user story described in a Phabricator issue, eg: T354390. If additional steps, like adding some configuration, are required to test the feature, I should be able to find them in the commit message or dedicated issue comment with a mention.

Edit: given the QA step happens after code review and merging of code into master/main branches of both repositories, it should be possible to test any CC2.0 task pre-beta deployment without checking out specific commits.

I'd say that if one of both extensions makes the local environment unusable we should rise a flag and investigate the reasons. Maybe a message in Slack for shared diagnosis and then a new issue.

Thank you, @Sgs for the comment and for clarification on usefulness of Docker-based local env for QA testing. Since this task was created as a exploratory spike on learning how useful Docker-based env is, I think the task scope is mainly done. Below is the summary of what was accomplished/found:

(2) to change config settings to explore various test cases in a more efficient way

I'm closing the task as Resolved; for all other related QA work (problems/issues or exploratory needs) different tasks might be created.


A more elaborate summary is in Docker for QA: Workflow Cases