Page MenuHomePhabricator

Implement SDS2.3 UAT findings
Open, MediumPublic2 Estimated Story Points

Description

SDS2.3 UAT discovered some interesting findings and friction points in the GB configuration experience.

  • Documentation should explicitly note that tracking key has to match code.
  • Documentation should guide users how to check allotted traffic after configuring (e.g. check API response)
  • Change RegEx guidance text in the checklist -- it is not understandable. /^[A-Za-z0-9][-_.A-Za-z0-9]{7,62}$/ Change to regular language: "Must start with a letter or number, containing only letters, numbers, hyphens, underscores, or periods. Must be 8-23 characters long." This needs to be updated in the UI and in the code which checks whether items are complete.

Open Questions:

  • Do we control tool tips in the GB UI or do they?
    • If yes, then: for traffic allotment %, explain that "100% reflects your full allotted traffic, not 100% of all site traffic."

Details

Related Changes in Gerrit:
Related Changes in GitLab:
TitleReferenceAuthorSource BranchDest Branch
Replace regex checklist text with plain languagerepos/data-engineering/test-kitchen!378cjmingT432629/regex-checklist-updatemain
Customize query in GitLab

Event Timeline

Do we control tool tips in the GB UI or do they?
If yes, then: for traffic allotment %, explain that "100% reflects your full allotted traffic, not 100% of all site traffic."

We don't control tooltips in GB UI. We already do explain this: https://wikitech.wikimedia.org/wiki/Test_Kitchen/GrowthBook_user_guide/Configuring_experiments#Step_2:_Specify_how_to_enroll_and_how_many

I also just added a warning to that section to hopefully make that even clearer.

JVanderhoop-WMF moved this task from Incoming to READY TO GROOM on the Test Kitchen board.
KReid-WMF set the point value for this task to 2.
KReid-WMF moved this task from READY TO GROOM to Backlog on the Test Kitchen board.

Change #1321824 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[operations/deployment-charts@master] Test Kitchen UI: Deploy v1.5.2 release to staging

https://gerrit.wikimedia.org/r/1321824

Change #1321827 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[operations/deployment-charts@master] Test Kitchen UI: Deploy v1.5.2 release to production

https://gerrit.wikimedia.org/r/1321827

Change #1321824 merged by jenkins-bot:

[operations/deployment-charts@master] Test Kitchen UI: Deploy v1.5.2 release to staging

https://gerrit.wikimedia.org/r/1321824

Change #1321827 merged by jenkins-bot:

[operations/deployment-charts@master] Test Kitchen UI: Deploy v1.5.2 release to production

https://gerrit.wikimedia.org/r/1321827