Page MenuHomePhabricator

QA needs a Quick Survey test environment on beta
Closed, ResolvedPublic

Description

QA currently cannot see survey behavior on Beta because the survey is deployed at coverage 0 to not bother other users of beta wiki.

We should deploy a safety survey test in a couple of specific pages at coverage 100 so that @Djackson-ctr and others can go to that page and see if dismissal works.

Suggested steps

  1. Create a couple of test pages locally and in https://en.wikipedia.beta.wmflabs.org/ for example to host the survey
    1. Essex's Beta Sandbox id - 265894
    2. Ellen's Beta Sandbox id - 265895
  1. Create a survey config for a fake safety survey with coverage 1 and configured to appear in the two test pages
    • Messages should go in Extension:WikimediaMessages (example)
  2. Test the survey works locally
  3. Submit a patch to operations/mediawiki-config in InitialiseSettings-labs.php adding the test survey and add other devs as reviewers to test
  4. Once the patch is code reviewed and +1d schedule it for a backport window:
    1. Read https://wikitech.wikimedia.org/wiki/Backport_windows, specifically the "Guidelines" and "How to submit a patch for backport" to know what will be requested of you.
    2. Follow the instructions and add the patch to https://wikitech.wikimedia.org/wiki/Deployments
  5. Be present for the window, and follow the instructions of the deployer
    1. Test the survey shows up in the defined pages as needed
WARNING: Changes to the surveys may take a few minutes to start being served. Last time it took 3 to 4 minutes, and they were seen earlier in incognito windows.

Acceptance criteria

  • Safety survey test is deployed to a beta cluster wiki and enabled with coverage 1 in a couple of pages
  • The links to the pages with the safety survey have been listed here and in the parent task for QA and others to notice

Event Timeline

Restricted Application added a subscriber: Aklapper. ยท View Herald TranscriptOct 19 2021, 3:21 PM

Hi, I'm working with Derrick to help address the test environment gap. Are there any special environmental needs for this kind of testing? I know that this task speaks specifically of Beta Cluster, but I am wondering if we could satisfy these needs outside of BC.

@Jrbranaa In this instance we have example surveys deployed and enabled on beta cluster but the sampling is at 0% so that they don't show up for everyone by default bugging people.

For testing the parent tasks the ideal thing is to have surveys with 100% sampling so that the tester can reliably get sampled into the survey and test behavior, which is what this change is about, but instead of enabling a survey globally we will scope it to a couple of pages only.

The complexities I think come up when testing changes that need specific custom config configurations that developers can easily try locally, but it is more complicated to do so in beta cluster.

I haven't tried patchdemo yet I have it on my list, so I don't know if it could make testing these kind of thing easier, but if there is any other environment or way to set this up it would be very helpful to know.

Let me know if we can provide more details or answer any questions.

This is the placeholder question that we could use on Beta Cluster "In the last 30 days, have you felt unsafe or uncomfortable contributing to [this wiki]?" Where [this wiki] is the name of the project, eg. Wikipedia, Wikimedia Commons, Wikidata.

And these are the placeholder answers "Yes" and "No".

319.jpg (1ร—664 px, 422 KB)

ERayfield renamed this task from QA needs a test environment on beta to QA needs a Quick Survey test environment on beta.Oct 27 2021, 5:51 PM
ERayfield claimed this task.
eigyan updated the task description. (Show Details)

Below are the code snippets required to configure the sample TST survey locally. The page ID value will need to be updated according to the page id where the survey should be displayed. In my local the pageId is 3. We will use the same "pageId" approach to displaying the test survey in Beta so we can lock the survey and prevent bugging all beta users with our sample. I have tested the below configuration with success in local. @ERayfield

LocalSettings.php

const QS_ANSWERS_MULTI_CHOICE =  [
	'ext-quicksurveys-example-internal-survey-answer-positive',
	//'ext-quicksurveys-example-internal-survey-answer-neutral',
	'ext-quicksurveys-example-internal-survey-answer-negative'
];


const QS_DEFAULTS = [
	// Who is the survey for? All fields are optional.
	'audience' => [
		'minEdits' => 0,
		'pageIds' => [3],
		// 'anons' => false,
		//'maxEdits' => 500,
		//'registrationStart' => '2021-01-01',
	//	'registrationEnd' => '2080-01-31',
		// You must have CentralNotice extension installed in order to limit audience by country
		// 'countries' => [ 'US', 'UK' ]
	],
	// The i18n key of the privacy policy text
	'privacyPolicy' => 'ext-quicksurveys-example-external-survey-privacy-policy',
	// Whether the survey is enabled
	'enabled' => true,
	// Percentage of users that will see the survey
	'coverage' => 1,
	// For each platform (desktop, mobile), which version of it is targeted
	'platforms' => [
		'desktop' => [ 'stable' ],
		'mobile' => [ 'stable' ]
	],
];

Inside $wgQuickSurveysConfig = [

ย 	[	
                // Survey name
		'name' => 'tst-safety-survey',
		// Internal or external link survey?
		'type' => 'internal',
		// The respondent can choose one answer from a list.
		'layout' => 'single-answer',
		// Survey question message key
		'question' => 'ext-quicksurveys-tst-internal-survey-question',
		// The message key of the description of the survey. Displayed immediately below the survey question.
		'description' => 'ext-quicksurveys-tst-internal-survey-description',
		// Possible answer message keys for positive, neutral, and negative
		'answers' => QS_ANSWERS_MULTI_CHOICE,
		// Label for the optional free form text answer
		//'freeformTextLabel' => 'ext-quicksurveys-example-internal-survey-freeform-text-label',
		// Whether to shuffle the display of the answers
		//'shuffleAnswersDisplay' => true,
	] + QS_DEFAULTS,

i18n - config - qqq.json

	"ext-quicksurveys-tst-internal-survey-question": "TST internal survey question",
	"ext-quicksurveys-tst-internal-survey-description": "Description of TST survey",
	"ext-quicksurveys-tst-internal-survey-answer-positive": "The positive answer for the example internal survey\n{{Identical|Yes}}",
	"ext-quicksurveys-tst-internal-survey-answer-negative": "The positive answer for the example internal survey\n{{Identical|No}}"

i18n - config - en.json

	"ext-quicksurveys-tst-internal-survey-question": "In the last 30 days, have you felt unsafe or uncomfortable contributing to {{SITENAME}}?",
	"ext-quicksurveys-tst-internal-survey-description": "",
	"ext-quicksurveys-tst-internal-survey-answer-positive": "Yes",
	"ext-quicksurveys-tst-internal-survey-answer-negative": "No"

Change 737503 had a related patch set uploaded (by EllenR; author: gerrit:ellenr):

[operations/mediawiki-config@master] Set up beta test environment for QuickSurvey

https://gerrit.wikimedia.org/r/737503

Change 737499 had a related patch set uploaded (by EllenR; author: gerrit:ellenr):

[mediawiki/extensions/WikimediaMessages@master] Update language files to support Beta QA testing for QuickSurvey

https://gerrit.wikimedia.org/r/737499

Change 737499 merged by jenkins-bot:

[mediawiki/extensions/WikimediaMessages@master] Update language files to support Beta QA testing for QuickSurvey

https://gerrit.wikimedia.org/r/737499

Not sure what to do at this point - move it to where? Believe it is done

When you have patches for your task and want others to look at it, you can move it to the Review column in the board. There is a couple of ways to do this, you can either go to the workboard and move the task, or in here in the comments, click on add action -> move on workboard:

image.png (624ร—645 px, 67 KB)

And then select the column:

image.png (723ร—1 px, 91 KB)

Then press submit.

In this case, there are still unaddressed comments, so this task is fine sitting in in progress for now, but once you have a new patchset and want new reviews, move it forward to the review column ๐Ÿ‘

@ERayfield... Is there a reason why you pushed this ticket to QA instead of to Product Sign-off?
Also I don't see any links to the safety survey listed in the ticket Description, and according to the Acceptance Criteria in the ticket Description you need to do the following: "The links to the pages with the safety survey have been listed here and in the parent task for QA and others to notice".

Since @ERayfield has left for today and is off tomorrow, I have spoken with @eigyan about the status of this ticket, and he mentioned that's its not ready to be moved out of In Progress because there are some items that still need to be completed.
So I am moving this back to In Progress on the workboard.

Once deployed, after the frontend assets are refreshed which may take a few minutes, the surveys should show up here:

I'll add them to the parent task.

Change 737503 merged by jenkins-bot:

[operations/mediawiki-config@master] Set up beta test environment for QuickSurveys

https://gerrit.wikimedia.org/r/737503

Mentioned in SAL (#wikimedia-operations) [2021-11-23T12:34:37Z] <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/InitialiseSettings-labs.php: Config: [[gerrit:737503|Set up beta test environment for QuickSurveys (T293798)]] (beta only) (duration: 00m 55s)

@eigyan can you list any QA steps that Derrick needs to perform for this task? We should always ensure there are proper QA steps when moving a task to that column

Since there was no specific QA Steps listed for this ticket, I figured it would be best to just verify I could reach the following URLs and also see the Quick Survey display on each of those URLs:

https://en.wikipedia.beta.wmflabs.org/wiki/User:Erayfield/sandbox

image.png (822ร—1 px, 96 KB)

https://en.wikipedia.beta.wmflabs.org/wiki/User:EssexIgyan/sandbox

image.png (758ร—1 px, 88 KB)

Madalina added a subscriber: Madalina.

Looks good, marking it as Resolved.