Page MenuHomePhabricator

[EPIC] Integrate QuickSurveys and CentralNotice
Open, MediumPublic

Description

This task tracks the work needed to either:

  1. enable QuickSurveys to support data served by CentralNotice (either as the only option, or an additional one, still TBD).
  2. add any missing and necessary functionality existing in QuickSurveys but not CentralNotice to the latter and sunset the former

Approach 1: support data served by CentralNotice

Full proposal documented here: https://www.mediawiki.org/wiki/Extension:QuickSurveys/Deploying_surveys

  • Determine whether to completely render surveys within CN banners or just serve the JSON data in the CN banners
  • Update QS extension to check for a CN banner and perform the appropriate action(s) (move survey or render it, depending on the above decision)
  • Remove the capability for configuration via mediawiki-config, if applicable
  • Document the process of creating a survey via CN
  • Run a test survey

Approach 2: sunset QuickSurveys

What does QuickSurveys provide that CentralNotice should not? Why maintain two extensions instead of one? QuickSurveys may provide little value over CentralNotice, is missing some features that CentralNotice has, is rarely used, and therefore the two should be considered for convergence.

QuickSurveys appears to have been used about 10 times in the last 3 years.

Surveys require a developer to both build the configuration and deploy it:

QuickSurveys config
{
	"@QuickSurveysRequireHttps": "Whether to require links to external surveys to be secure",
	"QuickSurveysRequireHttps": true,
	"QuickSurveysConfig": [
		{
		"@name": "survey name",
		"name": "internal example survey",
		"@type": "internal or external link survey",
		"type": "internal",
		"@question": "survey question message key",
		"question": "ext-quicksurveys-example-internal-survey-question",
		"@description": "The message key of the description of the survey. Displayed immediately below the survey question.",
		"description": "ext-quicksurveys-example-internal-survey-description",
		"@answers": "possible answer message keys for positive, neutral, and negative",
		"answers": [
			"ext-quicksurveys-example-internal-survey-answer-positive",
			"ext-quicksurveys-example-internal-survey-answer-neutral",
			"ext-quicksurveys-example-internal-survey-answer-negative"
		],
		"@enabled": "whether the survey is enabled",
		"enabled": true,
		"@coverage": "percentage of users that will see the survey",
		"coverage": 0.5,
		"@platforms": "for each platform (desktop, mobile), which version of it is targeted (stable, beta)",
		"platforms": {
			"desktop": ["stable"],
			"mobile": ["stable", "beta"]
		}
		},
		{
		"name": "external example survey",
		"@type": "internal or external link survey",
		"type": "external",
		"@question": "survey question message key",
		"question": "ext-quicksurveys-example-external-survey-question",
		"@description": "the i18n key of the description of the survey",
		"description": "ext-quicksurveys-example-external-survey-description",
		"@link": "external link to the survey",
		"link": "ext-quicksurveys-example-external-survey-link",
		"@instanceTokenParameterName": "parameter to add to link",
		"instanceTokenParameterName": "parameterName",
		"@privacyPolicy": "The i18n key of the privacy policy text.",
		"privacyPolicy": "ext-quicksurveys-example-external-survey-privacy-policy",
		"@enabled": "whether the survey is enabled",
		"enabled": true,
		"@coverage": "percentage of users that will see the survey",
		"coverage": 0.5,
		"@platforms": "for each platform (desktop, mobile), which version of it is targeted (stable, beta)",
		"platforms": {
			"desktop": ["stable"],
			"mobile": ["stable", "beta"]
		}
		}
	]
}

Whereas CentralNotice provides a GUI:

Open questions

  • Does QuickSurveys provide any value that CentralNotice should not? Does the regularity of its usage justify its maintenance cost?
  • What functionality would CentralNotice require to sunset QuickSurveys?

Previous discussions


Please feel free to update this task's description as necessary. This task may need to be reconsidered as an epic.

Event Timeline

jhobs raised the priority of this task from to Needs Triage.
jhobs updated the task description. (Show Details)
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptJan 28 2016, 6:53 PM

Initial questions from @Jdlrobson via email thread:

Some questions:

  • Do we have open tasks tracking the work that would be needed in CentralNotice?
  • Do we see an outcome of this work that QuickSurveys gets swallowed up by CentralNotice or depends on Central Notice (either hard dependency or soft dependency?)
  • Last time I checked CentralNotice loaded a large amount of JavaScript (more than the existing mobile site together) - how would this impact us? Is there a small chunk of code we can load given we'd have to load this code on every page view?
  • The work suggests integrating with the CentralNotice admin panel but also using CentralNotice for choosing the survey.
    • Are these two things tightly coupled as your write up suggests?
    • If the performance issue in the point above is still a problem could we safely make use of the admin panel but still maintain our own delegation mechanism?
    • This would help us with working around "A user can at max only ever see one banner (survey) from a campaign" too.
Jdlrobson set Security to None.

Another question

  • Should the functionality of QuickSurveys be folded into the CentralNotice extension and Extension:QuickSurveys cease to exist?
jhobs added a comment.EditedJan 28 2016, 7:35 PM

Another question

  • Should the functionality of QuickSurveys be folded into the CentralNotice extension and Extension:QuickSurveys cease to exist?

I can talk to that last one. I think almost all of the functionality could, and QuickSurveys could simply exist to reposition the survey on the page. CN banners always show up within the same div AFAIK, so we could use the extension simply for repositioning the survey and hiding the banner div. We could, of course, simply do this with CSS, but perhaps not without breaking page flow for accessibility purposes.

bd808 triaged this task as Medium priority.Feb 1 2016, 6:10 PM
bd808 added a subscriber: bd808.
Pcoombe added a subscriber: Pcoombe.Feb 3 2016, 7:16 PM
dr0ptp4kt renamed this task from Update QuickSurveys extension to support data served by CentralNotice to [EPIC] Update QuickSurveys extension to support data served by CentralNotice.Feb 6 2016, 3:07 PM
dr0ptp4kt added a project: Phlogiston-Category.
Jdlrobson moved this task from Backlog to Bugs on the QuickSurveys board.Jan 15 2019, 5:56 PM
Jdlrobson moved this task from Bugs to Feature requests on the QuickSurveys board.
Niedzielski renamed this task from [EPIC] Update QuickSurveys extension to support data served by CentralNotice to [EPIC] Integrate QuickSurveys and CentralNotice.Mar 1 2019, 7:51 PM
Niedzielski updated the task description. (Show Details)
Niedzielski added a subscriber: Niedzielski.

Hi all,

@Seddon is heading up the community product management of CentralNotice and I product manage the fundraising use cases. Seddon can weigh in on community priorities but I wanted to give some context.

I'm going a little high level first off. People approach us from time to time asking if CentralNotice can serve various research and testing purposes, like A/B testing features, controlling content on a page or surveys. All of the current teams and engineering leads around CentralNotice continue to be very much against expanding CN for these use cases. CN is a home grown, slightly fragile tool in desperate need of several overhauls. The current banner logic and targeting need constant attention so as not to impact performance (the performance team is very helpful and patient with us). When we test things like interactive banners and changes to how the content is displayed on wiki pages, we see major performance impacts. Adding any more functionality could really bloat CN and make untangling performance issues very difficult.

My opinions of these use cases:

  • I think there should be a survey extension that can run on its own page or be embedded in a page using some A/B testing solution. Also making this an extension would help other mediawiki users.
  • There needs to be some A/B testing solution that can test features, adjust wiki pages and serve things like surveys. I think the lack of this tool is why everyone looks at CentralNotice. We have talked to @MMiller_WMF about this and we are available to offer advice on this tool. CN could inspire a solution but we don't think it is the solution.

We conducted some in depth user research with CN admins and potential admins in 2017 [1]. This was a minor topic in interviews. We don't get requests for this that often. There really isn't any engineering capacity for CentralNotice to speak of. Fr-tech has a hard time just keeping it live and making fundraising changes. Seddon and I are making an appeal for a community engineer next year but that hire will likely work on the higher priorities from the research results [2] and any more recent issues Seddon might prioritize.

I (and I'm sure @Jseddon too) would be down to talk about this in greater detail in a meeting.

[1]
https://www.mediawiki.org/wiki/Extension:CentralNotice/Design_Research_2017

[2]
https://www.mediawiki.org/wiki/Extension:CentralNotice/Design_Research_2017/Results

I think a good person to talk to is @jlinehan, who is conducting a series of interviews right now to better understand how Audiences teams conduct instrumentation and A/B tests. CentralNotice and perspectives from Advancement and the communities might be good elements to add to the mix as the Better Use of Data effort thinks about what our future A/B testing frameworks should be like.

I (and I'm sure @Jseddon too) would be down to talk about this in greater detail in a meeting.

I'd be interested to attend. Separately, I'd also like to follow up on what kind of things you've had people ask if CN could do, go into the lessons learned running and maintaining the extension, and your thoughts about a future A/B testing solution. Gory engineering detail could be helpful. I can follow up off-ticket since this is not directly relevant to the task.

Putting a neutral hat on, I completely understand the fundraising concerns. We might also want to think about this in the other direction - improve QuickSurveys so it can eventually run banner campaigns. FWIW I don't have any opinions or recommendations either way.

Possibly, there's an opportunity to generalise the UI CentralNotice uses to setup banner campaigns so that QuickSurveys could use it. Either way we're talking about a lot of effort here and without this being a quarterly goal for either/both team I can't see it happening.