Change #1059437 had a related patch set uploaded (by Jdlrobson; author: Jdlrobson):

[mediawiki/skins/Vector@master] Allow gadget/browser extension extensibility of empty search state

https://gerrit.wikimedia.org/r/1059437

@bwang something like the above hook is probably a good idea to support however we decide to run this experiment. This would allow us to avoid having to use MutationObserver is unpredictable ways when building out a <banner/gadget/browser extension/WikimediaEvents/insert type here> experiment.

cc T371499

In T371294#10039946, @Jdlrobson wrote:

@bwang something like the above hook is probably a good idea to support however we decide to run this experiment. This would allow us to avoid having to use MutationObserver is unpredictable ways when building out a <banner/gadget/browser extension/WikimediaEvents/insert type here> experiment.

Yes me and @Jdrewniak discussed this as well, your patch seems like the right idea to me, given we are mainly worrying about the empty state.

Me and @Jdrewniak met to discuss the list of options from part 1 of this spike, which also includes notes on pro/cons for each option.

The first consideration is whether or not a delivery mechanism is subject to the train, which would be an obstacle with the 2 week build/setup constraint. This automatically means production A/B test, production Beta feature, production interactive survey and MediaWiki Extension arent feasible.

Patchdemo is a quick and easy way to get a realistic mediawiki environment, but wouldnt be good for getting accurate quantitative data, as people wont be able to actually browse/search on it realistically. A standalone webpage has the same issues with quantitative data, except it does a worse job of mimicking mediawiki than patchdemo.

QuickSurveys and CentralNotice both arent well suited for this kind of experiment and would likely be more trouble to work around.

That leaves our number 1 and 2 candidates, browser extension and gadgets!
Browser extension is most the promising, as it has relatively minimal overhead and only requires some familiarity with chrome extension store. The main downside to it is that our audience is limited and that we cant test mobile. Gadgets allow us to reach more people, but potentially come with community concerns, and might require approval. It would also be riskier and they presumably would have to be enabled by default on a smaller wiki if we want anon users. Both browser extensions and gadgets need a workaround for languages. We could access and use mediawiki messages for both, or directly access translate wiki pages.

Jan and I also discussed the metrics, we believe most of them should already be tracked by the Search team, as the ones listed in the description are related to after the user already types into search, and is no longer seeing the empty search state. We want to confirm with the search team that these can be tracked for our experiement

Gadgets allow us to reach more people, but potentially come with community concerns, and might require approval.

Did you consider banners? It's a bit of an unconventional use of them, but it could also be used as a delivery mechanism without those same community concerns.

@Jdlrobson

Did you consider banners? It's a bit of an unconventional use of them, but it could also be used as a delivery mechanism without those same community concerns.

We did consider a banner as one of the options, but I think there are some complexities regarding the interaction of the banner and the campaign, as well as what happens when running multiple campaigns simultaneously. For example, CentralNotice can target user languages, but not wikis, so that logic has to live in the banner. Previously when we used CentralNotice to show the dark mode launch notification, we housed the 'real' banner code in the Minerva repo, and used the banner as a script loader. I'm not sure if that was a good idea or if it would be better to keep the experiment code in the on-wiki banner page itself (combined with the styles I guess?). Banner persistence might also be an issue if cookies are disabled.

These are all open questions that would need further investigation, so although CentralNotice does have potential for this use-case, the browser extension seems like a more straight-forward way of injecting code onto the page at this time. We did discuss CentralNotice during the sprint-kickoff meeting today though, e.g. if we deployed the experiment as a gadget, we could create a CentralNotice campaign that enables the experiment via clicking a banner.

Change #1059437 merged by jenkins-bot:

[mediawiki/skins/Vector@master] Allow gadget/browser extension extensibility of empty search state

https://gerrit.wikimedia.org/r/1059437