I just spent some time playing with the search API to get a sense of the feasibility of this idea with our current search capabilities. I am concerned that search results won't be good enough for this feature to be useful, and I hope we can discuss. The two main problems I see are:

• The search includes "question words" in what it's searching. So if I search "how to add a photo", it prioritizes results that include "how" and "to". I wonder if there's a way to consider those to be stop words.
• There are a lot of pages in the Wikipedia and Help namespaces that definitely do not contain helpful information. For instance, if I search "how to change picture size", I get several pages about "featured picture" candidates and "picture of the day". Maybe @Trizek-WMF's idea of using categories could help if we encouraged communities to apply a category to all pages that could be helpful to a newbie, and then only those would be included in the search.

Also -- what are our expectations around how search performs in other languages? Is it known to perform similarly?

Here are some of the results I got when searching both the Wikipedia and Help namespaces (top three results for each). All top results came from the Wikipedia space:

• "how to change picture size"
  • Wikipedia:How to improve image quality
  • Wikipedia:How to create charts for Wikipedia articles
  • Wikipedia:How to draw SVG circuits using Xcircuit
• "edit got deleted"
  • Wikipedia:So your article got deleted
  • Wikipedia:Don't delete the main page
  • Wikipedia:Lamest edit wars
• "can i cite a documentary"
  • Wikipedia:Articles for deletion/\"web documentary\
  • Wikipedia:Peer review/Lord of the Universe (documentary)/archive1
  • Wikipedia:Articles for deletion/Bloody Island (documentary)
• "text i wrote disappeared"
  • Wikipedia:WikiProject Articles for creation/Help desk/Archives/2018 February 3
  • Wikipedia:WikiProject Military history/Assessment/Battle of Ostrach
  • Wikipedia:WikiProject Articles for creation/Help desk/Archives/2015 February 5

And here are the results when just restricting to the Help namespace:

• "how to change picture size"
  • Help:Infobox picture
  • Help:Pictures
  • Help:Files
• "edit got deleted"
  • Help:Wikipedia: The Missing Manual/Editing, creating, and maintaining articles/Editing for the first time
  • Help:Wikipedia: The Missing Manual/Editing, creating, and maintaining articles/Documenting your sources
  • Help:Wikipedia: The Missing Manual/Customizing Wikipedia/Easier editing with JavaScript
• "can i cite a documentary"
  • Help:Maintenance template removal
  • (no other results)
• "text i wrote disappeared"
  • (no results)

Change 479236 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Help Panel: Search

https://gerrit.wikimedia.org/r/479236

Change 484732 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Help Panel: Bold search label for consistency with other elements

https://gerrit.wikimedia.org/r/484732

Change 484732 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Help Panel: Bold search label for consistency with other elements

https://gerrit.wikimedia.org/r/484732

The question about whether the returned results are useful (or even relevant) that @MMiller_WMF raised in his comment is complex and has no easy answers. wiki Search functionality is far from ideal in general.

@RHo - please review the following and whether they need to be fixed:
(1) There is a general mobile problem with formatting - checked on real iPhone and Android devices: no additional lines between articles on the list

(2) The excerpts from the articles returned by the search

• look too long on mobile

• refer to seemingly random places on a page

• sometime have markup or info that does not relate to the content

• do not have ellipses - the mockup has them

(continue from the previous comment)
(3) No info to a user if the search does not return any results - just a blank area

(4) If some input does not returned any results, the input fields persistently display - "waiting for auto suggestions".

• enter a search that returns no results
• start deleting the typed search one character by one - the "waiting for auto suggestions" appears and stays until a user reloads a page or open Help panel in a different article (closing the Help panel and re-open does not help).

(5) (on mobile) If a question was typed and then a search is typed, the search results are not scrollable (confirmed only on iPhone).

Thanks @Etonkovidova - please see responses inline.

! In T209301#4886723, @Etonkovidova wrote:
The question about whether the returned results are useful (or even relevant) that @MMiller_WMF raised in his comment is complex and has no easy answers. wiki Search functionality is far from ideal in general.

Per discussion with @MMiller_WMF offline, think the first step is seeing if and how often people use search when they see it available in the help panel. If it turns out to be highly sought, we can prioritize ways to improve searching help to make it actually useful; and if not then we know to put it on the back-burner.

(1) There is a general mobile problem with formatting - checked on real iPhone and Android devices: no additional lines between articles on the list

@SBisson - can we fix by adding a 1em top padding each search result?
I.e., add .mw-ge-help-panel-popup-search-search-result {padding-top:1em} to give this:

(2) The excerpts from the articles returned by the search

• look too long on mobile

@SBisson - can we limit excerpts to 100 characters?

• refer to seemingly random places on a page

@SBisson - Is there a way to change it so that the excerpt is always from the start of the page instead?

• sometime have markup or info that does not relate to the content

@SBisson - can we hide template wikitext from being shown?

• do not have ellipses - the mockup has them

@SBisson - can we please add ellipses to the excerpts. Also please update the excerpt text {color:#54595d} and {font-size:.875em}

More comments inline....

In T209301#4888661, @Etonkovidova wrote:

(continue from the previous comment)
(3) No info to a user if the search does not return any results - just a blank area

hi @SBisson - can we add a "There were no results found." message:

(4) If some input does not returned any results, the input fields persistently display - "waiting for auto suggestions".

• enter a search that returns no results
• start deleting the typed search one character by one - the "waiting for auto suggestions" appears and stays until a user reloads a page or open Help panel in a different article (closing the Help panel and re-open does not help).

I wasn't able to replicate this, but would this error be solved by the proposed addition of the null results message?

(5) (on mobile) If a question was typed and then a search is typed, the search results are not scrollable (confirmed only on iPhone).

• I'm also experiencing this bug on iPhone only (6S, OS 11.4)

Search results margin and color

This is already in place but not being applied for mobile. Will fix it.

Length and selection of excerpt

The excerpt is coming as-is from the search infrastructure. I will look if there are API options to customize it but I doubt it.

Ellipsis around excerpt

Truncation is language and context dependant. They don't make sense in all cases depending on where in the text the excerpt is coming from. We can bing it up with the search team and check why they don't have them in their search results page. I would prefer not to hardcode them in our panel.

No-results message and pending state

I will make sure the pending state goes away even if there's no results. I haven't tested it but I can see from the code that it can happen.

I will add a no-results message.

In T209301#4889443, @SBisson wrote:

Search results margin and color

This is already in place but not being applied for mobile. Will fix it.

Length and selection of excerpt

The excerpt is coming as-is from the search infrastructure. I will look if there are API options to customize it but I doubt it.

We don't currently have any options for this at the moment, but it should be possible to build out. The highlighter already falls back to returning the start of the page if it can't find a reasonable match in the content.

Ellipsis around excerpt

Truncation is language and context dependant. They don't make sense in all cases depending on where in the text the excerpt is coming from. We can bing it up with the search team and check why they don't have them in their search results page. I would prefer not to hardcode them in our panel.

It's not clear what this is, is the request for the highlighted string to be separate pieces of the content joined by elipsis? The wikitext circled is part of the content of the page, which happens to have the search words repeated quitea few times (which is why it was selected to show).

In T209301#4889163, @RHo wrote:

• sometime have markup or info that does not relate to the content

@SBisson - can we hide template wikitext from being shown?

Templates that shouldn't be included in the search results need to be excluded by the editors. Docs at https://www.mediawiki.org/wiki/Help:CirrusSearch#Exclude_content_from_the_search_index

(5) (on mobile) If a question was typed and then a search is typed, the search results are not scrollable (confirmed only on iPhone).

I'm also experiencing this bug on iPhone only (6S, OS 11.4)

I took a quick look on Safari iOS, this is kind of a weird one. Sometimes, deleting or adding a character results in the results being scrollable. Other times, it's stuck. In yet other instances, the whole window seems kind of broken in that you can scroll left/right even though there's nothing to see. I don't see any console errors and I don't see any difference in behavior between source or VE being the editor in the background when the help panel is opened.

Change 485086 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/GrowthExperiments@master] Help panel search: add a message when there is no search results

https://gerrit.wikimedia.org/r/485086

Change 485086 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Help panel search: add a message when there is no search results

https://gerrit.wikimedia.org/r/485086

In T209301#4889443, @SBisson wrote:

Search results margin and color

This is already in place but not being applied for mobile. Will fix it.

Fixed:

No-results message and pending state

I will make sure the pending state goes away even if there's no results. I haven't tested it but I can see from the code that it can happen.

I will add a no-results message.

Fixed:

In T209301#4889930, @EBernhardson wrote:

Ellipsis around excerpt

Truncation is language and context dependant. They don't make sense in all cases depending on where in the text the excerpt is coming from. We can bing it up with the search team and check why they don't have them in their search results page. I would prefer not to hardcode them in our panel.

It's not clear what this is, is the request for the highlighted string to be separate pieces of the content joined by elipsis? The wikitext circled is part of the content of the page, which happens to have the search words repeated quite a few times (which is why it was selected to show).

Hi, the request is for an ellipsis to be shown at the *end* of the excerpt on each search result (to indicate that is is a snippet of content on that result page). Ideally there would also be an ellipsis at the start of the excerpt if it is not taken from the start of the page content either.

In T209301#4896761, @RHo wrote:

In T209301#4889930, @EBernhardson wrote:

Ellipsis around excerpt

Truncation is language and context dependant. They don't make sense in all cases depending on where in the text the excerpt is coming from. We can bing it up with the search team and check why they don't have them in their search results page. I would prefer not to hardcode them in our panel.

It's not clear what this is, is the request for the highlighted string to be separate pieces of the content joined by elipsis? The wikitext circled is part of the content of the page, which happens to have the search words repeated quite a few times (which is why it was selected to show).

Hi, the request is for an ellipsis to be shown at the *end* of the excerpt on each search result (to indicate that is is a snippet of content on that result page). Ideally there would also be an ellipsis at the start of the excerpt if it is not taken from the start of the page content either.

We don't know if the snippet was taken from the beginning, middle, or end of the article. The search results page also doesn't try to add ellipsis.

@RHo What do you think we should do?

In T209301#4900311, @SBisson wrote:

We don't know if the snippet was taken from the beginning, middle, or end of the article. The search results page also doesn't try to add ellipsis.

@RHo What do you think we should do?

hi @SBisson - that's not ideal, but seeing as the excerpt would almost never end at the end of the page contents, is it possible to just always have an ellipsis to the end of the excerpt shown?

Change 486290 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/GrowthExperiments@master] Help panel: add ellipsis to search results excerpts

https://gerrit.wikimedia.org/r/486290

Change 486290 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Help panel: add ellipsis to search results excerpts

https://gerrit.wikimedia.org/r/486290

Ellipses have been added:

The search is identical to - the screenshots are from betalabs with links to point to non-existent betalabs pages, but the page title and the text excerpts are coming from actual pages in the source wiki, e.g. en.wikipedia.org

@RHo @Etonkovidova @SBisson --

@Etonkovidova, thank you for the thorough testing you did. You found some subtle but important things!

I have just gone through and done some testing. In short, I think this feature is good to go, pending these things:

• Testing from @revi on T214509. (@Urbanecm has already tested on T214520).
• Adding instrumentation (see discussion on T211166).
• Approval from @RHo (putting it back in her column now).
• Addressing/discussing the things that I have below.

• @RHo -- I'm not sure about the language, "Search FAQs". I'm not sure about it because (1) we're not really searching FAQs, and (2) I don't know if "FAQs" makes sense and translates well in every language. Do you think that language is important, or could we change it to something like, "Search for help"?

• @RHo -- your original mockups contained a button on the search results screen sayings, "Open Help in new tab". What would that button have done? I just want to make sure we didn't miss something important from the design.

• @RHo -- @Urbanecm brought up something interesting on T214520. Would it be better if the rest of the help panel didn't disappear until the user started typing in the search bar? As opposed to now -- how it disappears as soon as a user clicks into the bar?

• @SBisson -- I also encountered a lot of scrolling issues on my iPhone like @kostajh did. I don't think this is a blocker, because in all circumstances I was able to get the top three results to display (though I frequently could not scroll beyond those -- I could tell that the background page behind the overlay was scrolling instead). But could you comment on this?

• @SBisson -- just for our information, do you know how possible it is to restrict search to pages having certain categories? Our ambassadors have suggested that as a way to restrict the search to the most relevant pages in the future.
• @SBisson -- are there new messages that need to be translated?

In T209301#4907603, @MMiller_WMF wrote:

@RHo @Etonkovidova @SBisson --
@Etonkovidova, thank you for the thorough testing you did. You found some subtle but important things!
I have just gone through and done some testing. In short, I think this feature is good to go, pending these things:

• Testing from @revi on T214509. (@Urbanecm has already tested on T214520).
• Adding instrumentation (see discussion on T211166).
• Approval from @RHo (putting it back in her column now).
• Addressing/discussing the things that I have below. ---
• @RHo -- I'm not sure about the language, "Search FAQs". I'm not sure about it because (1) we're not really searching FAQs, and (2) I don't know if "FAQs" makes sense and translates well in every language. Do you think that language is important, or could we change it to something like, "Search for help"?

Yes, sorry this was just what was in the original mock since I've not had a chance to update the help panel with search.
@SBisson – can we update the text to say "Search for help" on English, with CZ and KO to translate according to whatever is suitable in their respective languages.

• @RHo -- your original mockups contained a button on the search results screen sayings, "Open Help in new tab". What would that button have done? I just want to make sure we didn't miss something important from the design.

That button is based on the same idea as the "Open article in a new tab" button when contents of a search result link appear inside the help panel filed on T211513. The idea is to enable the user to open a Help contents page in a new tab instead of continuing to search inside the help panel. We don't need it for day 1 though, esp. since it's less clear where this would go on KO and CS (whereas in EN it'd go to the Help:Contents page).

• @RHo -- @Urbanecm brought up something interesting on T214520. Would it be better if the rest of the help panel didn't disappear until the user started typing in the search bar? As opposed to now -- how it disappears as soon as a user clicks into the bar?

Yes, agree. @SBisson - sorry that was not clear in the static mocks but the contents shouldn't clear in the help panel until a search is started.

• @SBisson -- I also encountered a lot of scrolling issues on my iPhone like @kostajh did. I don't think this is a blocker, because in all circumstances I was able to get the top three results to display (though I frequently could not scroll beyond those -- I could tell that the background page behind the overlay was scrolling instead). But could you comment on this?

• @SBisson -- just for our information, do you know how possible it is to restrict search to pages having certain categories? Our ambassadors have suggested that as a way to restrict the search to the most relevant pages in the future.
• @SBisson -- are there new messages that need to be translated?

In T209301#4907603, @MMiller_WMF wrote:

[...]

• @SBisson -- I also encountered a lot of scrolling issues on my iPhone like @kostajh did. I don't think this is a blocker, because in all circumstances I was able to get the top three results to display (though I frequently could not scroll beyond those -- I could tell that the background page behind the overlay was scrolling instead). But could you comment on this?

I don't have an iOS device to test or debug so I don't know what's going on but it does sound similar to one of the issues Kosta worked on. I would suggest breaking this out into it's own task so we can work on it later (or decide not to).

• @SBisson -- just for our information, do you know how possible it is to restrict search to pages having certain categories? Our ambassadors have suggested that as a way to restrict the search to the most relevant pages in the future.

This appears to be supported by the incategory and deepcat filters. I think I can implement it fairly quickly for our ambassadors to test if you want. They would need to provide the list of relevant categories.

• @SBisson -- are there new messages that need to be translated?

CS is missing translations of the "Search FAQs" and "There are not results" messages [1]. KO has them and they appear to be deployed already but if we decide to reword "Search FAQs" it will probably invalidate the current translations. VI has nothing translated for the help panel [2].

[1] https://translatewiki.net/w/i.php?title=Special:Translate&group=ext-growthexperiments-helppanel&language=cs&filter=%21translated&action=translate

[2] https://translatewiki.net/w/i.php?title=Special:Translate&group=ext-growthexperiments-helppanel&language=vi&filter=%21translated&action=translate

Change 486483 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/GrowthExperiments@master] Help panel: reword search label

https://gerrit.wikimedia.org/r/486483

Change 486483 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Help panel: reword search label

https://gerrit.wikimedia.org/r/486483

Change 486510 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/GrowthExperiments@master] Help panel search: hide help content when searching

https://gerrit.wikimedia.org/r/486510

In T209301#4908784, @RHo wrote:

@SBisson – can we update the text to say "Search for help" on English, with CZ and KO to translate according to whatever is suitable in their respective languages.

Done and merged. Translators will see this message as "fuzzy" in TWN. It means something about it has changed and they should reevaluate whether the current translation is still ok.

Yes, agree. @SBisson - sorry that was not clear in the static mocks but the contents shouldn't clear in the help panel until a search is started.

Patch up for review.

Change 486510 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Help panel search: hide help content when searching

https://gerrit.wikimedia.org/r/486510

Change 486993 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Help Panel: Fix vertical scroll on search results in iOS

https://gerrit.wikimedia.org/r/486993

Change 486993 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Help Panel: Fix vertical scroll on search results in iOS

https://gerrit.wikimedia.org/r/486993

@RHo -- I think the final blockers for search are your review and the instrumentation (I am working on that).

@Etonkovidova -- did you want to do any additional testing on the interface? We will need to test the instrumentation once that is ready.

hi @MMiller_WMF - LGTM now on desktop and mobile web (caveat: not tested on iOS)

Mobile:

Desktop (open to view animated gif):

Change 488675 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[operations/mediawiki-config@master] GrowthExperiments: Enable search for help panel on testwiki

https://gerrit.wikimedia.org/r/488675

Change 488675 merged by jenkins-bot:
[operations/mediawiki-config@master] GrowthExperiments: Enable search for help panel on testwiki

https://gerrit.wikimedia.org/r/488675

Change 490372 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[operations/mediawiki-config@master] GrowthExperiments: Enable help panel search on kowiki and cswiki

https://gerrit.wikimedia.org/r/490372

Change 490372 merged by jenkins-bot:
[operations/mediawiki-config@master] GrowthExperiments: Enable help panel search on kowiki and cswiki

https://gerrit.wikimedia.org/r/490372

Mentioned in SAL (#wikimedia-operations) [2019-02-14T00:24:39Z] <catrope@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Enable help panel search on cswiki and kowiki (T209301) (duration: 00m 55s)