Hello @Volker_E , @Aklapper , @CKoerner_WMF , @Ladsgroup , @Jonas , @WMDE-leszek , and whoever you think could also provide input (please ping them),

We're currently trying to find out if the words we're using for the advanced search input fields (see image in the description) are comprehensive to a user.

It would be great if you could provide us some feedback on that by:

1. Looking at the terms and imagining what they do when used.

2. What search result you would expect if you'd type in the words we suggest.

3. Click the link to see if the result matches your expectation.

4. Let us know in the comments if it matched your expectations and if not, why not and what you expected to happen instead.

Thanks so much in advance. Any feedback would be appreciated :)

Hello @Volker_E , @Aklapper , @CKoerner_WMF , @Ladsgroup , @Jonas , @WMDE-leszek,

just a quick reminder asking for feedback on the keywords. We are concluding the test phase on the 20 of July so it would be great to have your feedback by Wednesday the 19th.

Looking forward to your input!

Section: The page should include…

This word
What happens if you type in...
...“Car”
...“sesame street”
...“car mustang”

I was expecting that I would get pages where all of words I put in the field are included. This seems to be what I actually got.
I was not thinking about captial/lower characters differences at all, but I noticed that all ways of spelling a word are included in the results which I find what I would expect too.

Remark: Seeing "This word" and being allowed to actually put N words seems a bit confusing. What this field stands for as "All of these words" but I understand it is 1) lengthy, and 2) possibly not the best label to see when I actually want to put a single word only.

Exactly this text
What happens if you type in…
“This is strange”
Did you assume that it searches for these words in exactly this order, in exactly this writing?

I was expecting to get pages that contain the exact phrase as in all words appear and in the exact given order. This is what I received.
I did not think about the spelling (lower/upper case) being relevant before doing a search. It seems it is not in the result, which I am not surprised or confused about.

Not this word
What happens if you type in…
Car (and in “this word” you typed “Mustang”)
Did you assume that you will get results on the plane and the horse with that name, but nothing about the car?

I was expecting articles about Mustangs but not a car. This seems to be what I got.

One of these words
What happens if you type in…
“Laptop Notebook”

I was expecting to get articles that contain either "laptop" or "notebook" or both. This was what I got. I somewhat thought that probably articles containing both words would be higher in the search results but I cannot tell if this is a case or not. This is not a very "big" assumption though, just slipped through my mind.

Section: Structure

Subpages of this page
What happens when you type in…
"wikipedia: wikiproject"
What kind of results do you expect?

I was expecting to get pages like Wikipedia:Wikiproject/XYZ, Wikipedia:Wikiproject/Foobar, etc. What I got instead is something that looks like pages that have names starting with Wikipedia:Wikiproject.

Page title contains
What happens when you type in:
"Yellow"
What kind of results did you expect

I was expecting to get pages that have title containing Yellow (in any position). This seems to be what I got.
Just after clicking the example linked I thought that if I put several words in there I would probably expect to get pages that contain all of those words in their titles, in whatever position and order. I am not sure if this is relevant here and also did not know how would I test if this assumption was meeting the actual behaviour or not.

Page in this category
“bicycles”
Did you assume that it also searches subcategories?

I didn't think about it before reading the question but I would be rather surprised/confused of it did not include subcategories as well.

Page contains this template
What happens when you type in...
"Cleanup infobox"
Do you expect it to show you all articles that have this template?

I didn't think about it before reading the question but I would probably be fine with the limitation to the main namespace initially as I see it is visible on the search result page that only Articles have been searched, and it is also easy to include other namespaces.

Source code contains...
What happens when you type in…
"thumb"
Do you expect it to show you all the articles that have “thumb” in their wikitext?

Again, as a not very active user of wikis, I didn't think about the namespace limitation beforehand. Without a specific need to search in some other namespace, I was not hugely surprised that only Article namespace was covered. And as above, it was pretty clear on the search results page, and easy to change there.

Note: text says "Source code contains" whereas the screenshot has it as "Wikitext contains". I was not asked about it but I am not sure which is better, to me they seem more or less synonyms in this context.

In T168729#3431101, @WMDE-leszek wrote:

Note: text says "Source code contains" whereas the screenshot has it as "Wikitext contains". I was not asked about it but I am not sure which is better, to me they seem more or less synonyms in this context.

I have a slight preference for "Wikitext" since "source code" could be ambiguous to some users. You can't search the program's source code, but some people also use "source code" to refer to the HTML of a website (which you can search out on the web).

In other news, some of us who work on search have subscribed to this ticket, but not answered your questionnaire because everything works like we've come to expect—we are too familiar with the underlying advanced keywords.

Hi @Charlie_WMDE - have you considered doing some direct user testing for this type of feedback? That might give you a bit more information that would be more of what a "normal person" would expect from these search queries, rather than most of the folks on this ticket that are very intimately aware of what search terms/requests/responses should be (or will be). :)

I'm happy to help out with getting something set up.

@Charlie_WMDE I don't have anything to add that hasn't already been said. I've bugged a few folks a hope more show up with additions. :)

@TJones:

I have a slight preference for "Wikitext" since "source code" could be ambiguous to some users

Yes, that makes sense. @Charlie_WMDE: What do you think?

@debt:

have you considered doing some direct user testing for this type of feedback?

We’d love to! We recently did a small test though, which covered interaction with the equivalent fields to "OR" and "AND". It turned out that it is more tricky than we thought (particularly if we want to cover many key words and their interaction), since the tasks we give users already structure their strategies of searching a lot.

have you considered doing some direct user testing for this type of feedback?

Hey @debt thanks for offering your support.

To answer the question, yes we have! We're currently testing the wording with the german-speaking community here. I think the results will do for now.

Getting some feedback on this ticket is supposed to cover a bit of the english-speaking community as well. If you know anyone who's not involved in this project but is active on en-wp and hangs around on phabricator feel free to ping them :)

@Charlie_WMDE I don't have anything to add that hasn't already been said. I've bugged a few folks a hope more show up with additions. :)

@CKoerner_WMF Thanks!

In T168729#3442775, @Jan_Dittrich wrote:

@TJones:

I have a slight preference for "Wikitext" since "source code" could be ambiguous to some users

Yes, that makes sense. @Charlie_WMDE: What do you think?

I agree. Also not sure how that got in there :| will fix it. Thanks for pointing that out @WMDE-leszek and your feedback @TJones

In T168729#3443576, @Charlie_WMDE wrote:

have you considered doing some direct user testing for this type of feedback?

Hey @debt thanks for offering your support.

To answer the question, yes we have! We're currently testing the wording with the german-speaking community here. I think the results will do for now.

Getting some feedback on this ticket is supposed to cover a bit of the english-speaking community as well. If you know anyone who's not involved in this project but is active on en-wp and hangs around on phabricator feel free to ping them :)

Cool, glad you've got good feedback! We have a usertesting.com account that is available for small batch testing done by real people that might be of interest in the future. :)