Page MenuHomePhabricator

Test the wording of the Search
Closed, ResolvedPublic

Description

Would be great to have feedback latest by 19th of July.

Based on workshops and conversations we had with Wikipedia editors, we tried to create an interface that makes the advanced functions of the search (“Keywords”) more accessible. We need your input if we got the wording right. Further down below, you will find a list of words describing a search function, and an example that describes how it works.

Please tell us if they make sense to you and if your expectations match the provided description and/or the linked example search results for the example.

Your feedback helps to improve the search function for all users, so that in future, instead of remembering many keywords the advanced search interface can be used effectively for better search results. In addition to Phabricator, we're currently gathering feedback on German Wikipedia and organised test sessions IRL.

Below you can see a screenshot of the interface.

Section: The page should include…

This word
What happens if you type in...
...“Car”
...“sesame street”
...“car mustang”

Exactly this text
What happens if you type in…
“This is strange”
Did you assume that it searches for these words in exactly this order, in exactly this writing?

Not this word
What happens if you type in…
Car (and in “this word” you typed “Mustang”)
Did you assume that you will get results on the plane and the horse with that name, but nothing about the car?

One of these words
What happens if you type in…
“Laptop Notebook”

Section: Structure

Subpages of this page
What happens when you type in…
"wikipedia: wikiproject"
What kind of results do you expect?

Page title contains
What happens when you type in:
"Yellow"
What kind of results did you expect

Page in this category
“bicycles”
Did you assume that it also searches subcategories?

Page contains this template
What happens when you type in...
"Cleanup infobox"
Do you expect it to show you all articles that have this template?

Wikitext contains...
What happens when you type in…
"thumb"
Do you expect it to show you all the articles that have “thumb” in their wikitext?

Event Timeline

Incabell created this task.Jun 23 2017, 2:29 PM
Restricted Application added projects: TCB-Team, Design. · View Herald TranscriptJun 23 2017, 2:29 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Charlie_WMDE edited subscribers, added: Charlie_WMDE; removed: Incabell.Jun 23 2017, 2:32 PM
Charlie_WMDE updated the task description. (Show Details)Jun 23 2017, 3:24 PM
Charlie_WMDE renamed this task from Test to the wording of the Search to Test the wording of the Search.Jun 28 2017, 11:34 AM
Charlie_WMDE moved this task from TWL Test to Testing on the WMDE-Design board.Jun 29 2017, 4:10 PM

Hello @Volker_E , @Aklapper , @CKoerner_WMF , @Ladsgroup , @Jonas , @WMDE-leszek , and whoever you think could also provide input (please ping them),

We're currently trying to find out if the words we're using for the advanced search input fields (see image in the description) are comprehensive to a user.

It would be great if you could provide us some feedback on that by:

  1. Looking at the terms and imagining what they do when used.
  1. What search result you would expect if you'd type in the words we suggest.
  1. Click the link to see if the result matches your expectation.
  1. Let us know in the comments if it matched your expectations and if not, why not and what you expected to happen instead.

Thanks so much in advance. Any feedback would be appreciated :)

Charlie_WMDE updated the task description. (Show Details)Jul 6 2017, 4:47 PM
James_Budday moved this task from Testing to Done on the WMDE-Design board.Jul 11 2017, 11:19 AM
Charlie_WMDE moved this task from Done to Doing on the WMDE-Design board.Jul 12 2017, 2:12 PM
Charlie_WMDE updated the task description. (Show Details)

Hello @Volker_E , @Aklapper , @CKoerner_WMF , @Ladsgroup , @Jonas , @WMDE-leszek,

just a quick reminder asking for feedback on the keywords. We are concluding the test phase on the 20 of July so it would be great to have your feedback by Wednesday the 19th.

Looking forward to your input!

Section: The page should include…
This word
What happens if you type in...
...“Car”
...“sesame street”
...“car mustang”

I was expecting that I would get pages where all of words I put in the field are included. This seems to be what I actually got.
I was not thinking about captial/lower characters differences at all, but I noticed that all ways of spelling a word are included in the results which I find what I would expect too.

Remark: Seeing "This word" and being allowed to actually put N words seems a bit confusing. What this field stands for as "All of these words" but I understand it is 1) lengthy, and 2) possibly not the best label to see when I actually want to put a single word only.

Exactly this text
What happens if you type in…
“This is strange”
Did you assume that it searches for these words in exactly this order, in exactly this writing?

I was expecting to get pages that contain the exact phrase as in all words appear and in the exact given order. This is what I received.
I did not think about the spelling (lower/upper case) being relevant before doing a search. It seems it is not in the result, which I am not surprised or confused about.

Not this word
What happens if you type in…
Car (and in “this word” you typed “Mustang”)
Did you assume that you will get results on the plane and the horse with that name, but nothing about the car?

I was expecting articles about Mustangs but not a car. This seems to be what I got.

One of these words
What happens if you type in…
“Laptop Notebook”

I was expecting to get articles that contain either "laptop" or "notebook" or both. This was what I got. I somewhat thought that probably articles containing both words would be higher in the search results but I cannot tell if this is a case or not. This is not a very "big" assumption though, just slipped through my mind.

Section: Structure
Subpages of this page
What happens when you type in…
"wikipedia: wikiproject"
What kind of results do you expect?

I was expecting to get pages like Wikipedia:Wikiproject/XYZ, Wikipedia:Wikiproject/Foobar, etc. What I got instead is something that looks like pages that have names starting with Wikipedia:Wikiproject.

Page title contains
What happens when you type in:
"Yellow"
What kind of results did you expect

I was expecting to get pages that have title containing Yellow (in any position). This seems to be what I got.
Just after clicking the example linked I thought that if I put several words in there I would probably expect to get pages that contain all of those words in their titles, in whatever position and order. I am not sure if this is relevant here and also did not know how would I test if this assumption was meeting the actual behaviour or not.

Page in this category
“bicycles”
Did you assume that it also searches subcategories?

I didn't think about it before reading the question but I would be rather surprised/confused of it did not include subcategories as well.

Page contains this template
What happens when you type in...
"Cleanup infobox"
Do you expect it to show you all articles that have this template?

I didn't think about it before reading the question but I would probably be fine with the limitation to the main namespace initially as I see it is visible on the search result page that only Articles have been searched, and it is also easy to include other namespaces.

Source code contains...
What happens when you type in…
"thumb"
Do you expect it to show you all the articles that have “thumb” in their wikitext?

Again, as a not very active user of wikis, I didn't think about the namespace limitation beforehand. Without a specific need to search in some other namespace, I was not hugely surprised that only Article namespace was covered. And as above, it was pretty clear on the search results page, and easy to change there.

Note: text says "Source code contains" whereas the screenshot has it as "Wikitext contains". I was not asked about it but I am not sure which is better, to me they seem more or less synonyms in this context.

mpopov added a subscriber: mpopov.Jul 12 2017, 10:49 PM
TJones added a subscriber: TJones.Jul 13 2017, 3:31 PM

Note: text says "Source code contains" whereas the screenshot has it as "Wikitext contains". I was not asked about it but I am not sure which is better, to me they seem more or less synonyms in this context.

I have a slight preference for "Wikitext" since "source code" could be ambiguous to some users. You can't search the program's source code, but some people also use "source code" to refer to the HTML of a website (which you can search out on the web).

In other news, some of us who work on search have subscribed to this ticket, but not answered your questionnaire because everything works like we've come to expect—we are too familiar with the underlying advanced keywords.

debt added a subscriber: debt.Jul 13 2017, 3:48 PM

Hi @Charlie_WMDE - have you considered doing some direct user testing for this type of feedback? That might give you a bit more information that would be more of what a "normal person" would expect from these search queries, rather than most of the folks on this ticket that are very intimately aware of what search terms/requests/responses should be (or will be). :)

I'm happy to help out with getting something set up.

@Charlie_WMDE I don't have anything to add that hasn't already been said. I've bugged a few folks a hope more show up with additions. :)

dchen added a subscriber: dchen.Jul 13 2017, 11:36 PM
Bmueller updated the task description. (Show Details)Jul 14 2017, 8:37 AM

@TJones:

I have a slight preference for "Wikitext" since "source code" could be ambiguous to some users

Yes, that makes sense. @Charlie_WMDE: What do you think?

@debt:

have you considered doing some direct user testing for this type of feedback?

We’d love to! We recently did a small test though, which covered interaction with the equivalent fields to "OR" and "AND". It turned out that it is more tricky than we thought (particularly if we want to cover many key words and their interaction), since the tasks we give users already structure their strategies of searching a lot.

have you considered doing some direct user testing for this type of feedback?

Hey @debt thanks for offering your support.

To answer the question, yes we have! We're currently testing the wording with the german-speaking community here. I think the results will do for now.

Getting some feedback on this ticket is supposed to cover a bit of the english-speaking community as well. If you know anyone who's not involved in this project but is active on en-wp and hangs around on phabricator feel free to ping them :)

Charlie_WMDE added a comment.EditedJul 17 2017, 1:53 PM

@Charlie_WMDE I don't have anything to add that hasn't already been said. I've bugged a few folks a hope more show up with additions. :)

@CKoerner_WMF Thanks!

@TJones:

I have a slight preference for "Wikitext" since "source code" could be ambiguous to some users

Yes, that makes sense. @Charlie_WMDE: What do you think?

I agree. Also not sure how that got in there :| will fix it. Thanks for pointing that out @WMDE-leszek and your feedback @TJones

Charlie_WMDE updated the task description. (Show Details)Jul 17 2017, 1:56 PM
debt added a comment.Jul 17 2017, 5:20 PM

have you considered doing some direct user testing for this type of feedback?

Hey @debt thanks for offering your support.
To answer the question, yes we have! We're currently testing the wording with the german-speaking community here. I think the results will do for now.
Getting some feedback on this ticket is supposed to cover a bit of the english-speaking community as well. If you know anyone who's not involved in this project but is active on en-wp and hangs around on phabricator feel free to ping them :)

Cool, glad you've got good feedback! We have a usertesting.com account that is available for small batch testing done by real people that might be of interest in the future. :)

Charlie_WMDE moved this task from Doing to Done on the WMDE-Design board.Nov 13 2017, 12:02 PM
Lea_WMDE closed this task as Resolved.Feb 15 2018, 3:30 PM
Lea_WMDE claimed this task.