Page MenuHomePhabricator

[SPIKE 8hrs] How does search work?
Closed, ResolvedPublicSpike

Description

Description

We would like to learn more about search prior to making improvements

User stories

As a reader or editor of Wikipedia, I would like the desktop site to feature a more prominent search functionality, so that I can find it quicker

As a reader or editor of Wikipedia, I would like to receive photos and suggested articles while I search, so I can find the article that I'm looking for (desktop search acting like mobile search)

As a product team, we would like to measure the impact on search any changes we implement will have

Acceptance criteria

Research the following questions and write up your results:

  • What is the difference between the 4 different and known implementations of search:
    • -- desktop jquery autocomplete (used by Vector,Monobook,Cologne Blue)
    • mobilefrontend SearchOverlay (used by Minerva)
    • portal (wikipedia.org)
    • OOUI (used by Structured Data team and Visual Editor when adding links)
  • What are the pros/cons and difficulty of creating yet another implementation of search.
  • What is the relative difficulty and pros/cons of bringing search functionality from one implementation to another (from mobile to desktop, for example)
  • What would it cost to make all the implementations look and function similarly?
  • How are each of these instrumented?
  • How difficult would it be to resurrect this instrumentation

Resulting report

https://www.mediawiki.org/wiki/User:JDrewniak_(WMF)/notes/Seach_widgets_at_Wikimedia

Event Timeline

ovasileva triaged this task as Medium priority.Sep 23 2019, 11:48 AM
Jdlrobson updated the task description. (Show Details)
ovasileva renamed this task from [SPIKE] How does search work? to [SPIKE 8hrs] How does search work?.Sep 25 2019, 4:40 PM

HI all, so tldr;

What is the difference between the 4 different and known implementations of search:

Biggest difference are in the underlying frameworks. Desktop one uses OOUI, the mobile one uses Mustache, the old one uses just jQuery and the portal one doesn't use any framework.

What are the pros/cons and difficulty of creating yet another implementation of search.

With any shiny new JS framework, it's not that hard. I made this one in an hour with Preact https://codepen.io/j4n/pen/PooqQLq

What is the relative difficulty and pros/cons of bringing search functionality from one implementation to another (from mobile to desktop, for example)

Bring mobile to desktop would require loading the Mustache library on each page (+8kb), the other way around would require loading OOUI on each page (+100kb)

What would it cost to make all the implementations look and function similarly?

Theres a task for that: T153417 It would require a lot of co-ordination among teams, but it would be best if we had a canonical reference design to work from.

How are each of these instrumented? & How difficult would it be to resurrect this instrumentation

Only the mobile version has dedicated instrumentation code, with Schema:MobileWebSearch and MobileWebSearchLogger.js.
This instrumentation is still functional, since the search overlay still emits the same events used in the instrumentation code.
The OOUI one could be instrumented fairly easily since it can also use global events to trigger EventLogging events.

more here: 👉 https://www.mediawiki.org/wiki/User:JDrewniak_(WMF)/notes/Seach_widgets_at_Wikimedia

Confusingly desktop does have a Search schema which is not marked as inactive. Is this instrumentation still available? https://meta.wikimedia.org/wiki/Schema_talk:Search If not, I'm guessing desktop search instrumentation got turned off but mobile search didn't? Is the desktop code still around or can we locate the bug that turned it off if it is not?

Note: We have bug in the mobile search schema. I would love if nobody is using that, that we turned that off...

Confusingly desktop does have a Search schema which is not marked as inactive. Is this instrumentation still available? https://meta.wikimedia.org/wiki/Schema_talk:Search If not, I'm guessing desktop search instrumentation got turned off but mobile search didn't? Is the desktop code still around or can we locate the bug that turned it off if it is not?

There've been no Search events for a long time.

A quick codesearch for the more esoteric fields in that schema leads you to the mediawiki.searchSuggest module, which mw.tracks at several stages during its lifecycle (see here, for example). Those analytics events are picked up and converted to SearchSatisfaction events by the [[ https://gerrit.wikimedia.org/g/mediawiki/extensions/WikimediaEvents/+/b29e3cad1532c2a8575ef9c4357696b4a962773f/modules/all/ext.wikimediaEvents.searchSatisfaction.js | ext.wikimediaEvents.searchSatisfaction ]] module defined in the WikimediaEvents extension. We're still receiving a high volume of SearchSatisfaction events.

Note: We have bug in the mobile search schema. I would love if nobody is using that, that we turned that off...

… Is there a task for the bug?

See T226164: Invalid MobileWebSearch events being logged. Thanks for digging out those details Sam!

@Jdlrobson , @phuedx thank you for further investigating the instrumentation aspects! I added some more details to the instrumentation section in the document.

@phuedx I didn't realize that searchSatisfaction uses the SearchSuggest widget.

The implications of this is that if we change the desktop search widget, or make a new one, it should remain compatible with the SearchSatisfaction schema and it should emit events at roughly the same points of interaction as the current widget.

@EBernhardson, can you confirm that the SearchSatisfaction schema is still actively used?

It might also be prudent to reconcile the MobileWebSearch and SearchSatisfaction schemas while we're at it.

SearchSatisfaction is still used and is the primary method we have for understanding user interaction with search. The search satisfaction data feeds into our AB testing, into a superset dashboard about query rewriting, etc. Basically this isn't going away anytime soon.

MBinder_WMF changed the subtype of this task from "Task" to "Spike".Oct 15 2019, 5:09 PM

Thanks for confirming @EBernhardson, one of the key learning from this task is that if we're to change the search experience on desktop as part of the Desktop Refresh project, we should consult the Search Platform team beforehand because changes to the UI can impact your key metrics, that and any new UI should maintain compatibility with the SearchSatisfaction schema.

Jdrewniak added a subscriber: Jdrewniak.