Page MenuHomePhabricator

[S] Fallback Behavior When Semantic / Deep Search Retrieval Fails
Open, MediumPublic3 Estimated Story Points

Assigned To
Authored By
JTannerWMF
Dec 18 2025, 8:59 PM
Referenced Files
Restricted File
Dec 19 2025, 5:39 PM
F71147230: skeleton-with-empty-state.gif
Dec 19 2025, 5:35 PM
F71147225: skeleton-with-toast.gif
Dec 19 2025, 5:35 PM
F71147222: indeterminate-progress-bar.gif
Dec 19 2025, 5:35 PM
F71147216: indeterminate-progress-bar-no-toast.gif
Dec 19 2025, 5:35 PM

Description

User Story

As a reader using Search, I want a clear and functional experience when semantic (deep) search results are unavailable,
so that I’m not confused by empty or broken sections and can continue searching or exploring other topics.

Description

Semantic (deep) retrieval may fail or be unavailable for certain queries or topics due to limited embedding coverage, unsupported content, or API failures. When this happens, the product should degrade gracefully.

At minimum, the experience should avoid showing incomplete or confusing UI. Ideally, the experience should also help the user understand what’s happening and encourage them to continue exploring by trying a different query or topic.

This task defines both:

  • a worst-case fallback (required for MVP) and
  • a best-case fallback (nice to have) when semantic retrieval does not return usable results.

Requirements [Required]

Worst-case fallback (MVP / must-have)

When semantic retrieval fails (empty results, error response, or below-confidence threshold):

  • Do not render the semantic / deep search section.
  • Do not show loading skeletons, or error UI related to semantic content.
  • Lexical search results continue to display normally.
  • No user-facing error messaging is required for MVP.
  • Log an instrumentation event indicating semantic fallback was shown.

Acceptance Criteria

  • Users never see an empty or broken semantic section.
  • Search remain fully usable even when semantic retrieval fails.
  • Failure of semantic retrieval does not block or degrade lexical results.

Nice to Have

Best-case fallback (non-blocking enhancement)

When semantic retrieval fails:

  • Replace the semantic section with a lightweight informational state.
  • Messaging should:
    • Indicate lack of semantic coverage for the topic or query (not a user error).
    • Encourage trying a different query or topic.
  • Provide one or more affordances:
    • “Try a different search”
    • “Explore related topics”
    • Editable search field with suggested alternative queries

Example copy (placeholder)

“We don’t yet have semantic coverage for this topic.
Try searching another way or explore a related subject.”

Optional Enhancements

  • Show example semantic-style queries as tappable suggestions.
  • Include a “Why am I seeing this?” affordance for transparency.

Event Timeline

I feel that the skeleton state happens before we can confirm if semantic search can happen or not, and therefore, unavoidable. Would like to double check this with engineer, and I also have a proposal for a third path that would improve user experience with a probable low lift (I think?) - video with those items for discussion so we don't forget: https://drive.google.com/file/d/1b-_4xC5nWc6mTKwBEGrh35_-5bUKc9q2/view?usp=drive_link

Could we use an alternative progress indicator using material design for the sake of the MVP? For example this one that uses Jetpack compose? That seems like the easiest route. Another possible path is show the full skeleton, if retrieval fails, remove the semantic section skeleton after a short delay, then remove the section completely I don't think we should expand the list of lexical results, I think continue to show 3 lexical results (if they exist).

I also want to point out the section below that says "run a deep search" is a nice to have so there is a world where that section doesn't exist or the copy is different like "search for pages containing...." to mirror web.

Its also note worthy that that our existing loading indicator for search is the linear progress bar and if there isn't coverage for lexical users will see "no results".

Thanks for the comments above, Jaz - yep, I am aware that our existing loading indicator is the linear progress bar. My suggestion of skeleton loading is due to the constraint that we may face regarding the latency of the semantic search. Skeleton loading states are proven to reduce perceived delays, so the user experience is smooth, minimizing the risk of negative impact on user perception and metrics (for example, users feeling that semantic search is slower and therefore disliking the experience).

My recommendations here are:

  1. Leave as is - we have a skeleton for deep search, and if it doesn't load, it is okay - this state should be fast (hopefully), this is a rare scenario users will face, and it is also common on apps that the skeleton doesn't match 100% of the upcoming UI. This is already designed and ready to go. I would like to still advocate for the toast to communicate to users that the experience isn't a full one. This encourages them to try new queries and experience the semantic search, which is aligned with our goals.
  1. If the skeleton is a heavy lift for the MVP and we are okay with the risk of perceived slowness of the semantic search experience, we can remove the skeleton and use our existing loading indicator, as you suggested. The toast is still beneficial to this scenario. I can design this one since it isn't ready yet.

Sure, if you can format the file and leave the link for the first recommendation without the toast. The second recommendation with a toast. And a best case scenario as described in the task.

Here they are:

  1. Indeterminate progress bar flow with no toast (Figma link)

indeterminate-progress-bar-no-toast.gif (1×712 px, 321 KB)

  1. Indeterminate progress bar flow with toast (Figma link)

indeterminate-progress-bar.gif (1×712 px, 365 KB)

  1. Skeleton state with toast flow (Figma link)

{F71147307}

  1. Skeleton state with empty state message flow (Figma link)

skeleton-with-empty-state.gif (1×712 px, 554 KB)

Thanks for doing this. The third file appears to be locked, you may have to edit the task and unrestrict the file for us to see it @TLessa-WMF

Thanks for letting me know! Odd, I can't reproduce that - checked the links and access...we can troubleshoot that once we meet again - worst case I can duplicate the file and try to share again.

If we implement the "nice to have", we will need to better understand the possible errors so we can communicate those states to users:

  1. When semantic retrieval fails (empty results, error response, or below-confidence threshold) - we can say something like "Something went wrong. Please try again later."
  2. If no coverage for the topic we can use the copy: "This topic isn’t supported by deep search yet.
Here are some things you can try: Rephrase your search / Try a related topic [Learn more]
  3. Any others?
JTannerWMF renamed this task from Fallback Behavior When Semantic / Deep Search Retrieval Fails to [S] Fallback Behavior When Semantic / Deep Search Retrieval Fails.Jan 6 2026, 9:58 PM
JTannerWMF triaged this task as Medium priority.Tue, Jan 27, 5:42 PM