Page MenuHomePhabricator

[Design spike] Should we continue to collapse sections by default?
Closed, ResolvedPublic

Description

With the migration from the legacy parser to parsoid and following T359001 we should reassess decisions made 10 years ago to see if they still apply before jumping into possibly expensive code implementation efforts.

Question: Should mobile continue to collapse sections by default?

Sign off steps

  • Create implementation ticket.

Event Timeline

ovasileva added a project: Design.
ovasileva moved this task from Incoming to Groomed on the Web-Team-Backlog-Archived board.

Tl;dr
My answer to whether we should continue having mobile article sections closed by default is a qualified no with 2 caveats:

  1. We need to learn more about open/closed reading behaviours before making a final decision
  2. Just opening all the sections by default won't give us an ideal reading experience either.

Here's how I got there...

I came across some old research in my desk research for next fiscal year's "content discovery" OKR. It found that "On 60% of mobile pageviews, not a single section is opened, i.e. the reader only looks at the introduction." We also know from this study and others that readers use infoboxes and TOCs as important navigation affordances on desktop. The team ran an A/B test of open vs. closed sections and found that open sections readers tend to read more, tend to engage with more content further down the page, and spend less time navigating. The team did 2 small qualitative studies (n = 7, n = 12) and found that their participants preferred the closed section reading experience overall, but also wanted some way to "zoom in and out" of an article via ToC, sticky headers, or some combination of the two (prototypes). At the time, design recommended sticky headers in Minerva, but those were never implemented, and the small scale qualitative studies outweighed the quantitative analysis.

Also, a recent bug found that the implementation of the section collapsing "hides" that content from some programatic readers like the Safari voice to text feature (see: T356091).

Just on principle, hiding article content is probably not a good idea.

Assumptions:

  • Needing to tap a section to open it introduces higher cognitive load to read the content than just scrolling to it.
  • Some readers want to view content that is currently hidden by collapsed article sections and do not open them because doing so requires more work.
  • If sections were expanded by default, some of those readers will click on links in the content that is easier to access.
  • The relevance and quality of a section title as a navigation signpost summarizing the content within that section varies widely.
  • Giving easier access to more content in articles will improve "content performance"
  • An effective mobile ToC will improve content performance.

Proposal:

  • Review our clickstream data and see whether this trend has changed.
    • How often do readers open sections on Minerva?
    • Which sections tend to get opened?
    • How many sections tend to get opened?
    • How does article length impact these behaviours?
    • How does font-size preference influence these behaviours?
    • How does language influence these behaviours?
  • Based on this analysis, formulate a hypothesis about how much more readers may read and how many more internal referrals may come from articles where we have sections expanded by default.
  • We could also look at readers who have sections expanded by default in their preferences and see how different their reading behaviour is from people with the default, although that comparison probably isn't apples to apples.
  • Run an A/B test to see if expanding article sections by default changes reading behaviours
  • Design, test, and implement a table of contents for Minerva.

Open questions

  • Why doesn't Minerva have a ToC? Did it ever have one? If so, what was it like and why was it removed?
  • We might want to collapse some sections like references, citations, and external links.
  • Does the auto-collapse feature impact pages other than articles?
JScherer-WMF added a subscriber: ovasileva.

Created follow-up tickets:

  • Initial data analysis T366020
  • A/B test T366024
  • Design in-article navigation affordances for Minerva (ToC, sticky header) T366026

Moving to sign off for @ovasileva and @Jdlrobson

JScherer-WMF renamed this task from [Design Question] Should we continue to collapse sections by default in Parsoid? to [Design spike] Should we continue to collapse sections by default in Parsoid?.May 28 2024, 5:48 PM
JScherer-WMF reassigned this task from JScherer-WMF to ovasileva.
JScherer-WMF subscribed.

The next steps here make sense to me with the caveat this sounds like it could be a lot of work. It's possible just keeping the status quo might be the best way to approach the work relating to the parser migration depending on the timeline there, but that also doesn't feel like the most appealing next step.

I am writing a digest email for Olga for when she gets back and will include this in that.

opening this back up because it's pertinent to FY24/5 work and so that @ovasileva has a chance to review it.

JScherer-WMF renamed this task from [Design spike] Should we continue to collapse sections by default in Parsoid? to [Design spike] Should we continue to collapse sections by default ~~in Parsoid?~~.Jun 3 2024, 8:50 PM
JScherer-WMF renamed this task from [Design spike] Should we continue to collapse sections by default ~~in Parsoid?~~ to [Design spike] Should we continue to collapse sections by default?.

Thank you for the in-depth analysis here @JScherer-WMF! I think the next steps sound good to me as well, but I share @Jdlrobson's concern around potentially adding a lot of work here in the instrumentation. @Jdlrobson - do you know if we have any existing instrumentation for any of the questions below? I've bolded the ones that I think would be of specific priority:

How often do readers open sections on Minerva?
Which sections tend to get opened?
How many sections tend to get opened?
How does article length impact these behaviours?
How does font-size preference influence these behaviours?
How does language influence these behaviours? (i.e. are we able to get some of this data per wiki)

I wonder if another potential next step would be to use the previous data (which imo showed that a redesign of the toc/section collapsing is necessary) to commit to a redesign and use the metrics above to establish a baseline as a part of that. I think we have enough data here to commit and the navigation concern is high enough in the case of uncollapsing that we can skip the fully uncollapsed state altogether.

@JScherer-WMF - what do you think?

Thank you for the in-depth analysis here @JScherer-WMF! I think the next steps sound good to me as well, but I share @Jdlrobson's concern around potentially adding a lot of work here in the instrumentation. @Jdlrobson - do you know if we have any existing instrumentation for any of the questions below? I've bolded the ones that I think would be of specific priority:

How often do readers open sections on Minerva?
Which sections tend to get opened?
How many sections tend to get opened?
How does article length impact these behaviours?
How does font-size preference influence these behaviours?
How does language influence these behaviours? (i.e. are we able to get some of this data per wiki)

I wonder if another potential next step would be to use the previous data (which imo showed that a redesign of the toc/section collapsing is necessary) to commit to a redesign and use the metrics above to establish a baseline as a part of that. I think we have enough data here to commit and the navigation concern is high enough in the case of uncollapsing that we can skip the fully uncollapsed state altogether.

@JScherer-WMF - what do you think?

Thanks for this @ovasileva. I agree, we should look at having some sections closed and most of the sections open by default. And that we need to get a minerva ToC in order. I'm adding our Content Discovery tag so we can find this again. I think the tickets I created when I did the analysis cover the data and design needs (T366026).