Page MenuHomePhabricator

BASELINE: How do people use Tools and Appearance menus while editing?
Closed, ResolvedPublic

Description

This task involves the work of establishing a baseline understanding of how and who is using the Tools and Appearance menus while using the visual editor.

Learning about how people use these menus will enable us – the Editing and Web Teams – to confirm the following assumption we're holding: People who have published ≤100 cumulative edits will not be disrupted by losing access to the Tools and Appearance menus while editing with the visual editor on desktop.

Research questions

Research questionMetric(s)
Who is/is not using the "Tools" and "Appearance" menus?1. What percentage of people elect to hide the Tools and/or Appearance menus? 2. Of the people who published ≥1 unreverted edit in the main namespace, what percentage of them used the Tools and/or Appearance menu while editing using the visual editor (desktop)? 3. Of all the unreverted edits people make in the main namespace using the visual editor, what proportion of edits involved someone using the Tools and/or Appearance menus?
How heavily do people depend on the "Tools" and "Appearance" menus while editing?What percentage of editing sessions in the main namespace that result in an unreverted published edit (broken out by experience level) do people use the Tools and/or Appearance menu within?
NOTE: each of the metrics will be limited to people who have made ≤100 cumulative edits as they are the people impacted by this change.

Decision(s) to be made

Open questions

  • 1. What – if any – new instrumentation would need to be added for us to be able to answer the questions above?
    • Per T378036#10304363, no additional instrumentation is needed to conduct the analysis scoped in this task.

Background

This investigation is prompted by a desire for two things to be true for volunteers:

  • They feel like they have sufficient space (width) to make changes
  • They are able to see Edit Checks in close proximity to the content they're related to without any part of the editable content being obfuscated

References

  • We recently conducted a similar "feature use" analysis of Citoid in T368988.

Event Timeline

ppelberg updated the task description. (Show Details)
ppelberg added a subscriber: ovasileva.
ppelberg renamed this task from [SPIKE] How – if at all – do people use Tools and Appearance menus while editing? to BASELINE: How do people use Tools and Appearance menus while editing?.Oct 31 2024, 11:07 PM
ppelberg updated the task description. (Show Details)
ppelberg updated the task description. (Show Details)
MNeisler triaged this task as Medium priority.Nov 8 2024, 3:33 PM

I've confirmed instrumentation is available in event.mediawiki_web_ui_actions to calculate the metrics identified for this task. This includes ability to track click to the "hide" button and clicks to the menu options for both the appearance and tools menu.

Note: I will be attending an offsite next week (away Nov 11-19) but will plan to work on this analysis when when I return.

I've confirmed instrumentation is available in event.mediawiki_web_ui_actions to calculate the metrics identified for this task. This includes ability to track click to the "hide" button and clicks to the menu options for both the appearance and tools menu.

Great news; thank you for looking into this @MNeisler.

Note: I will be attending an offsite next week (away Nov 11-19) but will plan to work on this analysis when when I return.

Sounds great!

@ppelberg Please see the analysis results summarized below for review:

Methodology Notes
Reviewed a sample of events logged in October 2024 and recorded in mediawiki_web_ui_actions to identify sessions where the user interacted with the page tools or appearance menus. This data was joined with editattemptstep to determine if users completed an edit using VE during their session.

The analysis is limited to the types of edits and people that would be impacted by this change. This includes the following conditions:

  • people that completed 100 or fewer edits or unregistered users
  • edits made with Visual Editor
  • edits made on a Wikipedia main namespace
  • edits made on desktop (limited to Vector 2022 skin for this analysis)

Results

  • What percentage of people elect to hide the Tools and/or Appearance menus while editing in VE?
    • 1.1 % of registered users that made an edit attempt with VE elected to hide the Tools and/or Appearance menu.
    • A similar proportion of people elect to hide either the tools and or appearance menu during their edit (0.9% of users for each hide menu type).
    • On a per wiki basis, the highest proportion of editors that elected to hide the Tools and/or Appearance menus was 3.5% of users at arwiki (limited to wikis with sufficient events logged during the reviewed timeframe)
  • Of the people who published ≥1 unreverted edit in the main namespace, what percentage of them used the Tools and/or Appearance menu while editing using the visual editor (desktop)?
    • Overall, 3.3% of registered users that published ≥1 unreverted edit in the main namespace used either the Tools or Appearance menu while completing their edit.
    • Of those registered users, 1.5% clicked one of Page Tools menu options and 1.7% clicked on the Appearance menu options.
    • This higher per wiki rates of menu usage include 7% of registered editors at arwiki and nlwiki.
  • Of all the unreverted edits people make in the main namespace using the visual editor, what proportion of edits involved someone using the Tools and/or Appearance menus?
      • 2.3% of all editing sessions where people completed an unreverted edit in the main namespace using the visual editor including a click to either the tools or appearance menu. This includes unregistered users and registered users with 100 or fewer edits.
      • 1.5% of editing sessions included a click on one of the appearance menu options while 0.8% of editing sessions included a click to a page tools menu option.
    • The highest overserved per wiki rate was at nlwiki. At nlwiki, 7.4% of editing sessions included a click to a appearance or page tools menu option.
  • What percentage of editing sessions in the main namespace that result in an unreverted published edit (broken out by experience level) do people use the Tools and/or Appearance menu within?
    • Unregistered users: 5.6%
    • Newcomers: 4.3%
    • Junior Contributor: 1.2%

Per what @MNeisler and I discussed offline on 4 December 2024, the findings Megan shared in T378036#10349611 are leading us to include that proceeding with hiding the Tools and Appearance menus when Edit Check has the potential to activate is not likely to be disruptive.

As such, we will proceed with T379443.