Page MenuHomePhabricator

Evaluate use of tools available in mobile VE toolbar
Closed, ResolvedPublic

Description

This task is about establishing a baseline for how people are using tools within the mobile visual editor's toolbar.

This analysis will help us make a minimum of two decisions (described below).

This work is prompted, in large part, by Abstract Wikipedia needing to find a place within mobile VE to offer people the ability to insert a function.

Decisions to be made

  1. Which of the existing tools might we be able to relocate into a yet-to-be introduced Insert sub-menu with minimal consequence?
  2. What – if any – adjustments might we need to make to the design we move forward with because we notice the change producing negative consequeces

Research questions

Meta

  • Projects: all Wikipedias
  • Platform(s): Mobile
IDResearch questionMetric(s)
1.Who is using the tools within the mobile visual editor's toolbar?Of all the edits in the Wikipedia main namespace where a tool within mobile VE's toolbar is used (link, undo, text formatting, citation), what proportion of these edits were completed by 1) brand new editors, 2) Junior Contributors, 3) Senior Contributors?
2.How heavily do people depend on the tools?Of all the unreverted edits made with mobile VE in the Wikipedia main namespace, what proportion of edits involve people using each tool that is available within the toolbar (undo, text formatting, link, insert citation)?
3.How often do people "reach" for each tool within the toolbar (undo, text formatting, link, insert citation)?How many times within a single edit session do people use the various tools available within mobile VE's toolbar?

Event Timeline

ppelberg renamed this task from Evaluate to Evaluate use of tools available in mobile VE toolbar.
ppelberg updated the task description. (Show Details)
ppelberg updated the task description. (Show Details)
MNeisler triaged this task as Medium priority.Oct 23 2024, 6:27 PM
MNeisler moved this task from Triage to Current Quarter on the Product-Analytics board.

I completed an analysis exploring the use of tools available within the mobile VE toolbar. This currently includes the link, citation, and text formatting tools. The undo tool does not appear to be instrumented within VisualEditorFeatureUse. New instrumentation would be required if we wanted to include analysis of this tool. cc @ppelberg

Please see the summary below and let me know if you have any questions or comments.

Methodology Notes
Data collected includes a sample of visual editor mobile edits completed on a main page namespace across all Wikipedia projects between 1 January 2025 through 22 January 2025.

Who is using the tools within the mobile visual editor's toolbar?

Metric: Of all the edits in the Wikipedia main namespace where a tool within mobile VE's toolbar is used (link, undo, text formatting, citation), what proportion of these edits were completed by 1) newcomers, 2) Junior Contributors, 3) Senior Contributors?

Proportion of all mobile VE toolbar edits by experience level

User typeNumber of editsProportion of edits
Unregistered2677434.8%
Newcomer25283.3%
Junior Contributor1453418.9%
Senior Contributor3301843.0%
  • Senior contributors completed the highest proportion of mobile VE tool edits (43%) while newcomers completed the fewest (3%). However, this is likely due to a higher rate of editing activity by a senior contributor compared to newcomers or junior contributors.

Proportion of all distinct, registered mobile VE toolbar users by experience level

User typeNumber of usersProportion of users
Newcomer228220.6%
Junior Contributor550249.8%
Senior Contributor327129.6%

(Limited to registered users)

  • If we review all the distinct users that completed at least one edit using a mobile VE tool, there are more distinct Junior contributors that completed a mobile VE toolbar edit compared to Senior contributors. Of all the distinct users that completed an edit using a mobile VE tool, almost half (49.8%) were Junior contributors.

mobile_ve_too_edits_byexp.png (2,210×1,216 px, 190 KB)

  • The usage of each tool type varies based on experience level.
    • Citation tool: The majority of mobile ve edits (54%) where the citation tool was used were completed by Senior contributors.
    • Link tool: Senior contributors also complete the highest proportion of edits (41%) where the link tool was used, however; unregistered users complete close to the same proportion (36%).
    • Text formatting: Text formatting is the least frequently used tool by Senior Contributors within the mobile ve toolbar. 50% of all edits where the text formatting tool was used were completed by unregistered users.
How heavily do people depend on the tools?

Metric: Of all the unreverted edits made with mobile VE in the Wikipedia main namespace, what proportion of edits involve people using each tool that is available within the toolbar (undo, text formatting, link, insert citation)?

Proportion of published, unreverted edits completed with Mobile VE tools

Tool typeNumber of editsProportion of edits
No mobile ve tool used18461170.5%
link5192319.8%
citation204617.8%
text formatting47001.8%
  • 29.5% of all unreverted and published mobile VE edits included the use of a tool within the mobile VE toolbar (text formatting, link, insert citation).
  • Overall, the link tool was the most frequently used tool. 19.8% of all unreverted edits included the use of the link tool followed by the citation tool (7.8%) and text formatting (1.8%).

prop_edits_mobile_ve_tool.png (2,228×1,222 px, 178 KB)

  • These trends are consistent across each experience level group. For each user group, the link tool was used to complete the highest proportion of unreverted and published edits (15 to 20%) while the text formatting tool was the least used (3% or less) in these edits.
How often do people "reach" for each tool within the toolbar (text formatting, link, insert citation)?

Metric: How many times within a single edit session do people use the various tools available within mobile VE's toolbar?

  • The median number of clicks across all tool types in the mobile ve toolbar is 2 clicks in a single editing session.
  • Overall, each tool is typically only used once per editing session. The median number of clicks is 1 for each tool type (link, citation, and text formatting).
  • There’s some variation when results are broken down by experience level. Newcomers and Junior contributors are more likely to use the text formatting tool twice in a session while Senior contributors are more likely to use the link tool twice in an editing session.

Code repo

Decision: Which of the existing tools might we be able to relocate into a yet-to-be introduced Insert sub-menu with minimal consequence?

Based on the data presented in T376839#10500853, my recommendation is to relocate the text formatting menu as I anticipate this would have minimal consequences to the editor's mobile VE editing experience, including the ability to complete an edit that is not reverted.

This is based on the following insights and trends:

  • Editors are less dependent on text formatting compared to the other mobile VE tools to complete a published edit. Only 1.8% of all unreverted, published edits completed with mobile VE used this tool.
    • This is true for both newcomers making their first edit and more experienced editors so moving this tool should have minimal consequence for all types of editors.
  • The Link tool is the most frequently used mobile VE tool by all editor types. It was used in 19.8% of all unreverted, published edits. It is also the most reached for tool by Senior Contributors who complete the most edits using the mobile VE tool.
  • While editors don’t depend on the citation tool as much as the link tool, it is used in 4 times as many published edits as the text formatting tool. Additionally, relocating this tool may decrease newcomer and Junior Contributors' awareness of the need/option to add a citation for edits. This could decrease the proportion of edits that include a reference and overall edit quality.
  • We do not have data to confirm usage and identify the consequences of moving the undo tool; however, I imagine that users may use this tool frequently while editing on mobile. Mobile devices have smaller screens making accidental taps more common and the undo tool allows for more easily quick corrections.

cc @ppelberg

Decision: Which of the existing tools might we be able to relocate into a yet-to-be introduced Insert sub-menu with minimal consequence?

Per what the Editing Team converged on offline, we're going to move forward with the following:

  1. Move the Add citation button into the + menu T382454 introduces
  2. Leave the 🔗 exposed within the existing toolbar

This behavior will be implemented in T382454.

Rationale
The Editing Team converged on the path forward described above with the following assumptions in mind...

  • It's valuable to leave the text formatting menu exposed because:
    • The presence of – what we assume will be – a familiar icon/set of features could help people who are new recognize the editing interface as, well, a text editing interface
    • Moving the the text formatting menu into the + menu (the only other place at present where it could be placed) would introduce UX issues (nested dropdown menus)
  • It's okay to move the Add citation button within + menu because:
    • Edit Check will prompt newcomers who attempt to add new content without a reference to consider doing so
    • While we initially thought the mere presence of the Add citation button could serve as a helpful reminder for people to add references, we think Reference Check (per above) is sufficient. Further, over the years, we've heard a range of people express confusion about what the current " means causing us to question the icon's effectiveness at reminding people who are new to add references.
      • Note: by moving " into tho + menu, the button will also gain a label which could reduce confusion.
    • We assume experienced volunteers seeking the tool to add references will intuitively look for it within the + menu
    • Of the people who use the citation tool while editing, they're using it ~1x/edit session. Thus, we think people are not likely to grow frustrated by the tool being an additional tap away.