Page MenuHomePhabricator

[REQUEST] Determine level of no JS usage of Event Registration
Open, Needs TriagePublicMon, May 6

Description

Please refer to https://www.mediawiki.org/wiki/Product_Analytics#How_to_get_help_with_data_or_analysis for help answering these prompts

What team/program is this request for?

Campaigns Product

What are you requesting?

I would like to know the percentage of Event Registration users who accessing the tool without JavaScript support.

What is the problem you're trying to solve?

Right now, the Campaigns team builds all tools for events that have support no-JS users. We made this decision early on when developing Event Registration (se Front-end decisions for V0). We do this because we have learned that some organizers hold events in spaces that may have slow internet connectivity, so the no JS-experience makes it easier for them to access Event Registration as an organizer or participant, even if there are internet connection issues. However, we know that the internet has become more interactive, relying on JavaScript for more and more functionality. Meanwhile, there are engineering and designs trade-offs that we consistently make in our product development due to no-JS support. For these reasons, there is a potential argument to be made to drop no-JS support (see document for general thoughts).

For these reasons, we want to check our assumptions and see if users are truly using the no-JS version of event registration. We would also, if possible, like this data broken by region and by whether the users are organizers or participants.

What decision will you make or action will you take with the deliverable?

This information will help us determine:

  • If we want to keep on supporting no-JS users for Event Registration
  • If the no-JS experience is more useful for a certain subset of users (for example, from a certain region or whether they are organizers or participants) -- and, if yes, if we want to prioritize certain features or do certain community outreach to these users to learn more about what support they may need and where there are currently gaps in their experience in using the tool

Additional details

Details

Due Date
Mon, May 6, 4:00 AM

Event Timeline

FYI: two potentially related tickets: T240697 and T251464 (and also T263505) and T234695#6481955 for readers data

This analysis builds on the work completed in T240697#7127421 which built on the work in T240697#6217971.

The following event in the VisualEditorFeatureUse eventlogging schema allows us to track these events:

event.feature = 'mwSave' event.action = 'source-has-js'

This instrumentation only applies to saved edits made with WikiEditor.

In this analysis, we assume all non-wikitext edits (editor_interface ! = 'wikitext') or non-page edits (integration ! = 'page')(i.e. edits made with VisualEditor, Discussion Tool or app edits) have JS enabled as these editors require JS to work.

Interim results: 0 No-JS event-registration edits from January-April 2024.
For data on no-JS readers see T234695#6481955
See more on no-JS metrics.

Tables used:
event.editattemptstep
visualeditorfeatureuse

Query:

events_query = '''
WITH saveSuccess_sessions as (
    SELECT DISTINCT
      event.editing_session_id AS ss_session_id,
      event.user_id AS user_id,
      event.editor_interface AS editor_interface,
      event.platform AS platform,
      event.integration AS integration,
      useragent.device_family as user_agent,
      wiki AS ss_wiki
FROM event.editattemptstep
WHERE
    YEAR = 2024
    AND MONTH >= 01
    AND wiki = 'metawiki'
    AND event.page_title LIKE 'Event:%'
    AND event.action = 'saveSuccess' 
    AND useragent.is_bot = false
   AND NOT event.is_oversample
),

js_wt_sessions AS(
  SELECT
    event.editingSessionid AS wt_js_session_id,
    wiki AS js_wt_wiki,
    event.action AS wikitext_js_edit
FROM event.visualeditorfeatureuse 
WHERE
    YEAR = 2024
    AND MONTH >= 01
    AND wiki = 'metawiki'
    AND event.action = 'source-has-js'
    AND event.feature = 'mwSave'
    AND event.editor_interface = 'wikitext'
    AND useragent.is_bot = false
)


SELECT 
  wiki,
  user_agent, 
SUM(CAST(user_id = 0 AND editor_interface = 'wikitext' AND integration = 'page' AND platform = 'desktop' AND wikitext_js_edit is NULL AS INT)) AS anon_no_js_edit,
  SUM(CAST(user_id != 0 AND editor_interface = 'wikitext' AND integration = 'page' AND platform = 'desktop' AND wikitext_js_edit is NULL AS INT)) AS user_no_js_edit,
  SUM(CAST(user_id = 0 AS INT)) AS anon_all_edits,
  SUM(CAST(user_id != 0 AS INT)) AS user_all_edits 
FROM (
  SELECT 
    user_id AS user_id,
    ss_wiki AS wiki,
    editor_interface AS editor_interface,
    integration AS integration,
    platform AS platform,
    user_agent AS user_agent,
    ss_session_id AS ss_session_id,
    wikitext_js_edit
  FROM saveSuccess_sessions ss  
  LEFT JOIN 
    js_wt_sessions ON ss.ss_session_id = js_wt_sessions.wt_js_session_id
    AND ss.ss_wiki = js_wt_sessions.js_wt_wiki
  GROUP BY 
    ss.user_id, 
    ss.ss_wiki,
    ss.editor_interface,
    ss.integration,
    ss.platform,
    ss.user_agent,
    ss.ss_session_id, 
    js_wt_sessions.wikitext_js_edit
    ) edit_sessions
GROUP BY
    wiki,
    user_agent
'''

Questions from Ilana, to contextualize the task:

How many edits came up?

df['user_all_edits'].sum()

From January 1st 2024 to today there are 658 saved edits on pages on meta LIKE 'Event:%'. Zero of these edits are no-JS edits

Number of events that the above corresponds to? Ilana notes that she can also pull the event count from Special:AllPages data

SELECT COUNT(DISTINCT event.page_title) AS unique_event_count
FROM event.editattemptstep
WHERE
    YEAR = 2024
    AND MONTH >= 01
    AND wiki = 'metawiki'
    AND event.page_title LIKE 'Event:%'
    AND event.action = 'saveSuccess' 
    AND useragent.is_bot = false
   AND NOT event.is_oversample

114 events from January 1st to today.

As far as event counts:

events_query_count_mariadb = '''
    SELECT 
       COUNT(DISTINCT campaign_events.event_id) AS unique_event_count
    FROM campaign_events 
    WHERE 
        campaign_events.event_created_at BETWEEN "20240101000000" and "20240508000000" OR
        campaign_events.event_last_edit BETWEEN "20240101000000" and "20240508000000"
    '''

campaign_events_confirm_count = mariadb.run(events_query_count_mariadb , 'wikishared')
  1. The above query is telling me that there are 319 applicable unique events in the campaign_events dB over the same period.
  2. The metrics sheet tells me that there were 224 new events in Jan|Feb|March

Tried again with a list of 319 page ids. First I pulled campaign event page names, then I pulled page ids for those page names.
This confirms zero no_JS edits on those event pages.
With this approach I see 509 edits total.

events_query_review = '''
WITH saveSuccess_sessions as (
    SELECT DISTINCT
      event.editing_session_id AS ss_session_id,
      event.user_id AS user_id,
      event.editor_interface AS editor_interface,
      event.platform AS platform,
      event.integration AS integration,
      useragent.device_family as user_agent,
      wiki AS ss_wiki
FROM event.editattemptstep
WHERE
    YEAR = 2024
    AND MONTH >= 01
    AND wiki = 'metawiki'
    --AND event.page_title LIKE 'Event:%'
    AND event.page_id IN {events_list}
    AND event.action = 'saveSuccess' 
    AND useragent.is_bot = false
   AND NOT event.is_oversample
),

js_wt_sessions AS(
  SELECT
    event.editingSessionid AS wt_js_session_id,
    wiki AS js_wt_wiki,
    event.action AS wikitext_js_edit
FROM event.visualeditorfeatureuse 
WHERE
    YEAR = 2024
    AND MONTH >= 01
    AND wiki = 'metawiki'
    AND event.action = 'source-has-js'
    AND event.feature = 'mwSave'
    AND event.editor_interface = 'wikitext'
    AND useragent.is_bot = false
)


SELECT 
  wiki,
  user_agent, 
SUM(CAST(user_id = 0 AND editor_interface = 'wikitext' AND integration = 'page' AND platform = 'desktop' AND wikitext_js_edit is NULL AS INT)) AS anon_no_js_edit,
  SUM(CAST(user_id != 0 AND editor_interface = 'wikitext' AND integration = 'page' AND platform = 'desktop' AND wikitext_js_edit is NULL AS INT)) AS user_no_js_edit,
  SUM(CAST(user_id = 0 AS INT)) AS anon_all_edits,
  SUM(CAST(user_id != 0 AS INT)) AS user_all_edits 
FROM (
  SELECT 
    user_id AS user_id,
    ss_wiki AS wiki,
    editor_interface AS editor_interface,
    integration AS integration,
    platform AS platform,
    user_agent AS user_agent,
    ss_session_id AS ss_session_id,
    wikitext_js_edit
  FROM saveSuccess_sessions ss  
  LEFT JOIN 
    js_wt_sessions ON ss.ss_session_id = js_wt_sessions.wt_js_session_id
    AND ss.ss_wiki = js_wt_sessions.js_wt_wiki
  GROUP BY 
    ss.user_id, 
    ss.ss_wiki,
    ss.editor_interface,
    ss.integration,
    ss.platform,
    ss.user_agent,
    ss.ss_session_id, 
    js_wt_sessions.wikitext_js_edit
    ) edit_sessions
GROUP BY
    wiki,
    user_agent
'''
df_review = spark.run(events_query.format(events_list = events_list))