Page MenuHomePhabricator

[SPIKE] What events are being tracked within the "section=new" flow
Open, Needs TriagePublic

Description

This task is about understanding what events in, and metadata about, the section=new flow [i] are currently being tracked.

Background

One way we will evaluate the effectiveness of the New Discussion Tool is to look at whether people who use this new tool/workflow are more or less likely to publish new discussion topics than the people who use the existing workflow. [i]

In order to draw the comparison described above, we'll need to ensure the existing section=new workflow is instrumented sufficiently.

This task is about understanding how it is currently instrumented so we can decide what – if any – adjustments needs to be made to it.

Instrumentation documentation

EVENTS

ActionEvent nameSchema

META

PropertyValueenum

Done

  • Document what – if any – events the current section=new workflow is emitting in the EVENTS table above
    • Assuming some events are being emitted, what are these events called and what schema are they being logged in?
  • Document what pieces of metadata are being logged about the section=new form in the META table above
  • Determine what – if any – additional instrumentation will need to be added to the existing workflow to complete the analyses T263053 and T263054 are asking for.

i. E.g. https://en.wikipedia.org/w/index.php?title=User_talk:PPelberg_(WMF)&action=edit&section=new

Event Timeline

JTannerWMF assigned this task to DLynch.Oct 14 2020, 4:38 PM

Discussed in meeting: it's okay to summarize the table, since there's going to be a lot of VisualEditorFeatureUse events which are common to all integrations.

So! The answer here is "it depends", because there's several variants we could be talking about.

  • WikiEditor
  • VisualEditor NWE
  • VisualEditor (visually) -- which we can ignore, if we're only considering talk namespaces, but section=new does work outside of them

However, the summary for all of them is "you get exactly the events you get when editing an article, but with some metadata changed to reflect that you're in section mode".

For all options, events will be logged to EditAttemptStep and VisualEditorFeatureUse. The only practical difference from article-editing is that EditAttemptStep init event will be logged with init_type=section whenever there's a section parameter in the URL.

There's inconsistent behavior of the EditAttemptStep init_mechanism field:

  • WikiEditor:
    • If there's no HTTP referer, it'll always be url
    • If the referer is present it'll be new if section=new or click if section=[some-number]
  • VisualEditor:
    • If you clicked on the new section tab and it dynamically loaded it'll be click
    • If you clicked on the new section tab and it navigated to the new page it'll be url
    • It'll never be new

Visual / NWE will log all their normal VisualEditorFeatureUse events.

There's precisely one VisualEditorFeatureUse event logged from WikiEditor, which is editor-switch; visual-desktop for when you're switching away to VisualEditor. This should only be possible to trigger if you're doing section=new on a non-Talk page, of course.

Thank you for pulling this together, @DLynch. Before going into what, if anything, needs to be changed about the instrumentation of the existing section=new workflows to meet what will be asked for in T263053 and T263054, I'd like to confirm I am accurately understanding the current state.

With the above in mind, some clarifying questions...

So! The answer here is "it depends", because there's several variants we could be talking about.

  • WikiEditor
  • VisualEditor NWE
  • VisualEditor (visually) -- which we can ignore, if we're only considering talk namespaces, but section=new does work outside of them
  • Can you confirm this diagram accurately describes how events are being logged at a high level? Sometimes, seeing something visually helps me better understand it.
Diagram showing section=new logging
Link: Editing Team/Sketching

However, the summary for all of them is "you get exactly the events you get when editing an article, but with some metadata changed to reflect that you're in section mode".

  • Would it be accurate for me to understand "section mode" as any context where section=new is present in the URL?

For all options, events will be logged to EditAttemptStep and VisualEditorFeatureUse. The only practical difference from article-editing is that EditAttemptStep init event will be logged with init_type=section whenever there's a section parameter in the URL.

  • Would it be accurate for me to understand the above as follows? "section will always appear in the URL when someone is viewing the new section form, regardless of the affordance they clicked/tapped to cause the new form to be shown (e.g. "New section" tab, manually navigating to the form's URL, clicking a custom call to action (e.g. the "Ask a question" button at en.wiki's Teahouse)?
  • Would it be accurate for me to assume we do not currently have a way of detecting what someone clicked/pressed/tapped to open the section=new form?
ppelberg updated the task description. (Show Details)Wed, Nov 18, 11:34 PM

Task description update
ADDED the below to the task description's ===Done section:

  • Determine what – if any – additional instrumentation will need to be added to the existing workflow to complete the analyses T263053 and T263054 are asking for.

Can you confirm this diagram accurately describes how events are being logged at a high level? Sometimes, seeing something visually helps me better understand it.

I have to make my own diagram for this, so hold on.

Would it be accurate for me to understand "section mode" as any context where section=new is present in the URL?

Yes, that should be accurate. Even the "I clicked the new section tab and it replaced the page contents" route should update the URL.

Would it be accurate for me to understand the above as follows? "section will always appear in the URL when someone is viewing the new section form, regardless of the affordance they clicked/tapped to cause the new form to be shown (e.g. "New section" tab, manually navigating to the form's URL, clicking a custom call to action (e.g. the "Ask a question" button at en.wiki's Teahouse)?

I believe so. I don't see any routes to it that wouldn't either trigger from the URL or update the URL to contain it, anyway.

Would it be accurate for me to assume we do not currently have a way of detecting what someone clicked/pressed/tapped to open the section=new form?

Nothing fine-grained, certainly. We could make guesses based on the referrer, but that's not reliably present.

Does this help?