This task is about determining what – if any – new events need to be added to the existing section=new workflow order to analyze the metrics these tickets are asking for:
T265100 explains what events in the existing section=new workflow are being logged.
@DLynch - I have a follow-up question regarding the behavior of the init_mechanism field described in T265100#6618083.
WikiEditor:
- If there's no HTTP referer, it'll always be url
- If the referer is present it'll be new if section=new or click if section=[some-number]
Is it possible for wikitext section edits with no HTTP referrer (where editor_interface = wikitext, init_type = section, init_mechanism = url) to include edits to start a new section?
Context: I'm currently reviewing the current instrumentation to confirm if we are able to distinguish between talk page edits to start a new section using the section=new workflow versus edits to an already created section.
Is it possible for wikitext section edits with no HTTP referrer (where editor_interface = wikitext, init_type = section, init_mechanism = url) to include edits to start a new section?
That's currently what could happen, yes. Init events with type=section and mechanism=url are going to be a mixture of new and existing sections. Intuitively I'd expect mechanism=url to be comparatively uncommon, on the assumption that most ways we'd create a new section involve just clicking the link on the page.
This predates my involvement in the project, but I'd speculate that this particular logging was never intended to let us determine this particular bit of data. If the mechanism=url events are common, we could certainly add a new mechanism variant that would distinguish it while still preserving the "this was direct URL access" semantics.
(Basically, it's very possible to make changes so that init_mechanism is more consistent between the various implementing editors. I haven't touched it because I've never been assigned a task which directly involved it, and changing the semantics randomly without knowing whether any ongoing analysis depends on these differences seemed like a bad idea...)
Thanks for the clarification @DLynch.
Intuitively I'd expect mechanism=url to be comparatively uncommon, on the assumption that most ways we'd create a new section involve just clicking the link on the page.
I did a quick query of the number of wikitext section edits by init_mechanism from November through December 2020. init_mechanism = url events are the least common type (9.92% of all wikitext section edits) but given current instrumentation, there's no way to tell what percent of that is starting a new section vs editing a current section. I agree it's probably uncommon though.
| editing_interface | init_mechanism | edits | % of edits by editor interfaece |
|---|---|---|---|
| Wikitext | click | 5152731 | 78.85% |
| Wikitext | url | 647969 | 9.92% |
| Wikitext | new | 734177 | 11.23% |
If the mechanism=url events are common, we could certainly add a new mechanism variant that would distinguish it while still preserving the "this was direct URL access" semantics.
Based on my understanding, existing instrumentation can be used to identify all new section events created using the new discussion tool. Those events will be recorded as action = init , integration = discussiontools. and init_type = section and should only be recorded when a new section is started.
However, we are unable to identify new section events created using the existing workflow because the init_mechanism behavior is inconsistent. This is needed if we want to make any comparisons between the two workflows and to accurately calculate metrics such as Adoption metric #4 in T263053 (how many people used the New Discussion Tool to add X proportions of new topics ).
Based on this, I think it would make sense to go ahead and add a new mechanism to distinguish new section events if feasible. (If not, we will need to add a caveat to data on the existing new section workflow that it does not include new section edits where there is no HTTP referrer)
Notes from the conversation @MNeisler and I had today...
Let's do what Megan is suggesting above and what @DLynch alluded to doing in T269486#6746576. This work will happen in T272544.
Also @MNeisler, we did implement a revision tag for edits made with the New Discussion Tool. This work happened in T260467. See: the New Topic and Visual tags associated wit this edit: https://en.wikipedia.beta.wmflabs.org/w/index.php?title=Talk%3ACats&type=revision&diff=471357&oldid=470814.
Done
Following the investigation Megan did, the conversation she and I had today and me double confirming a revision tag has been created for edits made with the New Discussion Tool [i], we can confidently say T272544 is the only instrumentation that is currently needed for T263053.
As for T263054, Megan and I agreed that we are going to revisit the scope of that task once T269471 is resolved. At that point, we can determine whether additional instrumentation is necessary.
- If new events need to be added, list them in the ===New events section above
Done per T269486#6763524. The task description's ===New events section has been updated.
For reference, I started a new discussion tool instrumentation spec where I mapped current metrics outlined in T263053 and T263054 to existing instrumentation. As @ppelberg mentioned, this can be updated and revised when we revisit the workflow engagement scope.
After the patches in T272544 are merged, there's going to be more than just a change to split up init_mechanism=url. There was also an inconsistency whereby VE/NWE would log clicks to directly create a page via the "create" tab as init_mechanism=click rather than the correct init_mechanism=new. If you want to see how widespread this was, in existing data this would show up as init_mechanism=click && page_id=0.