Page MenuHomePhabricator

Click on modal link to nonexisting talk page should be logged as modalInternalClicked
Closed, ResolvedPublic3 Story Points

Description

We would like to filter out edit links that create new pages from normal edit clicks

Background

If the page issue template contains a link to the article talk page (like https://en.wikipedia.org/wiki/Template:NPOV : "Relevant discussion may be found on the talk page."), and the talk page doesn't exist, a click on that link will be incorrectly logged as modalEditClicked (instead of modalInternalClicked as intended). Example: https://lv.m.wikipedia.org/wiki/Ivars_Zaikins

I.e. many events that we interpret as an intention to change the article will instead have come from a mere intention to learn more about the issue. Hopefully this will largely cancel out in the A/B test as it affects both the old and new design, but it will impact our estimate of the size of the effects (in increasing edits, in particular), and our estimate for the frequency of edit attempts in general.

Developer notes

Tilman's patch filters out any red link edit clicks which gets us halfway to the solution. What remains, is making these links click elsewhere.

We have two possibilities

  1. Complicate our selectors and add red link clicks to modalInternalClicked (untested, but click a:not(.external):not([href*=edit]), a[href*="redlink=1"]:not(.external)' might work for modalInternalClicked
  2. Add a new event modalRedLinkClicked

Per https://phabricator.wikimedia.org/T204073#4577874 I propose the latter, but during estimation we may want to consider 1

Acceptance criteria

  • Update Schema:PageIssues to include the modalRedLinkClicked event
  • Amend Tilman's patch to add a new click handler ('click a[href*=redlink]") and bump the revision number.

QA steps

Make sure PageIssues instrumentation is enabled for 100% users
e.g.

$wgMinervaABSamplingRate = 1;

Visit a page where there are red links in the page issues overlay
e.g. http://reading-web-staging.wmflabs.org/W_Dowler_%26_Sons

Verify that different events are fired when you click "talk page" (modalRedLinkClicked), "improve it" (modalEditClicked) and sources (modalInternalClicked). Clicking no link should trigger 2 of these events, they are mutually exclusive.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 11 2018, 5:08 PM

We decided yesterday to not treat this as a blocker for T191532 , as we want to launch the A/B test as soon as possible (T200792). But it is still a serious bug that will affect the data quality. Probably more so on non-English language Wikipedias: Enwiki is assumed to have a lower ratio of articles without talk page, because of that community's systematic efforts to tag every article with a content assessment on the talk page.

ovasileva triaged this task as High priority.Sep 11 2018, 5:42 PM
ovasileva lowered the priority of this task from High to Normal.
ovasileva added a project: Readers-Web-Backlog.
ovasileva moved this task from Incoming to Needs Prioritization on the Readers-Web-Backlog board.

But it is still a serious bug that will affect the data quality.

Do we plan on running more A/B tests after the one we run (hopefully) next week?

I.e. many events that we interpret as an intention to change the article will instead have come from a mere intention to learn more about the issue.

IN the case of "Please help improve this article. If there are any suggestions, you can add them to the discussion. Read more in the user guide." the link to discussion is a red link. I would argue that clicking that red link is a form of editing- someone is taking action and providing feedback. Thus modalEditClicked makes complete sense to me here. However, yes it doesn't make sense in enwiki.
It sounds like actually the problem statement is "talk page links should not be counted as edit links". I would propose a new action "talkLinkClicked" or "redLinkClicked" for these particular links. Would that help solve the issue?

With regards to implementation, red links have the .new class but talk page links have no identifier so we'd need to scan the URL for talk (and given namespaces are translated this differs per url). Red links can also occur if the template links to other pages that don't exist (e.g. accidental typo)... We would either have to distinguish between URLS that have edits to talk pages and URLs that have edits to non-talk pages OR scan the URL for the talk namespace.

Easiest thing to do would be to have a redLinkClicked event, for logging any clicks to red links separately. It also feels like the lowest risk as it doesn't involve trying to work out if a link is to a talk page or not...

While it would be interesting to distinguish that "leave a comment" case, I don't think it's necessary to create a new action to resolve this bug. The main goal here is to make sure that the instrumentation reflects the schema specification, according to which modalEditClicked (like editClicked) refer to the user starting to edit the page itself (not leaving comments somewhere).

Besides, Latvian seems to be an exception with this invitation to add suggestions to the talk page - checking the NPOV template on a few other Wikipedias, most seem to link to the talk page for existing discussions instead, i.e. for reading purposes. Like these examples:

https://ja.wikipedia.org/wiki/人海戦術 "For discussion see the notes "

https://hu.m.wikipedia.org/wiki/H%C3%A1ziPatika.com " see talk page for details of the discussion"

https://pt.wikipedia.org/wiki/M%C3%A3e_Meera " reasons indicated in the discussion page " (although that NPOV template doesn't seem to be Ambox-based)

I don't think it's necessary to create a new action to resolve this bug refer to the user starting to edit the page itself (not leaving comments somewhere).

Having a redLinkClicked event seemed a low level of risk (but would of course delay things) so that's why I suggested it. Detecting only edits to the current page seems extremely high risk to me as it involves parsing the URL and would thus involve non-trivial changes to the instrumentation. I would push back at doing it at this time given things appear to be stable with the instrumentation. If we plan on doing another A/B test then sure, we can look into this post-A/B test.

Change 460003 had a related patch set uploaded (by HaeB; owner: HaeB):
[mediawiki/skins/MinervaNeue@master] Exclude redlink clicks from modalEditClicked event

https://gerrit.wikimedia.org/r/460003

...

With regards to implementation, red links have the .new class but talk page links have no identifier so we'd need to scan the URL for talk (and given namespaces are translated this differs per url). Red links can also occur if the template links to other pages that don't exist (e.g. accidental typo)... We would either have to distinguish between URLS that have edits to talk pages and URLs that have edits to non-talk pages OR scan the URL for the talk namespace.

I don't think we need to parse namespaces. As mentioned, it's fine if we just ensure that attempts to edit the page itself are distinguished from all the other internal clicks.
Fortunately, besides the CSS class, red links also have the "redlink=1" URL parameter included (which is also - in contrast to namespace prefixes - not localized and thus much easier to detect). E.g. in the second example from T204073#4575824 :
https://hu.m.wikipedia.org/w/index.php?title=Vita:H%C3%A1ziPatika.com&action=edit&redlink=1

I submitted a patch to at least fix the faulty detection method for modalEditClicked, by excluding URLs with "redlink" in them.
If someone else could likewise change the logic for modalInternalClicked to include those edit URLs that have "redlink" in them, that would be great.

By the way, I think the code should check for "action=edit" instead of just "edit" (and "redlink=1" instead of "redlink" in my patch), but I don't know enough about the escaping syntax to do that myself:

'click a:not(.external):not([href*=edit])': 'onInternalClick',
'click a[href*="edit"]': 'onEditClick'

(also, why does it use quotes in the second line, but not in the first?)

Tilman's patch filters out any red link edit clicks. I think this is a trivial change and I'm happy to merge it.

If someone else could likewise change the logic for modalInternalClicked to include those edit URLs that have "redlink" in them, that would be great.

This bit is the risky bit I talked about in T204073#4575826 as it requires complicating the selector and some additional testing that these events work as expected.

The existing selector for onInternalClick is

click a:not(.external):not([href*=edit])

which means
run on a click to a link that is not external and not an edit url with "edit" inside it. The problem with redlinks is they do have edit inside them so we'd need some kind of OR logic.

Untested, but this should work (At the cost of becoming less readable due to repeated .external checks:)) It however will track all red links as "onInternalClick" - so if the page issue for whatever reason says "This page has lots of red links, help fix the page by starting new pages and links to those pages", that wouldn't be tracked (not sure if that's a problem in practice)

			'click a:not(.external):not([href*=edit]), a[href*="redlink=1"]:not(.external)': 'onInternalClick',
			'click a[href*="edit"]:not([href*=redlink])': 'onEditClick'

My preference if we do this would be treat these as different actions as it seems an equal amount of work without compromising readability:

			'click a:not(.external):not([href*=edit])': 'onInternalClick',
			'click a[href*=redlink]": 'onRedClick'
			'click a[href*="edit"]:not([href*=redlink])': 'onEditClick'

Thanks Jon! Both options work for me. I was skeptical of the need of introducing such a new modalRedLinkClicked action, as that distinction is not needed for our existing research questions. But if it makes things easier in the code, feel free to go ahead with it - it's easy enough to add up both kinds of actions during analysis, and it might provide us with some interesting additional data. (Editors might consider that such redlinks are misleading in case the wording in the template implies that information can already be found there, and suppress them from the template with an ifexists condition.)

Jdlrobson updated the task description. (Show Details)Sep 12 2018, 8:10 PM
Jdlrobson moved this task from Needs Prioritization to Upcoming on the Readers-Web-Backlog board.
ovasileva raised the priority of this task from Normal to High.Sep 13 2018, 4:16 PM

Change 461179 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/skins/MinervaNeue@master] Red links are linked separately

https://gerrit.wikimedia.org/r/461179

Change 460003 merged by jenkins-bot:
[mediawiki/skins/MinervaNeue@master] Exclude redlink clicks from modalEditClicked event

https://gerrit.wikimedia.org/r/460003

Change 461179 merged by jenkins-bot:
[mediawiki/skins/MinervaNeue@master] Red links are linked separately

https://gerrit.wikimedia.org/r/461179

Jdlrobson reassigned this task from Jdlrobson to Ryasmeen.Sep 18 2018, 10:15 PM
Jdlrobson updated the task description. (Show Details)
Jdlrobson removed a project: Patch-For-Review.

Now on staging and ready for testing with correct config.

Ryasmeen reassigned this task from Ryasmeen to ovasileva.Sep 21 2018, 12:24 AM
Ryasmeen added a subscriber: Ryasmeen.

Checked on this article on staging: http://reading-web-staging.wmflabs.org/w/index.php?title=Ivars_Zaikins&mobileaction=toggle_view_mobile

Test coverage: Desktop Safari, Desktop Chrome, Desktop Firefox, iOS Safari, Chrome Android.

In all cases, correct events are getting fired when I click "talk page" (modalRedLinkClicked), "improve it" (modalEditClicked) and sources (modalInternalClicked). And in no cases I have found 2 of these events triggering at the same time.

Moving to Signoff.

ovasileva closed this task as Resolved.Sep 26 2018, 3:32 PM

looks good!

Restricted Application added a project: User-Ryasmeen. · View Herald TranscriptSep 26 2018, 3:32 PM