[minor] HelpPanel logs incorrect events for protected pages
Open, Needs TriagePublic
Actions

Assigned To

None

Authored By

	Etonkovidova
	Mar 20 2019, 11:57 PM

Description

Below are two edge cases when HelpPanel logs events incorrectly. Although it may lead to record data that seems incomplete or wrong, such cases most likely to be really rare.

(1) A user is blocked (site wide) from editing article. Attempting to edit will produce an impression event that attributed to 'reading' but has "page_ns": 0 which is logically correct (a user can only read the article) but might look confusing.

{
  "event": {
    "action": "impression",
    "action_data": "",
    "user_id": 16448,
    "user_editcount": 0,
    "editor_interface": "reading",
    "is_mobile": false,
    "page_id": 0,
    "page_title": "",
    "page_ns": 0,
    "user_can_edit": true,
    "page_protection": "",
    "session_token": "89a31ad4be6bd33dc32b",
    "help_panel_session_id": "515bf6301b3da6494691"
  },
  "revision": 18920393,
  "schema": "HelpPanel",
  "webHost": "en.wikipedia.beta.wmflabs.org",
  "wiki": "enwiki"
}

(2) A protected page - the Help panel is shown when a user clicks on 'View source' option (for users who cannot edit protected pages). Clicks on 'View source' won't trigger impression event. All other Help panel are recorded as usual.

Note: the case when a user attempts to visit a page to which they do not have rights to view, seems not to be applicable to HelpPanel presence. Such pages are Special pages where HelpPanel is not enabled.

Actionable items

Fix the protected page logging, so that impressions are logged.
When a user is blocked site-wide, don't show them the help panel.
When a user has a partial block, show the help panel.

Related Objects

Mentioned In: T222201: Improve error handling and instrumentation logging for blocked users
Mentioned Here: T222201: Improve error handling and instrumentation logging for blocked users

Event Timeline

Etonkovidova created this task.Mar 20 2019, 11:57 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 20 2019, 11:57 PM

Etonkovidova renamed this task from [minor] HelpPanel logs incorrect events to [minor] HelpPanel logs incorrect events for blocked users and protected pages.Mar 20 2019, 11:58 PM

Etonkovidova added subscribers: nettrom_WMF, MMiller_WMF.Jul 29 2019, 6:29 PM

(2) A protected page has 'View source' option for users who cannot edit protected pages. Clicks on 'View source' won't trigger impression event.

Are you saying the help panel is displayed, but no impression event is logged? Or is the help panel not displayed on View source page views at all?

Catrope moved this task from Inbox to Needs Discussion on the Growth-Team board.Jul 31 2019, 1:58 AM

In T218839#5378480, @Catrope wrote:

(2) A protected page has 'View source' option for users who cannot edit protected pages. Clicks on 'View source' won't trigger impression event.

Are you saying the help panel is displayed, but no impression event is logged? Or is the help panel not displayed on View source page views at all?

Yes, exactly that - the Help panel is displayed, and all subsequent events are recorded correctly - open, enter-question-text` etc - but not impression.

This seems like something we can easily knock out, and protected pages could be the most popular ones that people would try to edit.

JTannerWMF moved this task from Upcoming Work to Sprint 0 (Growth Team) on the Growth-Team board.Aug 7 2019, 5:58 PM

JTannerWMF edited projects, added Growth-Team (Sprint 0 (Growth Team)); removed Growth-Team.

JTannerWMF moved this task from Incoming to Ready for Development on the Growth-Team (Sprint 0 (Growth Team)) board.Aug 7 2019, 6:21 PM

(1) A user is blocked (site wide) from editing article. Attempting to edit will produce an impression event that attributed to 'reading' but has "page_ns": 0 which is logically correct (a user can only read the article) but might look confusing.

@nettrom_WMF what would you like to see changed about the event logged in this case? Maybe something about user block status in action_data, or user_can_edit set to false?

kostajh claimed this task.Aug 20 2019, 8:09 AM

kostajh moved this task from Ready for Development to In Progress on the Growth-Team (Sprint 0 (Growth Team)) board.

@MMiller_WMF @RHo do we want to show the help panel to blocked users? I recall some discussion of this but can't remember what we decided.

(2) A protected page has 'View source' option for users who cannot edit protected pages. Clicks on 'View source' won't trigger impression event.

The reason this doesn't work with our current code is that we log the impression after an editor has loaded. When viewing a protected page, the VisualEditor, MobileFrontend source/visual editor, or traditional wikitext editor is not loaded.

@nettrom_WMF would you prefer that we have an impression event in this case which indicates that the user is looking at a protected page? Or are you able to know that this is the case by looking at events for a given session_token that have no 'impression' event?

kostajh added projects: GrowthExperiments-Help panel, GrowthExperiments.Aug 20 2019, 2:37 PM

kostajh updated the task description. (Show Details)

In T218839#5424800, @kostajh wrote:

@MMiller_WMF @RHo do we want to show the help panel to blocked users? I recall some discussion of this but can't remember what we decided.

I agree that in case (2) we do want to show users the help panel so they have an opportunity to ask why they cannot edit the protected page; and presumably they may be more inclined to edit other pages if they receive a good response.
However, in case (1) with blocked users, how likely are they to get unblocked by asking a question at the help desk and go on to edit? If the answer is they are unlikely to be unblocked (presumably because the block is there for good reason in the first place) then I would say let's not show the help panel.

However, in case (1) with blocked users, how likely are they to get unblocked by asking a question at the help desk and go on to edit? If the answer is they are unlikely to be unblocked (presumably because the block is there for good reason in the first place) then I would say let's not show the help panel.

We could attempt to distinguish between the types of blocks, since in theory one could be blocked from a namespace but still be able to edit in another. An alternative is to show a modified version of the help panel without the question asking form for users who are under any type of block.

Re case (1): if the user is blocked sitewide, they also won't be able to post on the help desk, because a sitewide block blocks them from editing any page (except their own user talk page). If the user is subject to a partial block, they may or may not be able to post on the help desk depending on the details of the block (and also, a partial block that blocks them from the help desk may not block them from editing the page they're currently on!).

I propose that we don't display the help panel for users who are blocked sitewide. For users subject to a partial block, I propose we don't bother to figure out whether the block prevents them from editing the help desk, and just always display the help panel. In the (hopefully rare) case that they are blocked from editing the help desk, they'll get an error when trying to submit their question.

And in case (2), I agree that we should show the help panel as normal, and fix whatever instrumentation bug is causing the events to be logged incorrectly. Many popular articles are semi-protected (require 10 edits and 4 days, or some variant thereof) and some are fully-protected (only admins can edit), so it would be pretty normal for a newcomer to try to edit a protected page.

Etonkovidova updated the task description. (Show Details)Aug 21 2019, 8:21 PM

kostajh updated the task description. (Show Details)Aug 22 2019, 11:23 AM

For case (2), I agree with everyone else that we should show the Help Panel to the user and enable logging of impression events as well. As others have pointed to, many popular pages are semi-protected, something also mentioned in Hill & Shaw's research on the topic ("Page Protection: Another Missing Dimension of Wikipedia Research", OpenSym 2015, see Table 3 on page 4). Enabling newcomers to ask questions about that is meaningful, as it might not be obvious why pages get protected.

In the case of a protected page impression event, I'd expect user_can_edit = false and page_protection to reflect what the page protection settings are.

For case (1), if the user is blocked sitewide they can't edit, so the Help Panel is no longer useful and it makes sense to not show it to them. If they're blocked (either partially or sitewide), does that work as if the page was protected, meaning they only get to "View source"? Or are they actually able to open the editor?

Let's make this task just above protected pages, and handle blocked users in T222201.

Unassigning myself for now but plan to come back to this early next week -- if someone else wants to get to it before then, please do

In T218839#5432638, @nettrom_WMF wrote:

For case (1), if the user is blocked sitewide they can't edit, so the Help Panel is no longer useful and it makes sense to not show it to them. If they're blocked (either partially or sitewide), does that work as if the page was protected, meaning they only get to "View source"? Or are they actually able to open the editor?

The blocked user sees the Edit tabs normally; clicking on 'Edit' tabs will bring the warning and the Help panel:

Screen Shot 2019-08-23 at 9.50.28 AM.png (820×1 px, 203 KB)

When attempting to post a question via Help panel (which is considered as editing), will display error in the Help panel:

Screen Shot 2019-08-23 at 9.46.53 AM.png (525×508 px, 61 KB)

Thank you all for hashing this out on the task. I just wanted to say that the business rules you all have come up with sound good to me. Please let me know if there is anything else I can help clarify.

Aklapper removed a project: GrowthExperiments.Sep 11 2019, 12:47 PM

Moving off sprint board in favor of Newcomer Tasks V1.0 tasks.

MMiller_WMF moved this task from Inbox to Upcoming Work on the Growth-Team board.Nov 13 2019, 7:47 AM

MBinder_WMF added a project: Growth-Team-Filtering.Apr 15 2021, 6:51 PM

	F30077510: Screen Shot 2019-08-23 at 9.46.53 AM.png
	Aug 23 2019, 4:53 PM

	F30077507: Screen Shot 2019-08-23 at 9.50.28 AM.png
	Aug 23 2019, 4:53 PM

[minor] HelpPanel logs incorrect events for protected pagesOpen, Needs TriagePublicActions

Description

Actionable items

Related Objects

Event Timeline

[minor] HelpPanel logs incorrect events for protected pages
Open, Needs TriagePublic
Actions