Page MenuHomePhabricator

Context items: verify logging events are firing properly
Closed, ResolvedPublic

Description

Overview

This task involves the work with answering the following question:

For context items in mobile VE, are the right logging events being fired by the right user actions/inputs at the right times?

The above is intended to help us verify the work we recently did to enhance instrumentation for mobile context items and dialogs in mobile VE is working as we expect it to: T221252

Open issues

Instructions

In the mobile visual editor, perform each of the actions listed below and verify that a VisualEditorFeatureUse event fires with the appropriate feature and action values (written in the table as feature:action). These events should fire in all sessions (not just a sample).

The easiest way to monitor events is by enabling the in-site monitoring UI (see instructions at https://www.mediawiki.org/wiki/Extension:EventLogging/Programming#Monitoring_events).

Testing an event

Follow these steps to test whether the right event is firing when tapping an existing link:

  1. View the mobile version of the following page on your laptop, in Chrome: https://en.wikipedia.org/wiki/Biological_life_cycle?trackdebug=1&useformat=mobile
  2. Open the inspector in Chrome
  3. Make sure you are on the “Console” tab within the inspector (you most likely will have landed in the “Element” tab)
  4. Tap an edit pencil
  5. Tap an existing link internal link
  6. Look in the browser's console to confirm the following event was fired: mf.schemaVisualEditorFeatureUse {feature: "link/internal", action: "context-show", editingSessionId: "88ef878111dc034c9c5c"}

Actions and Events

Opening of windows

ActionEventEvent fired (Y/N)Next Steps
Tap edit in an existing context itemwindow-open-from-contextYesNone. Re-tested after this patch was merged. Working properly.
A keyboard shortcut is used to open a context item [1]window-open-from-triggerSkip testing
Enter {{ to open a template dialogwindow-open-from-sequence/
Tap an edit tool in the editing toolbarwindow-open-from-toolYesNone. Re-tested after this patch was merged. Working properly.
Tap to add a new citation, tap "Manual" tab, tap "Book"window-open-from-commandNo@Ryasmeen file ticket; place here once filed. Done: T229255
Open an existing link, tap edit, tap "X" in the search dialogdialog-abortYes
Insert a new link, in "Link" dialog, enter new text into "Search pages" search field, tap "Done"dialog-applyYes dialog-done showed instead of dialog-apply. This behavior is expected.
  1. window-open-from-trigger: not technically desktop only, but requires an external bluetooth keyboard on mobile, so unlikely this will be used.

Context items

ActionEventEvent fired (Y/N)Next Steps
Tap an existing link, citation, image, table, etc. [2]context-showYes
Tap an existing annotation - link, citation, etc., tap "trash" or "unlink" iconcontext-clearWhat I am seeing is that context-clear only gets fired when I click on "unlink" icon, and for all "trash" icons it fires context-delete@DLynch to confirm these events are firing as they should.
Delete an existing node - table, etc.context-deleteYes
  1. Several events at once could be fired

Links

ActionEventEvent fired (Y/N)Next steps
Tap an existing internal link, tap "edit"link/internal:context-show then link:window-open-from-context Yes
Tap an existing external link, tap "edit"link:context-show then link:window-open-from-context Yes
With a link dialog open, type text into the "Search pages" search fieldlink:search-pages-input Yes
With a link dialog open, type text into the "External link" link fieldlink:external-link-input Yes
With the link dialog open, switch between the "Search pages" and "External link" tabslink:panel-switch Yes
Tap a link from the "Search pages" article search results listlink:search-pages-choose Yes
Tap "Done" from either the "Search pages" or "External link" tabslink:dialog-applyYes dialog-done showed instead of dialog-apply. This behavior is expected.

Citations

ActionEventEvent fired (Y/N)Next steps
Tap citation icon in toolbar,citoid:window-openYesNone. Re-tested after this patch was merged. Working properly.
Tap the "Automatic", "Manual" and "Re-use" tabscitoid:panel-switch Yes-
Tap the "Manual" tab, tap a template: "Website", "Book" etc.citoid:manual-choosecitoid:manual-choose firing correctly for all cases; however citoid:dialog-abort is also firing. This behavior is expected.After patch 526444, citoid:dialog-abort is replaced by citoid:dialog-manual-choose.
Tap citation icon in toolbar, tap "Automatic" tab (if not already selected), insert URL, tap "Generate"citoid:automatic-generatecitoid:automatic-generate firing correctly when tapping "Automatic" tab; however citoid panel-switch is also firing. This behavior is expected.
Tap "insert"citoid:automatic-insert Yes
Tap citation icon in toolbar, tap "Re-use" tab, tap an existing citationcitoid:reuse-choose Yes
Edit an existing citation, make changes in dialog, tap "Apply changes"citoid:dialog-applyYesNone. Re-tested after this patch was merged. Working properly.
Edit an existing citation, tap "X" in the dialog that appearscitoid:dialog-abort Yes

"Done"

  • We know which of the actions listed in the "Instrumentation" section of the **VE feature instrumentation spec are firing logging events in the way they should be.

Event Timeline

ppelberg created this task.Jul 13 2019, 1:56 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 13 2019, 1:56 AM
ppelberg renamed this task from Verify logging events are firing properly to Context items: verify logging events are firing properly.Jul 13 2019, 1:56 AM
ppelberg updated the task description. (Show Details)
ppelberg updated the task description. (Show Details)
ppelberg added a subscriber: nshahquinn-wmf.
ppelberg updated the task description. (Show Details)Jul 13 2019, 6:11 AM
nshahquinn-wmf added a subscriber: Mayakp.wiki.

We generally keep tasks off our board unless we're planning to do them or the whole team needs to track them 🙂

JTannerWMF added subscribers: marcella, Ryasmeen, JTannerWMF.

@ppelberg is awaiting feedback from @marcella of whether or not @Ryasmeen is going to QA this or if we should pursue another course of action.

Update:
@Ryasmeen will lead this testing. I'm reassigning this ticket to @Neil_P._Quinn_WMF to provide further guidance which is needed before work on this ticket can begin.


cc @JTannerWMF

@ppelberg what do you need from me here?

If you're looking for a listing of which events we need to check, that should be in the VE feature instrumentation spec. @DLynch, is that document up to date? And will it contain details on any instrumentation changes for the new edit cards? Because both Rummana and I will need an up-to-date list of which events are supposed to be sent when 😁

In T227933#5345580, @Neil_P._Quinn_WMF wrote:

@ppelberg what do you need from me here?

This wasn't explicit in the ticket. Let me try again. As I currently understand things, below is what is needed.

Needed from David L.:

Needed from Neil:

  • A list of steps Rummana will need to check to make sure the right logging events are being fired by the right user actions/inputs at the right times.
    • @Ryasmeen, is this question accurate? Are there any other questions you need answered before you can start testing?
nshahquinn-wmf removed nshahquinn-wmf as the assignee of this task.Jul 18 2019, 3:51 PM

Needed from Neil:

  • A list of steps Rummana will need to check to make sure the right logging events are being fired by the right user actions/inputs at the right times.
    • @Ryasmeen, is this question accurate? Are there any other questions you need answered before you can start testing?

The list of events and when they are supposed to be fire is in the feature instrumentation spec. Writing that was my last involvement in this project, so I am not sure whether it fully reflects things as @DLynch implemented them. He's best placed to confirm that if so or update the spec if not 😁

I believe that document is up to date.

Ryasmeen added a comment.EditedJul 18 2019, 6:24 PM

Needed from Neil:

  • A list of steps Rummana will need to check to make sure the right logging events are being fired by the right user actions/inputs at the right times.
    • @Ryasmeen, is this question accurate? Are there any other questions you need answered before you can start testing?

That is correct, maybe not for all events but atleast an example for one event.

ppelberg assigned this task to Ryasmeen.Jul 18 2019, 9:17 PM
ppelberg updated the task description. (Show Details)
ppelberg updated the task description. (Show Details)Jul 19 2019, 3:14 PM
Restricted Application added a subscriber: Liuxinyu970226. · View Herald TranscriptJul 19 2019, 3:14 PM
ppelberg updated the task description. (Show Details)Jul 19 2019, 7:00 PM
nshahquinn-wmf updated the task description. (Show Details)

@ppelberg thanks for starting a great list!

I've added some instructions on how to monitor the events and noted some open issues. @Ryasmeen if you need more info, please add it to the list.

@DLynch, the table in the description doesn't have the feature listed for most of the events (other than the Citoid ones). For the links, can you add whatever the feature is supposed to be? For opening of windows and context items, do you have any thoughts about which windows and context items Rummana should test?

For event-monitoring, the instructions there are only really helpful in 100% oversampling situations. Otherwise you have to keep reloading the page until you manage to wind up in the sample, which is an incredibly annoying testing requirement. (Or do all your testing on one of the wikis which has oversampling turned on -- which only applies to mobile.)

An alternate method which works is to apply the trackdebug=1 URL parameter. It causes all VE-related tracking events to get shunted into the browser console instead, and will work on desktop or mobile.

For opening of windows/context items... links, citations, templates. Anything else seems lower priority.

Known issue I've found: Citoid won't fire a window-open-from-tool event when you tap the "cite" button in the toolbar until https://gerrit.wikimedia.org/r/525324 is merged.

DLynch updated the task description. (Show Details)Jul 24 2019, 5:33 PM
ppelberg updated the task description. (Show Details)Jul 24 2019, 7:33 PM
DLynch updated the task description. (Show Details)Jul 24 2019, 7:36 PM

I updated the link in the description to include useful URL parameters, and to link to a non-protected page. 👍🏻

I updated the link in the description to include useful URL parameters, and to link to a non-protected page. 👍🏻

Awesome – thank you for looking those steps over, David. I'm going to remove the warning [1] from the description then.


  1. "Warning": ⚠️These steps need to be verified by @Neil_P._Quinn_WMF or @DLynch
ppelberg updated the task description. (Show Details)Jul 24 2019, 7:46 PM
Ryasmeen updated the task description. (Show Details)Jul 24 2019, 8:18 PM
Ryasmeen updated the task description. (Show Details)Jul 24 2019, 8:32 PM
Ryasmeen updated the task description. (Show Details)Jul 24 2019, 8:47 PM
Ryasmeen updated the task description. (Show Details)Jul 24 2019, 8:52 PM
Ryasmeen updated the task description. (Show Details)Jul 24 2019, 9:01 PM
Ryasmeen updated the task description. (Show Details)Jul 24 2019, 9:10 PM
Ryasmeen updated the task description. (Show Details)Jul 24 2019, 10:09 PM
Ryasmeen updated the task description. (Show Details)Jul 24 2019, 10:21 PM
Ryasmeen updated the task description. (Show Details)Jul 24 2019, 10:35 PM
Ryasmeen updated the task description. (Show Details)Jul 24 2019, 10:52 PM
ppelberg updated the task description. (Show Details)Jul 25 2019, 9:45 PM
ppelberg updated the task description. (Show Details)Jul 25 2019, 9:48 PM

@ppelberg and I have reviewed the test results for the events and added the open questions to the "Next Steps" column for each of them.

Ryasmeen updated the task description. (Show Details)Jul 29 2019, 5:25 PM
ppelberg added a comment.EditedJul 29 2019, 10:33 PM

After a quick chat with @JTannerWMF, it sounds like the following is true:

  1. The window-open events listed in the "Opening of windows" section of this task's description are not blocking Edit Cards v2 release. This is b/c answering the measurement questions we have listed in the description of T221252 depends on us being able to track the opening of context items, not windows.
  2. The actions and associated events listed in the "Citations" section of this task's description are behaving in the way we expect to be able to answer the measurement questions we have listed in the description of T221252.

Edit: 1. and 2. are still open

ppelberg updated the task description. (Show Details)Jul 30 2019, 12:18 AM
ppelberg updated the task description. (Show Details)Jul 30 2019, 12:20 AM

Change 526444 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/Citoid@master] On manual template selection give the close a descriptive action

https://gerrit.wikimedia.org/r/526444

Change 526444 merged by jenkins-bot:
[mediawiki/extensions/Citoid@master] On manual template selection give the close a descriptive action

https://gerrit.wikimedia.org/r/526444

matmarex updated the task description. (Show Details)Jul 30 2019, 7:26 PM
  1. The window-open events listed in the "Opening of windows" section of this task's description are not blocking Edit Cards v2 release. This is b/c answering the measurement questions we have listed in the description of T221252 depends on us being able to track the opening of context items, not windows.

That is my understanding of Neil's position. (I.e. the missing ones are generally off the toolbar buttons, rather that directly from the contexts of existing nodes.)

  1. The actions and associated events listed in the "Citations" section of this task's description are behaving in the way we expect to be able to answer the measurement questions we have listed in the description of T221252.

That is also my understanding of Neil's position.

Got it – thanks for looking these over, @DLynch.

ppelberg updated the task description. (Show Details)Aug 1 2019, 2:53 PM
matmarex moved this task from Inbox to High Priority on the Editing QA board.Aug 9 2019, 6:17 PM
matmarex added a subscriber: matmarex.

Looks like we just want to re-check the issues Rummana identified previously, which should now be fixed by David's patches.

Looks like we just want to re-check the issues Rummana identified previously, which should now be fixed by David's patches.

Re-tested the remaining cases, everything is working as expected now. I believe, this can now be closed @ppelberg?

ppelberg closed this task as Resolved.Sep 3 2019, 1:37 AM
ppelberg updated the task description. (Show Details)
ppelberg updated the task description. (Show Details)

Looks like we just want to re-check the issues Rummana identified previously, which should now be fixed by David's patches.

Re-tested the remaining cases, everything is working as expected now. I believe, this can now be closed @ppelberg?

Excellent, @Ryasmeen . I've updated the task description to reflect this.

Restricted Application added a project: User-Ryasmeen. · View Herald TranscriptSep 3 2019, 1:38 AM