Thanks @phuedx for the sample events. It would be great to see some events with new "inputLocation": "header-moved", but since @Edtadros is doing a local test we can look at it from his test results.

<Provide link to Data QA checks document once it is created>

thank you @MNeisler ! for your review :)

09/15 Meeting Notes: SearchSatisfaction Instrumentation & Data-QA

awesome, thanks @MNeisler for the feedback. :)

Moving this to Backlog, to pick up after I get back from Parental Leave (in dec)

Stretch goal for Q1.
Assigning to @cchen as I will be on parental leave starting Sep 21.

We've been able to get a few recommendations from stakeholders as a part of the Content Dataset Questionnaire.

Created QC document, updated link in description : Search Feature QC : Instrumentation QA and data checks

Currently drafting the Requirement Document : Content Data Accessibility

Requested @JTannerWMF to create Phab task template.

Hi @ppelberg, since data-QA is completed and the additional instrumentation work has been captured in T259673, I believe the scope of this task has been completed. We can go ahead and resolve it.

Updated VisualEditor/FeatureUse_data_dictionary
Added the following features:
mwUsernameCompletion

• Tagging a user in a talk page comment - using @mention or clicking an icon

Status and Scenarios tested :
Replying QC Template Post Deployment Data QA
Link to noteboks-
Data QA_Part 1_DiscussionTools
Data QA_Part 2_VEfu-new-fields

Completed Data-QA. Verdict: Data in both the schemas look good!
Here are some notes/observations.

Reviewed the QA-Instrumentation Checklist Phab Form with Kate and Jazmin.
Sent the content of the Data-QA checklist that we would like included in the Phab task template and added to Guidelines document, to Jazmin.
Next Up:

1. Present the template to Editing and Web teams
2. Document this in the QA Process documentation T228228

Checked the data in Schema:DesktopWebUIActionsTracking and everything looks good. Events from test wikis are in the schema as expected.
Results posted in this notebook.

Created and sent out Questionnaire for inputs from stakeholders on July 31.
Responses received are being added to the Use Cases section for technical translation.

@cchen and @Mayakp.wiki are reviewing articles and topics to determine accuracy of the 2 models we discussed :
Wikidata based topic dataset sample review
Outlinks based topic dataset sample review

In T258058#6331041, @Jdlrobson wrote:

@Mayakp.wiki reports that we should expect around 362 init events per sec from French Wikipedia so I think we're good to continue here with the 20% sampling rate.

Next steps

• Assuming everything is working as expected [i], @Mayakp.wiki is there any work left to be done on this task on your end (e.g. confirm the events/fields are being logged in the DB correctly)?

---

i. "Working as expected": as evidenced by all of the events in rows 182-204 being marked as Passed.

The number of events have picked up as of today. Here's the break up by day and wiki.

@alexhollender : Can you please file a task, set a priority and tag Product-Analytics ? We can triage this request tomorrow in our Board Refinement meeting.

Currently reviewing T238449 with Kate, Jazmin etc. before publishing it to Product teams.

Closing this task since we have started deploying on production test wikis.

• T255727 is complete, tested in T257909
• Donate (and other external links) T258513 : observed that they are not logged in Network tab but they are making it to the db.

Post deployment data-QA is being done in T256075

On July 23 @Edtadros helped fire many events to external links on Testwiki to check on this issue and I can confirm that all events have made it to the database.
Posted initial results here
We will be checking on the test wikis listed here T258058 in the next 2 weeks.

@Ryasmeen : I have added the events we finalized to our QC document and provided the exact section so you can find it easily.
pls feel free to reach out if you have any questions!

I have done some preliminary checks on the data and posted results here : https://github.com/wikimedia-research/Collapsible-Sidebar-Analysis/blob/master/Post%20Deployment%20Data%20QA%20for%20Collapsible%20Sidebar%20Instrumentation.ipynb

@Ryasmeen I have added a new section to the QC document and provided instructions in the Test Description of this task.
We are primarily looking at the data being stored in the 6 new fields. For that, I've added a few scenarios where you can trigger specific actions to validate them.
Please feel free to ping/chat or let me know if you have any questions.

Hi @CGlenn, thank you for opening this task and letting us know about its priority. We will triage this task during our next Board Refinement meeting on Tuesday 7/21 and provide you an update.

Observations :

• We are unable to check Donate link on Testwiki

Jon has added the link to testwiki for us. Thanks Jon ! :)

• Collapsed/Uncollapsed state of the sidebar is sometimes persistent, sometimes not

This is dependent on the deployment of T255727 and yet to be completed. Edward will re-test once this is fixed.

I will be assisting @Edtadros with this task to check events in the Data Lake.

Posting an update on this task: During our weekly 1:1s, Peter and Megan are actively brainstorming on this problem statement :

Sent to Jazmin Tanner to review. Will review with Kate in the next 1:1
https://docs.google.com/document/d/1K4Sqn-LgrgHkfXE_WuaRrKqjxWSgBkl-6ZqbWGqpsD8/edit#heading=h.6rt4ek2j66xx

@jwang : We have been discussing this in our 1:1s with the product teams. @MNeisler will be investigating this for Web and Editing.

In T256992#6275396, @Krinkle wrote:

Is there a tracking task for the larger initiative this is a part of so that others can understand what it is for and how it is used?

@ppelberg I agree. Its best to record it in the current document. @Ryasmeen should test all the editing interfaces in each test scenario under the Production section of the Replying QC Template.
If she sees an unexpected value in any of the fields she can mark that scenario as 'failed' and explain the bug in the issue description (this should work even if she encounters issues with one scenario in multiple editing interfaces).

Discussed the below items in our instrumentation meeting today:

I don't believe this takes into account GlobalPreferences, correct? That would mean we're potentially missing some highly active cross-wiki users.

June 25, 2020: new Open question have come up

1. name of "donate" link

Every sidebar is different depending on the wiki. It’s defined by https://en.wikipedia.org/wiki/MediaWiki:Sidebar . As you can see there donate is defined as “sitesupport-url|sitesupport” which means we will see name=n-sitesupport for this specific link. This is not an ideal value to be stored in the database table.
Action: @Mayakp.wiki @has asked if we can transform the name. Jdlrobson is seeing if it's possible to change just the donate link and at what cost.

2. Sidebar instrumentation gotcha - any community liaison support needed?

We will be discussing on email to get to a consensus and then posting updates here, if any.

@Nuria , I did reach out to @Ottomata back in May when we were designing the new schema. I would like to clarify that after discussions with @ovasileva and the web team's engineers we eventually decided to go with legacy EL as mentioned in my earlier comment-

hey @ovasileva , can we have this task updated to include the use of VectorSkinVersion that we spoke about in our last 1:1 ?

Thanks @DLynch for the clarification and update.
I have added a new test scenario for when you type in the field inside the link window. Can you confirm if its correct?

When user clicks on "Link" i.e. "🔗" icon to add a hyperlink on selected text and types in the field inside the link window

"feature": "link"
"action": "external-link-input"

Hi @ppelberg, I agree, it would be better if David could answer this.
sidenote: not sure why but I never got an email or notified for the ping on June 18. sorry about that.

Duplicate task, created by mistake.

Next Up: From meetings with @JTannerWMF it is required to make a Phab ticket checklist of all the Data QA steps that need to be performed and attach it to an instrumentation task - to get the various people involved in the instrumentation QA to do the things, update their statuses and keep the analyst informed.