@LZaman

Attached is the documentation of the proposed talk_page_event schema [i], which can be forwarded to legal for their review. The attached documentation includes a description of the new schema, the purpose for collecting these events, and a description of all the fields that will be tracked.

In your request to legal, you can also let them know that I am available to walk through the new schema in a meeting if they prefer (They've sometimes requested this in the past to help understand the type of data being collected). Let me know if there's anything else you need for the request.

(cc @ppelberg )

i. Note: This reflects the same fields documented in the instrumentation spec but presented in a clearer format for legal review and without the details on existing instrumentation and metrics that will be used in the project.

Change 731333 had a related patch set uploaded (by DLynch; author: DLynch):

[schemas/event/secondary@master] talk_page_event schema

https://gerrit.wikimedia.org/r/731333

Change 731334 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/DiscussionTools@master] Logging for new comments

https://gerrit.wikimedia.org/r/731334

Change 731854 had a related patch set uploaded (by DLynch; author: DLynch):

[operations/mediawiki-config@master] Add event stream config for discussiontools

https://gerrit.wikimedia.org/r/731854

Change 731860 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/VisualEditor@master] Pass editingSessionId through to API save requests

https://gerrit.wikimedia.org/r/731860

Change 731333 merged by jenkins-bot:

[schemas/event/secondary@master] talk_page_edit schema

https://gerrit.wikimedia.org/r/731333

Change 731860 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Pass editingSessionId through to API save requests

https://gerrit.wikimedia.org/r/731860

Change 731334 merged by jenkins-bot:

[mediawiki/extensions/DiscussionTools@master] Logging for new comments

https://gerrit.wikimedia.org/r/731334

Change 731854 merged by jenkins-bot:

[operations/mediawiki-config@master] Add event stream config for discussiontools

https://gerrit.wikimedia.org/r/731854

Mentioned in SAL (#wikimedia-operations) [2021-11-02T00:02:17Z] <legoktm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Add event stream config for discussiontools (T286076) (duration: 00m 55s)

QA has verified the new instrumentation/events are being emitted from clients in expected ways

Per what @DLynch shared in today's – 1 November – team standup, the above is not necessary because the new instrumentation added as part of this task only impacts server-side code. cc @Ryasmeen

The config patch that merged this evening was necessary for the deployment -- it just hooked up the event stream so the logging knew what to do once events start appearing when the train goes out.

I've updated the talk_page_edit schema spec document that Megan provided with the actual field names we wound up using.

Hi @MNeisler ! This is ready for you to QA by Nov 4.

I've completed a quick check and confirmed that the new mediawiki_talk_page_edit schema is logging events as of 2021 November 03. To date, we've logged about 97,543 events across all wikis.

@DLynch I noticed on logstash that we are getting some validation errors around the performer.user_edit_count_bucket field. See dashboard and details below:

'.performer.user_edit_count_bucket' should be string, '.performer.user_edit_count_bucket' should be equal to one of the allowed values

This appears to occur for events by anon users where the performer.user_edit_count_bucket is assigned to null.

'.performer.user_edit_count_bucket' should be equal to one of the allowed values

This occurs for logged-in users that fall into the 1-4 edit bucket count. Taking a look at the schema, it looks like there might be an extra space in the `1-4 edits ' enum that is causing this.

Keeping this assigned to me right now as I'm still double-checking that the other fields are populating as expected, but at quick glance, everything else looks good. I will follow-up on this task once I've completed the rest of the QA.

Taking a look at the schema, it looks like there might be an extra space in the `1-4 edits ' enum.

Amusingly enough, this is because I copied that entire field-definition from the existing content_translation_event schema... which thus also suffers from this problem. 😁

This occurs for events by anon users where the performer.user_edit_count_bucket is assigned to null.

I think this would also occur for events in content_translation_event. Would you like me to make it log them as "0 edits" even though that's arguably not-accurate?

Is there any instance we would expect to see component_type = topic? If not, I'd recommend removing this from the enum list.

Nope, nothing would currently cause that. I only put it in the enum because it was in the spec and I figured you might have some future reason for it to be there.

Would it be possible to make sure all mediawiki_talk_page_edit events are also sampled in EditAttemptStep?

I think what I have to do here is turn down how many events are being sampled for integration = page -- I can see that the current override for integration = discussiontools is applying to it, so I should make that sampling only apply when it really should be. I don't think I can 100% guarantee there'll always be 1:1 presence for EditAttemptStep and talk_page_edit sessions, just because of data that's not available when the talk_page_edit logging is happening... but it can be closer.

In T286076#7512394, @DLynch wrote:

Taking a look at the schema, it looks like there might be an extra space in the `1-4 edits ' enum.

Amusingly enough, this is because I copied that entire field-definition from the existing content_translation_event schema... which thus also suffers from this problem. 😁

The good news is that, through manual materialization, we can actually fix the typos in both schemas without bumping the version. Structurally nothing changes so if we delete all the files except current.yaml, update that (without changing the version specified), and run ./node_modules/.bin/jsonschema_tools materialize jsonschema/analytics/mediawiki/talk_page_edit/current.yaml it'll be fine.

This occurs for events by anon users where the performer.user_edit_count_bucket is assigned to null.

I think this would also occur for events in content_translation_event. Would you like me to make it log them as "0 edits" even though that's arguably not-accurate?

I would recommend adding an explicit 'N/A' to the enum to use with IP editors. This too can be a same-version modification (via the process described above). Quick note: if we want to make the addition through a new version then it would have to be appended to the end of the enum.

Change 739865 had a related patch set uploaded (by DLynch; author: DLynch):

[schemas/event/secondary@master] Update talk_page_edit schema

https://gerrit.wikimedia.org/r/739865

Change 739884 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/DiscussionTools@master] Fixes for talk_page_edit logging

https://gerrit.wikimedia.org/r/739884

Change 739865 merged by jenkins-bot:

[schemas/event/secondary@master] Update talk_page_edit schema

https://gerrit.wikimedia.org/r/739865

Thanks @DLynch! The changes look good to me.

I'll do a final QA check once the new changes are deployed to make sure that anon and 1-4 edit count bucket events are now logging correctly.

Change 739884 merged by jenkins-bot:

[mediawiki/extensions/DiscussionTools@master] Fixes for talk_page_edit logging

https://gerrit.wikimedia.org/r/739884

Note: QA on this task is still blocked considering these fixes were included in Mediawiki versions .10 and onwards which has not yet reached production wikis.