Page MenuHomePhabricator

Instrument the Translation activity for the persistent entry point
Closed, ResolvedPublic

Description

As part of the instrumentation work on Content/Section Translation entry points we want to register when users access the tool by using the "Contribute" persistent entry point (T286466). This will allow to know how frequently translation is accessed and how much of the translation activity is generated through it.

Event Timeline

Pginer-WMF created this task.

Change 880494 had a related patch set uploaded (by Nik Gkountas; author: Nik Gkountas):

[mediawiki/extensions/ContentTranslation@master] SX: Instrument Contributions page entrypoint

https://gerrit.wikimedia.org/r/880494

@santhosh should we take any actions for instrumenting this entrypoint for ContentTranslation or it's something that is done automatically?

@santhosh should we take any actions for instrumenting this entrypoint for ContentTranslation or it's something that is done automatically?

campaign parameters will be already event logged in CX

Change 880494 merged by jenkins-bot:

[mediawiki/extensions/ContentTranslation@master] SX: Instrument Contributions page entrypoint

https://gerrit.wikimedia.org/r/880494

Change 882594 had a related patch set uploaded (by Santhosh; author: Santhosh):

[mediawiki/extensions/ContentTranslation@master] CX3 Build 0.2.0+20230123

https://gerrit.wikimedia.org/r/882594

Change 882594 merged by jenkins-bot:

[mediawiki/extensions/ContentTranslation@master] CX3 Build 0.2.0+20230123

https://gerrit.wikimedia.org/r/882594

Pginer-WMF raised the priority of this task from Medium to High.Jan 25 2023, 8:22 AM

@ngkountas - Should we start to see events for this new entry point logged in content_translation_event? I did a quick query of the dataset and noticed that we are still not logging any events for this entry point.

Here is a summary of the number of events by event source type logged from 23 Jan 2023 through 14 Feb 2023:

event_sourcen_events
content_language_selector3513
direct2569
direct_preselect909
frequent_languages11487
invite_new_article_creation944
recent_translation11

I confirmed that this event source value (event_source = contributions_page) has already been added to the schema so I don't believe that's the issue.

@ngkountas - Should we start to see events for this new entry point logged in content_translation_event? I did a quick query of the dataset and noticed that we are still not logging any events for this entry point.

The entry point is not fully deployed yet since we were waiting for instrumentation. Although it cannot be reached yet through the Contributions page, it can be accessed directly. I made a test on Bengali Wikipedia on mobile and desktop which is expected to have generated at least 2 events for the translate option.

The links used for the "translate" option include a campaign parameter ("campaign=specialcontribute"). This seems to be using the URL campaign approach so it may not be reflected in the events. Based on that:

  • @ngkountas is there a way in which the campaigns can be queried to check how many people accessed them? This would be useful to check if the current instrumentation works as expected.
  • @MNeisler the new instrumentation schema was accounting for capturing this kind of campaigns, do you know how to best reflect those there? I see the schema has both "utm_campaign" (maybe that is more for editing events more than tool integration?) and "event_source". Based on that I can create a ticket for Url Campaigns to be reflected in the new instrumentation.

@MNeisler the new instrumentation schema was accounting for capturing this kind of campaigns, do you know how to best reflect those there? I see the schema has both "utm_campaign" (maybe that is more for editing events more than tool integration?) and "event_source". Based on that I can create a ticket for Url Campaigns to be reflected in the new instrumentation.

@Pginer-WMF My understanding was that this campaign parameter should already be instrumented in the new instrumentation schema (content_translation_event). The event_source in content_translation_event is based on the campaign URL param. In this case, the event source for the "campaign=specialcontribute" parameter is event_source = contributions_page, which has already been identified as a possible value in the current schema. Although I still have not seen any events logged with this event_source so it doesn't seem to be capturing these events yet.

According to schema documentation, the utm_campaign field is meant to "The value of the URL's utm_campaign query parameter, used to identify the campaign responsible for the traffic". I believe this is more focused on capturing editing events but can also be used to track the specific value of campaign parameters (e.g specialcontribute in this case) if needed. This part does not appear to be instrumented yet.

Change 905591 had a related patch set uploaded (by Nik Gkountas; author: Nik Gkountas):

[mediawiki/extensions/ContentTranslation@master] SX instrumentation: Always take "campaign" URL parameter into account

https://gerrit.wikimedia.org/r/905591

Change 905591 merged by jenkins-bot:

[mediawiki/extensions/ContentTranslation@master] SX instrumentation: Always take "campaign" URL parameter into account

https://gerrit.wikimedia.org/r/905591

I've rechecked the events logged in content_translation_event and am still currently not seeing any events for this entry point since the new patch was merged. @Pginer-WMF mentioned he triggered a couple of events that we're expecting to see logged here.

I've provided the query I'm running and the resulting data below for reference. Let me know if there's any other info that would be useful in help troubleshooting.

Entry points logged since 12 April 2023

event_sourceevent_typeaccess_methodn_events
content_language_selectordashboard_openmobile web924
directdashboard_openmobile web2389
direct_preselectdashboard_openmobile web299
frequent_languagesdashboard_openmobile web7817
invite_new_article_creationdashboard_openmobile web492
recent_translationdashboard_openmobile web71

Data via:

"SELECT 
event_source,
event_type,
access_method, 
COUNT(1) as n_events
FROM 
event.mediawiki_content_translation_event
WHERE
YEAR = 2023 
AND MONTH = 04
AND DAY >= 12
GROUP BY
event_source,
event_type,
access_method
"

cc @ngkountas

@MNeisler although the patch has been merged, there has been no CX build since then, so the changes are not yet deployed. I have wrongfully informed @Pginer-WMF that the changes should be visible by now, but that is not the case, yet. Sorry for providing misleading input!

No problem @ngkountas. Thanks for the update! I'll plan to recheck once the changes are deployed and some test events have been generated.

Change 911252 had a related patch set uploaded (by Nik Gkountas; author: Nik Gkountas):

[mediawiki/extensions/ContentTranslation@master] CX3 Build 0.2.0+20230424

https://gerrit.wikimedia.org/r/911252

Change 911252 merged by jenkins-bot:

[mediawiki/extensions/ContentTranslation@master] CX3 Build 0.2.0+20230424

https://gerrit.wikimedia.org/r/911252

I rechecked events logged in content_translation_event and confirmed that we are now seeing events for this entry point.

There have only been 2 event_source = 'contributions_page' events logged to date since the cx build was deployed on 27 April 2023. However, all associated data with these events appear to be logging correctly. Let me know if you need any further details.

I rechecked events logged in content_translation_event and confirmed that we are now seeing events for this entry point.

There have only been 2 event_source = 'contributions_page' events logged to date since the cx build was deployed on 27 April 2023. However, all associated data with these events appear to be logging correctly. Let me know if you need any further details.

Perfect. Thanks @MNeisler. Can you confirm that the events detected cover both mobile and desktop?
If that is the case, I guess we can close this ticket and create a new one to measure initial events after the deployment (T319362) when the feature will be visible to users.

Can you confirm that the events detected cover both mobile and desktop?

@Pginer-WMF
There have been 5 total events logged for this new entry point (event_source = 'contributions_page') but I'm not seeing any events recorded as occurring on desktop. All 5 events have been on mobile web. I've provided some additional event details below in case helpful.

datewiki_dbtranslation_typetranslation_target_languagetranslation_source_languageevent_sourceaccess_method
2023-04-27testwikisectionelencontributions_pagemobile web
2023-04-28bnwikisectionbnencontributions_pagemobile web
2023-05-16bnwikisectionbnencontributions_pagemobile web
2023-05-06yowikisectionyoencontributions_pagemobile web
2023-05-06yowikisectionyoencontributions_pagemobile web

Thanks Megan!
I made the tests on Bengali Wikipedia on mobile and desktop which is expected to have generated events for each platform. @ngkountas any thoughts about what may be happening?

Change 934486 had a related patch set uploaded (by Nik Gkountas; author: Nik Gkountas):

[mediawiki/extensions/ContentTranslation@master] CX eventlogging: Add instrumentation for "specialcontribute" entrypoint

https://gerrit.wikimedia.org/r/934486

@Pginer-WMF I have submitted a patch about fixing the issue for CX. It is still under review. Until this patch is merged and deployed, the persistent entrypoint won't be instrumented for desktop.

Change 934486 merged by jenkins-bot:

[mediawiki/extensions/ContentTranslation@master] CX eventlogging: Add instrumentation for "specialcontribute" entrypoint

https://gerrit.wikimedia.org/r/934486

The "specialcontribute" entrypoint is now instrumented in production for both desktop and mobile devices.

Screencast of the events being logged in bn production wiki:

Thanks @ngkountas!

I reviewed the data logged in mediawiki_content_translation_event and confirmed that data for the new "specialcontribute" entrypoint (event_source = 'contributions_page') appear to be logging as expected and align with the events shown in the screencast provided above.

Below are some initial results of data logged for this event since 7 September 2023

By date and access_method

dateevent_sourceaccess_methodn_events
2023-09-07contributions_pagedesktop5
2023-09-07contributions_pagemobile web6
2023-09-08contributions_pagedesktop3
2023-09-08contributions_pagemobile web2

By translation type and language pair

event_sourcetranslation_typetranslation_source_languagetranslation_target_languageevent_typeaccess_methodn_events
contributions_pagearticlebnendashboard_opendesktop2
contributions_pagearticleenbndashboard_opendesktop5
contributions_pagearticlefrbndashboard_opendesktop1
contributions_pagesectionenbndashboard_openmobile web8

(cc @Pginer-WMF, @KCVelaga_WMF )