As part of the instrumentation work on Content/Section Translation entry points we want to register when users access the tool by using the "Contribute" persistent entry point (T286466). This will allow to know how frequently translation is accessed and how much of the translation activity is generated through it.
Description
Details
Status | Subtype | Assigned | Task | ||
---|---|---|---|---|---|
Open | None | T286641 Entry points to improve discoverability of Section Translation | |||
Open | None | T322740 Contribute entry point | |||
Resolved | ngkountas | T321386 Instrument the Translation activity for the persistent entry point |
Event Timeline
Change 880494 had a related patch set uploaded (by Nik Gkountas; author: Nik Gkountas):
[mediawiki/extensions/ContentTranslation@master] SX: Instrument Contributions page entrypoint
@santhosh should we take any actions for instrumenting this entrypoint for ContentTranslation or it's something that is done automatically?
Change 880494 merged by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] SX: Instrument Contributions page entrypoint
Change 882594 had a related patch set uploaded (by Santhosh; author: Santhosh):
[mediawiki/extensions/ContentTranslation@master] CX3 Build 0.2.0+20230123
Change 882594 merged by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] CX3 Build 0.2.0+20230123
@ngkountas - Should we start to see events for this new entry point logged in content_translation_event? I did a quick query of the dataset and noticed that we are still not logging any events for this entry point.
Here is a summary of the number of events by event source type logged from 23 Jan 2023 through 14 Feb 2023:
event_source | n_events |
---|---|
content_language_selector | 3513 |
direct | 2569 |
direct_preselect | 909 |
frequent_languages | 11487 |
invite_new_article_creation | 944 |
recent_translation | 11 |
I confirmed that this event source value (event_source = contributions_page) has already been added to the schema so I don't believe that's the issue.
The entry point is not fully deployed yet since we were waiting for instrumentation. Although it cannot be reached yet through the Contributions page, it can be accessed directly. I made a test on Bengali Wikipedia on mobile and desktop which is expected to have generated at least 2 events for the translate option.
The links used for the "translate" option include a campaign parameter ("campaign=specialcontribute"). This seems to be using the URL campaign approach so it may not be reflected in the events. Based on that:
- @ngkountas is there a way in which the campaigns can be queried to check how many people accessed them? This would be useful to check if the current instrumentation works as expected.
- @MNeisler the new instrumentation schema was accounting for capturing this kind of campaigns, do you know how to best reflect those there? I see the schema has both "utm_campaign" (maybe that is more for editing events more than tool integration?) and "event_source". Based on that I can create a ticket for Url Campaigns to be reflected in the new instrumentation.
@MNeisler the new instrumentation schema was accounting for capturing this kind of campaigns, do you know how to best reflect those there? I see the schema has both "utm_campaign" (maybe that is more for editing events more than tool integration?) and "event_source". Based on that I can create a ticket for Url Campaigns to be reflected in the new instrumentation.
@Pginer-WMF My understanding was that this campaign parameter should already be instrumented in the new instrumentation schema (content_translation_event). The event_source in content_translation_event is based on the campaign URL param. In this case, the event source for the "campaign=specialcontribute" parameter is event_source = contributions_page, which has already been identified as a possible value in the current schema. Although I still have not seen any events logged with this event_source so it doesn't seem to be capturing these events yet.
According to schema documentation, the utm_campaign field is meant to "The value of the URL's utm_campaign query parameter, used to identify the campaign responsible for the traffic". I believe this is more focused on capturing editing events but can also be used to track the specific value of campaign parameters (e.g specialcontribute in this case) if needed. This part does not appear to be instrumented yet.
Change 905591 had a related patch set uploaded (by Nik Gkountas; author: Nik Gkountas):
[mediawiki/extensions/ContentTranslation@master] SX instrumentation: Always take "campaign" URL parameter into account
Change 905591 merged by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] SX instrumentation: Always take "campaign" URL parameter into account
I've rechecked the events logged in content_translation_event and am still currently not seeing any events for this entry point since the new patch was merged. @Pginer-WMF mentioned he triggered a couple of events that we're expecting to see logged here.
I've provided the query I'm running and the resulting data below for reference. Let me know if there's any other info that would be useful in help troubleshooting.
Entry points logged since 12 April 2023
event_source | event_type | access_method | n_events |
---|---|---|---|
content_language_selector | dashboard_open | mobile web | 924 |
direct | dashboard_open | mobile web | 2389 |
direct_preselect | dashboard_open | mobile web | 299 |
frequent_languages | dashboard_open | mobile web | 7817 |
invite_new_article_creation | dashboard_open | mobile web | 492 |
recent_translation | dashboard_open | mobile web | 71 |
Data via:
"SELECT event_source, event_type, access_method, COUNT(1) as n_events FROM event.mediawiki_content_translation_event WHERE YEAR = 2023 AND MONTH = 04 AND DAY >= 12 GROUP BY event_source, event_type, access_method "
cc @ngkountas
@MNeisler although the patch has been merged, there has been no CX build since then, so the changes are not yet deployed. I have wrongfully informed @Pginer-WMF that the changes should be visible by now, but that is not the case, yet. Sorry for providing misleading input!
No problem @ngkountas. Thanks for the update! I'll plan to recheck once the changes are deployed and some test events have been generated.
Change 911252 had a related patch set uploaded (by Nik Gkountas; author: Nik Gkountas):
[mediawiki/extensions/ContentTranslation@master] CX3 Build 0.2.0+20230424
Change 911252 merged by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] CX3 Build 0.2.0+20230424
I rechecked events logged in content_translation_event and confirmed that we are now seeing events for this entry point.
There have only been 2 event_source = 'contributions_page' events logged to date since the cx build was deployed on 27 April 2023. However, all associated data with these events appear to be logging correctly. Let me know if you need any further details.
Can you confirm that the events detected cover both mobile and desktop?
@Pginer-WMF
There have been 5 total events logged for this new entry point (event_source = 'contributions_page') but I'm not seeing any events recorded as occurring on desktop. All 5 events have been on mobile web. I've provided some additional event details below in case helpful.
date | wiki_db | translation_type | translation_target_language | translation_source_language | event_source | access_method |
2023-04-27 | testwiki | section | el | en | contributions_page | mobile web |
2023-04-28 | bnwiki | section | bn | en | contributions_page | mobile web |
2023-05-16 | bnwiki | section | bn | en | contributions_page | mobile web |
2023-05-06 | yowiki | section | yo | en | contributions_page | mobile web |
2023-05-06 | yowiki | section | yo | en | contributions_page | mobile web |
Thanks Megan!
I made the tests on Bengali Wikipedia on mobile and desktop which is expected to have generated events for each platform. @ngkountas any thoughts about what may be happening?
Change 934486 had a related patch set uploaded (by Nik Gkountas; author: Nik Gkountas):
[mediawiki/extensions/ContentTranslation@master] CX eventlogging: Add instrumentation for "specialcontribute" entrypoint
@Pginer-WMF I have submitted a patch about fixing the issue for CX. It is still under review. Until this patch is merged and deployed, the persistent entrypoint won't be instrumented for desktop.
Change 934486 merged by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] CX eventlogging: Add instrumentation for "specialcontribute" entrypoint
The "specialcontribute" entrypoint is now instrumented in production for both desktop and mobile devices.
Screencast of the events being logged in bn production wiki:
Thanks @ngkountas!
I reviewed the data logged in mediawiki_content_translation_event and confirmed that data for the new "specialcontribute" entrypoint (event_source = 'contributions_page') appear to be logging as expected and align with the events shown in the screencast provided above.
Below are some initial results of data logged for this event since 7 September 2023
By date and access_method
date | event_source | access_method | n_events |
---|---|---|---|
2023-09-07 | contributions_page | desktop | 5 |
2023-09-07 | contributions_page | mobile web | 6 |
2023-09-08 | contributions_page | desktop | 3 |
2023-09-08 | contributions_page | mobile web | 2 |
By translation type and language pair
event_source | translation_type | translation_source_language | translation_target_language | event_type | access_method | n_events |
---|---|---|---|---|---|---|
contributions_page | article | bn | en | dashboard_open | desktop | 2 |
contributions_page | article | en | bn | dashboard_open | desktop | 5 |
contributions_page | article | fr | bn | dashboard_open | desktop | 1 |
contributions_page | section | en | bn | dashboard_open | mobile web | 8 |
(cc @Pginer-WMF, @KCVelaga_WMF )