Page MenuHomePhabricator
Create Task
Maniphest T348613

Implement Wiki-highlights microsite instrumentation
Closed, ResolvedPublic
Actions

Description

Background

To be able to measure “Engagement” of wiki-highlights through an A/B test, we would need to instrument the micro-sites. Our definition of engagement for this experiment is

  1. Time on site/ session length i.e. willingness to consume article summaries. (Primary metric)
  2. Summaries completion rate i.e. willingness to complete a summary. (Secondary)
  3. Number of summaries consumed per session i.e. willingness to view subsequent summaries.(Secondary)
  4. Popular topics i.e. which topics had majority of reads. (Secondary)

Details of the instrumentation spec can be viewed here

Test Scenarios

TBA

Details

SubjectRepoBranchLines +/-
Configure wiki-highlights experiment streamoperations/mediawiki-configmaster+5 -0
Create analytics/external/wiki_highlights_experimentschemas/event/secondarymaster+203 -0
Create mediawiki/wiki_highlights_experimentschemas/event/secondarymaster+104 -5
Customize query in gerrit

Related Objects
Search...

Event Timeline

PWaigi-WMF created this task.Oct 11 2023, 8:25 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 11 2023, 8:25 AM
PWaigi-WMF triaged this task as High priority.Oct 11 2023, 8:29 AM
PWaigi-WMF added a project: Wiki-Highlights.
PWaigi-WMF updated the task description. (Show Details)
PWaigi-WMF renamed this task from Wiki-highlights microsite instrumentation to Implement Wiki-highlights microsite instrumentation.Oct 11 2023, 2:26 PM
PWaigi-WMF updated the task description. (Show Details)Oct 11 2023, 2:36 PM
SBisson moved this task from Definition to Ready for Dev on the Inuka-Team (Kanban) board.Oct 11 2023, 3:31 PM
SBisson claimed this task.Oct 11 2023, 5:44 PM
SBisson moved this task from Ready for Dev to Dev on the Inuka-Team (Kanban) board.
PWaigi-WMF added a parent task: T339842: Wiki Highlights (hypothesis testing).Oct 12 2023, 7:46 AM
SBisson added a subscriber: cchen.Oct 13 2023, 1:11 PM

@cchen I have done most of the instrumentation code. Let me know when the schema is ready so we can start testing. Thanks!

gerritbot added a comment.Oct 16 2023, 10:05 PM

Change 966304 had a related patch set uploaded (by Conniecc1; author: Conniecc1):

[schemas/event/secondary@master] T348613 Add new wiki_highlights_experiments schema

https://gerrit.wikimedia.org/r/966304

gerritbot added a project: Patch-For-Review.Oct 16 2023, 10:05 PM
gerritbot added a comment.Oct 18 2023, 4:39 AM

Change 966658 had a related patch set uploaded (by Conniecc1; author: Conniecc1):

[schemas/event/secondary@master] T348613 add latest.yaml

https://gerrit.wikimedia.org/r/966658

gerritbot added a comment.Oct 18 2023, 4:40 AM

Change 966304 abandoned by Conniecc1:

[schemas/event/secondary@master] T348613 Add new wiki_highlights_experiments schema

Reason:

duplicate

https://gerrit.wikimedia.org/r/966304

gerritbot added a comment.Oct 19 2023, 6:48 PM

Change 966658 had a related patch set uploaded (by Bearloga; author: Conniecc1):

[schemas/event/secondary@master] Create mediawiki/wiki_highlights_experiment

https://gerrit.wikimedia.org/r/966658

PWaigi-WMF edited projects, added Wiki-Highlights (Microsite); removed Wiki-Highlights.Oct 26 2023, 12:37 PM
mpopov subscribed.Nov 2 2023, 10:01 PM

@cchen @PWaigi-WMF: can you please confirm that Legal & Security reviewed and approved the data collection / measurement plan for this experiment?

cchen added a comment.Nov 14 2023, 4:37 PM
In T348613#9303544, @mpopov wrote:

@cchen @PWaigi-WMF: can you please confirm that Legal & Security reviewed and approved the data collection / measurement plan for this experiment?

here's the request for legal review https://app.asana.com/0/1201418263074643/1205790739661148/f

gerritbot added a comment.Nov 16 2023, 8:22 PM

Change 966658 abandoned by Conniecc1:

[schemas/event/secondary@master] Create mediawiki/wiki_highlights_experiment

Reason:

need to rebase it

https://gerrit.wikimedia.org/r/966658

gerritbot added a comment.Nov 16 2023, 8:36 PM

Change 975079 had a related patch set uploaded (by Conniecc1; author: Conniecc1):

[schemas/event/secondary@master] Create mediawiki/wiki_highlights_experiment This schema is required for instrumenting the Wiki Highlights experiment that the Inuka team is working on. Bug: T348613

https://gerrit.wikimedia.org/r/975079

gerritbot added a comment.Nov 17 2023, 1:18 PM

Change 975079 had a related patch set uploaded (by Bearloga; author: Conniecc1):

[schemas/event/secondary@master] Create mediawiki/wiki_highlights_experiment

https://gerrit.wikimedia.org/r/975079

PWaigi-WMF changed the task status from Open to In Progress.Nov 17 2023, 5:36 PM
gerritbot added a comment.Nov 28 2023, 4:13 PM

Change 975079 merged by jenkins-bot:

[schemas/event/secondary@master] Create analytics/external/wiki_highlights_experiment

https://gerrit.wikimedia.org/r/975079

gerritbot added a comment.Nov 28 2023, 4:30 PM

Change 978096 had a related patch set uploaded (by Sbisson; author: Sbisson):

[operations/mediawiki-config@master] Configure wiki-highlights experiment stream

https://gerrit.wikimedia.org/r/978096

gerritbot added a comment.Nov 29 2023, 2:11 PM

Change 978096 merged by jenkins-bot:

[operations/mediawiki-config@master] Configure wiki-highlights experiment stream

https://gerrit.wikimedia.org/r/978096

Stashbot added a comment.Nov 29 2023, 2:12 PM

Mentioned in SAL (#wikimedia-operations) [2023-11-29T14:12:36Z] <lucaswerkmeister-wmde@deploy2002> Started scap: Backport for [[gerrit:978096|Configure wiki-highlights experiment stream (T348613)]]

Stashbot added a comment.Nov 29 2023, 2:36 PM

Mentioned in SAL (#wikimedia-operations) [2023-11-29T14:36:49Z] <lucaswerkmeister-wmde@deploy2002> sbisson and lucaswerkmeister-wmde: Backport for [[gerrit:978096|Configure wiki-highlights experiment stream (T348613)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Stashbot added a comment.Nov 29 2023, 2:55 PM

Mentioned in SAL (#wikimedia-operations) [2023-11-29T14:55:34Z] <lucaswerkmeister-wmde@deploy2002> Finished scap: Backport for [[gerrit:978096|Configure wiki-highlights experiment stream (T348613)]] (duration: 42m 58s)

cchen mentioned this in T352317: QA Wiki Highlights Instrumentation.Nov 29 2023, 4:32 PM
SBisson moved this task from Dev to Code Review on the Inuka-Team (Kanban) board.Nov 29 2023, 4:52 PM
SBisson added a comment.Nov 30 2023, 2:47 PM

Initial data testing documented in T352317: QA Wiki Highlights Instrumentation

SBisson moved this task from Code Review to QA on the Inuka-Team (Kanban) board.Dec 6 2023, 2:01 PM
SBisson removed a project: Patch-For-Review.Dec 14 2023, 4:20 PM
SBisson moved this task from QA to Design sign off on the Inuka-Team (Kanban) board.Jan 10 2024, 3:36 PM
SGautam_WMF moved this task from Design sign off to Product sign off on the Inuka-Team (Kanban) board.Jan 11 2024, 8:36 AM
PWaigi-WMF closed this task as Resolved.Jan 12 2024, 9:33 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL