Page MenuHomePhabricator

nettrom_WMF (Morten Warncke-Wang)
Staff Data Scientist, Product Analytics

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Thursday

  • Clear sailing ahead.

User Details

User Since
Aug 21 2018, 8:23 PM (290 w, 6 d)
Availability
Available
IRC Nick
Nettrom
LDAP User
Unknown
MediaWiki User
MWang (WMF) [ Global Accounts ]

Recent Activity

Tue, Mar 5

nettrom_WMF moved T311531: Add an Image: experiment analysis from Incoming to Doing on the Growth-Team (Sprint 9 (Growth Team)) board.
Tue, Mar 5, 10:04 PM · Growth-Team (Sprint 9 (Growth Team)), Growth-Structured-Tasks, Image-Suggestions, Product-Analytics (Kanban)

Tue, Feb 27

nettrom_WMF added a comment to T300273: [IP Masking] Temporary account to registered account creation flow.

I agree with what's being said here. The performer values would reflect what the current user is, and if that user successfully logs in it'll reflect their logged in status. If the login attempt fails, it'll reflect their either anon or temp user status.

Tue, Feb 27, 1:44 AM · Growth-Team (Sprint 9 (Growth Team)), Temporary accounts, MediaWiki-User-login-and-signup, IP-Masking-Growth-Team, MW-1.41-notes (1.41.0-wmf.30; 2023-10-10), MW-1.42-notes (1.42.0-wmf.4; 2023-11-07), Patch-For-Review

Feb 5 2024

nettrom_WMF moved T311531: Add an Image: experiment analysis from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Feb 5 2024, 7:17 PM · Growth-Team (Sprint 9 (Growth Team)), Growth-Structured-Tasks, Image-Suggestions, Product-Analytics (Kanban)

Jan 30 2024

nettrom_WMF added a comment to T346327: Track impressions, success and abandonment rate on the signup form.
  1. I'm definitely in favour of not having duplicate events. Just so I understand the event generation here, we're instrumenting both account creation and login, but the auto-login doesn't trigger a login event through this instrumentation? It's instead that it creates a second account creation "success" event?

See also my comments above T346327#9446991, a way of solving this could be to have a different dedicated schema for login attempts (and impressions?). Would that be appropriate @nettrom_WMF ?

Jan 30 2024, 2:18 AM · Growth-Team (Sprint 9 (Growth Team)), Temporary accounts, MediaWiki-User-login-and-signup, IP-Masking-Growth-Team, Patch-For-Review

Jan 23 2024

nettrom_WMF closed T348898: Mentorship + Personalized Praise: Analysis for Spanish Wikipedia as Resolved.

A written summary of our findings have been published on the Positive reinforcement project page in the section "Personalized praise experiment results". The notebooks associated with this analysis have been uploaded to this Gitlab repository and carry the name of this phab task. Closing as resolved.

Jan 23 2024, 12:09 AM · GrowthExperiments-PersonalizedPraise, GrowthExperiments-Mentorship, Product-Analytics (Kanban), Growth-Team (Sprint 6 (Growth Team))
nettrom_WMF closed T348898: Mentorship + Personalized Praise: Analysis for Spanish Wikipedia, a subtask of T341387: Mentorship + Personalized Praise: Analysis, as Resolved.
Jan 23 2024, 12:09 AM · Growth-Team, Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise
nettrom_WMF closed T348897: Mentorship + Personalized Praise: Arabic, Bangla, and Czech analysis as Resolved.

A written summary of our findings have been published on the Positive reinforcement project page in the section "Personalized praise experiment results". The notebooks associated with this analysis have been uploaded to this Gitlab repository and carry the name of this phab task. Closing as resolved.

Jan 23 2024, 12:08 AM · GrowthExperiments-PersonalizedPraise, GrowthExperiments-Mentorship, Product-Analytics (Kanban), Growth-Team (Sprint 6 (Growth Team))
nettrom_WMF closed T348897: Mentorship + Personalized Praise: Arabic, Bangla, and Czech analysis, a subtask of T341387: Mentorship + Personalized Praise: Analysis, as Resolved.
Jan 23 2024, 12:07 AM · Growth-Team, Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise

Jan 22 2024

nettrom_WMF closed T341387: Mentorship + Personalized Praise: Analysis as Resolved.

The notebooks for the previously mentioned preliminary analysis of the effects of mentorship have been added to this Gitlab repository using a naming convention that reflects that they're associated with this phab task. Resolving this parent task now, and I'll close the child tasks once their notebooks are published as well.

Jan 22 2024, 11:38 PM · Growth-Team, Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise
nettrom_WMF closed T341387: Mentorship + Personalized Praise: Analysis, a subtask of T317817: [EPIC] Positive Reinforcement: Personalized Praise, as Resolved.
Jan 22 2024, 11:38 PM · Growth-Team, GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise, Growth-Positive-Reinforcement, Epic
nettrom_WMF added a comment to T352899: performer struct fields NULL in event_sanitized.mediawiki_revision_tags_change.

Out of curiosity, and in hoping to make this easier to fix, I grabbed daily aggregates for October 2023. Looks like something was deployed the week of Oct 23 as that's when the proportion increases from the expected 6–7%:

Jan 22 2024, 8:31 PM · Patch-For-Review, MW-Interfaces-Team, Product-Analytics, Data-Platform

Jan 9 2024

nettrom_WMF added a comment to T346327: Track impressions, success and abandonment rate on the signup form.

@nettrom_WMF would you mind taking a second look to the analytics schema change 962569, I've added some questions. Mainly, are we instrumenting all account creations or just from temporary users? And how should the auto-login after account creation be logged? Currently it would produce a "success"-Special:CreateAccount event which duplicates the account creation success. cc @Cyndymediawiksim

Jan 9 2024, 1:30 AM · Growth-Team (Sprint 9 (Growth Team)), Temporary accounts, MediaWiki-User-login-and-signup, IP-Masking-Growth-Team, Patch-For-Review

Jan 8 2024

nettrom_WMF closed T329907: [REQUEST] Analysis of available data on clients calling individual services through RESTbase as Declined.

Closing this task as declined for now. If the need for additional data analysis comes up, feel free to file another request.

Jan 8 2024, 4:43 PM · API Platform (RESTbase Deprecation Roadmap), Product-Analytics (Kanban)
nettrom_WMF closed T329907: [REQUEST] Analysis of available data on clients calling individual services through RESTbase, a subtask of T319330: Document current WMF service usage across the WMF ecosystem, as Declined.
Jan 8 2024, 4:43 PM · Epic, API Platform

Jan 4 2024

nettrom_WMF added a comment to T352900: Thank You page experiment: Final analysis of English Wikipedia experiment.

I've collected data and calculated the statistics for both of the campaigns using the date/time ranges listed in the task description.

Jan 4 2024, 7:27 PM · Growth-Team (Sprint 5 (Growth Team)), Product-Analytics (Kanban)
nettrom_WMF updated the task description for T352900: Thank You page experiment: Final analysis of English Wikipedia experiment.
Jan 4 2024, 5:48 PM · Growth-Team (Sprint 5 (Growth Team)), Product-Analytics (Kanban)
nettrom_WMF updated the task description for T352900: Thank You page experiment: Final analysis of English Wikipedia experiment.
Jan 4 2024, 5:47 PM · Growth-Team (Sprint 5 (Growth Team)), Product-Analytics (Kanban)
nettrom_WMF moved T352900: Thank You page experiment: Final analysis of English Wikipedia experiment from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Jan 4 2024, 5:43 PM · Growth-Team (Sprint 5 (Growth Team)), Product-Analytics (Kanban)
nettrom_WMF moved T352900: Thank You page experiment: Final analysis of English Wikipedia experiment from Incoming to Doing on the Growth-Team (Sprint 5 (Growth Team)) board.
Jan 4 2024, 5:43 PM · Growth-Team (Sprint 5 (Growth Team)), Product-Analytics (Kanban)
nettrom_WMF edited projects for T352900: Thank You page experiment: Final analysis of English Wikipedia experiment, added: Product-Analytics (Kanban), Growth-Team (Sprint 5 (Growth Team)); removed Product-Analytics, Growth-Team.
Jan 4 2024, 5:43 PM · Growth-Team (Sprint 5 (Growth Team)), Product-Analytics (Kanban)

Jan 3 2024

nettrom_WMF added a comment to T343823: Conda analytics environments breakage - conflicting dependencies between r-base and other.

I've tested creating new environments on two stat machines with different OSes, and both with and without adding the conda-meta/pinned file. From what I can tell, the conda-forge R 4.2.3 package no longer results in a broken environment because it works just fine without the pin-file. Looking at the file listing on conda-forge, there have been several new versions of the package released since this bug was opened. Based on documentation I've found from googling it should be possible to install a specific build version, but it looks like the Mamba solver doesn't support it properly yet. It doesn't make sense to try this with the classic solver because that wasn't able to resolve the dependencies when I tried that back in August.

Jan 3 2024, 2:13 PM · Data-Platform-SRE, Product-Analytics, Data-Engineering

Dec 12 2023

nettrom_WMF moved T341651: Define editor retention related instrumentation for IP masking from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Dec 12 2023, 8:01 PM · Growth-Team (Sprint 9 (Growth Team)), Temporary accounts, IP-Masking-Growth-Team, Product-Analytics (Kanban)
nettrom_WMF edited projects for T341651: Define editor retention related instrumentation for IP masking, added: Product-Analytics (Kanban), Growth-Team (Sprint 5 (Growth Team)); removed Growth-Team.
Dec 12 2023, 8:01 PM · Growth-Team (Sprint 9 (Growth Team)), Temporary accounts, IP-Masking-Growth-Team, Product-Analytics (Kanban)
nettrom_WMF edited projects for T311531: Add an Image: experiment analysis, added: Product-Analytics (Kanban), Growth-Team (Sprint 5 (Growth Team)); removed Growth-Team, Product-Analytics.
Dec 12 2023, 8:00 PM · Growth-Team (Sprint 9 (Growth Team)), Growth-Structured-Tasks, Image-Suggestions, Product-Analytics (Kanban)

Dec 6 2023

spatton awarded T352900: Thank You page experiment: Final analysis of English Wikipedia experiment a Love token.
Dec 6 2023, 11:03 PM · Growth-Team (Sprint 5 (Growth Team)), Product-Analytics (Kanban)
nettrom_WMF added a comment to T352116: Thank You page experiment: First week analysis of English Wikipedia experiment.

I've collected data to understand the effects of this campaign after about one week. During the data gathering for this update, I discovered a data quality issue in one of the underlying datasets that rendered us unable to correctly identify reverts (this has been filed as T352899). I was able to work around this by modifying the query that collects editing data.

Dec 6 2023, 6:35 PM · Growth-Team (Sprint 5 (Growth Team)), Product-Analytics (Kanban)
nettrom_WMF created T352900: Thank You page experiment: Final analysis of English Wikipedia experiment.
Dec 6 2023, 6:17 PM · Growth-Team (Sprint 5 (Growth Team)), Product-Analytics (Kanban)
nettrom_WMF updated the task description for T352899: performer struct fields NULL in event_sanitized.mediawiki_revision_tags_change.
Dec 6 2023, 6:04 PM · Patch-For-Review, MW-Interfaces-Team, Product-Analytics, Data-Platform
nettrom_WMF added a project to T352899: performer struct fields NULL in event_sanitized.mediawiki_revision_tags_change: Product-Analytics.
Dec 6 2023, 5:59 PM · Patch-For-Review, MW-Interfaces-Team, Product-Analytics, Data-Platform
nettrom_WMF created T352899: performer struct fields NULL in event_sanitized.mediawiki_revision_tags_change.
Dec 6 2023, 5:58 PM · Patch-For-Review, MW-Interfaces-Team, Product-Analytics, Data-Platform

Nov 29 2023

KStoller-WMF awarded T352116: Thank You page experiment: First week analysis of English Wikipedia experiment a Love token.
Nov 29 2023, 8:05 PM · Growth-Team (Sprint 5 (Growth Team)), Product-Analytics (Kanban)
nettrom_WMF added a comment to T352116: Thank You page experiment: First week analysis of English Wikipedia experiment.

I've run the numbers on the currently available data^1 to get a sense of where we're at. Overall, we get the following counts and proportions of page views, "unique visitors" (based on combinations of IPs and user-agents for page views of the account creation page), registrations, and activations.

Nov 29 2023, 7:46 PM · Growth-Team (Sprint 5 (Growth Team)), Product-Analytics (Kanban)
nettrom_WMF moved T352116: Thank You page experiment: First week analysis of English Wikipedia experiment from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Nov 29 2023, 7:32 PM · Growth-Team (Sprint 5 (Growth Team)), Product-Analytics (Kanban)

Nov 28 2023

nettrom_WMF added a comment to T350745: Thank You page experiment: custom account creation page for English Wikipedia experiment.

@KStoller-WMF and/or @nettrom_WMF, could you please let us know if you'd be able to provide an update on account signups some time after the first 24 hour mark, and then occasionally in December? Thanks!

Nov 28 2023, 4:26 PM · Growth-Team (Sprint 3 (Growth Team)), Thank-You-Page
nettrom_WMF edited projects for T352116: Thank You page experiment: First week analysis of English Wikipedia experiment, added: Product-Analytics (Kanban); removed Product-Analytics.
Nov 28 2023, 4:11 PM · Growth-Team (Sprint 5 (Growth Team)), Product-Analytics (Kanban)

Nov 15 2023

nettrom_WMF added a comment to T350745: Thank You page experiment: custom account creation page for English Wikipedia experiment.

Do we have any kind of instrumentation for this in place? Or should we add it to the requirements and prepare some experiment so we can analyze the impact of the TY pages cc @nettrom_WMF

Nov 15 2023, 9:50 PM · Growth-Team (Sprint 3 (Growth Team)), Thank-You-Page

Nov 14 2023

nettrom_WMF moved T348898: Mentorship + Personalized Praise: Analysis for Spanish Wikipedia from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Nov 14 2023, 6:04 PM · GrowthExperiments-PersonalizedPraise, GrowthExperiments-Mentorship, Product-Analytics (Kanban), Growth-Team (Sprint 6 (Growth Team))

Nov 10 2023

nettrom_WMF moved T348897: Mentorship + Personalized Praise: Arabic, Bangla, and Czech analysis from Incoming to Doing on the Growth-Team (Sprint 2 (Growth Team)) board.
Nov 10 2023, 5:19 PM · GrowthExperiments-PersonalizedPraise, GrowthExperiments-Mentorship, Product-Analytics (Kanban), Growth-Team (Sprint 6 (Growth Team))

Nov 7 2023

nettrom_WMF added a project to T350718: Prod Superset down, showing HTTP 500 instead: Data Products.
Nov 7 2023, 6:51 PM · Data-Platform-SRE, superset.wikimedia.org

Oct 24 2023

nettrom_WMF moved T348897: Mentorship + Personalized Praise: Arabic, Bangla, and Czech analysis from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Oct 24 2023, 10:58 PM · GrowthExperiments-PersonalizedPraise, GrowthExperiments-Mentorship, Product-Analytics (Kanban), Growth-Team (Sprint 6 (Growth Team))

Oct 10 2023

nettrom_WMF edited projects for T332344: Section-Level Images: leading indicators, added: Growth-Team; removed Growth-Team (Sprint 0 (Growth Team)).
Oct 10 2023, 12:01 AM · Growth-Team, Product-Analytics, Section-Level-Image-Suggestions, Growth-Structured-Tasks
nettrom_WMF changed the status of T341387: Mentorship + Personalized Praise: Analysis from Open to In Progress.
Oct 10 2023, 12:01 AM · Growth-Team, Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise
nettrom_WMF moved T341387: Mentorship + Personalized Praise: Analysis from Ready for Development to Blocked / Needs work on the Growth-Team (Sprint 0 (Growth Team)) board.
Oct 10 2023, 12:00 AM · Growth-Team, Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise

Oct 2 2023

nettrom_WMF closed T347911: Review proposed Metrics Platform Event Types coverage of existing schemas, a subtask of T345729: [REQUEST] Review of new Metrics Platform Event types in relation to existing schemas, as Resolved.
Oct 2 2023, 9:46 PM · Data Products, Product-Analytics
nettrom_WMF closed T347911: Review proposed Metrics Platform Event Types coverage of existing schemas as Resolved.
Oct 2 2023, 9:46 PM · Data Products (Sprint 01), Product-Analytics
nettrom_WMF created T347911: Review proposed Metrics Platform Event Types coverage of existing schemas.
Oct 2 2023, 8:41 PM · Data Products (Sprint 01), Product-Analytics

Sep 25 2023

nettrom_WMF removed Due Date on T332344: Section-Level Images: leading indicators.
Sep 25 2023, 3:29 PM · Growth-Team, Product-Analytics, Section-Level-Image-Suggestions, Growth-Structured-Tasks

Sep 19 2023

nettrom_WMF moved T346844: Broken data in edits_hourly Druid data cube from Triage to Tracking on the Product-Analytics board.
Sep 19 2023, 10:10 PM · Data Engineering and Event Platform Team (Sprint 2), Data Products, Product-Analytics, Language-Team (Language-2023-July-September)
nettrom_WMF created T346844: Broken data in edits_hourly Druid data cube.
Sep 19 2023, 10:08 PM · Data Engineering and Event Platform Team (Sprint 2), Data Products, Product-Analytics, Language-Team (Language-2023-July-September)

Sep 18 2023

nettrom_WMF closed T279384: Add a link: update reporting notebook as Invalid.

This is no longer relevant, closing as invalid.

Sep 18 2023, 5:57 PM · Growth-Team-Filtering, Add-Link, Growth-Team, Growth-Structured-Tasks
nettrom_WMF closed T279384: Add a link: update reporting notebook, a subtask of T273827: Add a link: instrumentation, as Invalid.
Sep 18 2023, 5:57 PM · Growth-Team, Add-Link, Growth-Structured-Tasks

Sep 8 2023

nettrom_WMF added a comment to T332344: Section-Level Images: leading indicators.

I pulled data from Growth's KPI Grafana board from 2023-07-31 to 2023-08-28 (available here) for Section-Level and Article-Level suggestions. This timeframe was chosen because it should not be as much affected by the June/July slump in activity that we often see on the wikis. The end date is limited by the team shutting off image suggestions in late August (see T345188 for more information). This data range covers four whole weeks of data. While this dataset does not allow us to separate it by platform (desktop and mobile web), nor does it allow us more fine-grained user filtering, it was easily available and provides us with a reasonably good picture that's sufficient for this kind of analysis at this time.

Sep 8 2023, 6:40 PM · Growth-Team, Product-Analytics, Section-Level-Image-Suggestions, Growth-Structured-Tasks

Aug 29 2023

nettrom_WMF awarded T344854: Email notifications of new MediaWiki history snapshot availabilty a Yellow Medal token.
Aug 29 2023, 7:30 PM · Data Engineering and Event Platform Team (Sprint 1), Product-Analytics, Data-Engineering

Aug 28 2023

nettrom_WMF moved T341387: Mentorship + Personalized Praise: Analysis from Doing to Next 2 weeks on the Product-Analytics (Kanban) board.

Moving this out of the various doing/in progress columns as I'll instead be focusing on Levelling Up for a bit.

Aug 28 2023, 2:49 PM · Growth-Team, Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise
nettrom_WMF moved T341387: Mentorship + Personalized Praise: Analysis from In Progress to Prioritized on the GrowthExperiments-Mentorship board.
Aug 28 2023, 2:48 PM · Growth-Team, Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise
nettrom_WMF moved T341387: Mentorship + Personalized Praise: Analysis from In Progress to Ready for Development on the Growth-Team (Sprint 0 (Growth Team)) board.
Aug 28 2023, 2:48 PM · Growth-Team, Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise
nettrom_WMF moved T343290: Leveling Up: post-experiment questions & analysis from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Aug 28 2023, 2:47 PM · Growth-Team (Sprint 0 (Growth Team)), Product-Analytics (Kanban), GrowthExperiments-LevelingUp

Aug 24 2023

nettrom_WMF lowered the priority of T341387: Mentorship + Personalized Praise: Analysis from High to Medium.

Reducing the priority on this to Medium since we've completed the time-sensitive part of this.

Aug 24 2023, 11:00 PM · Growth-Team, Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise

Aug 23 2023

nettrom_WMF added a comment to T340702: [SPIKE] Experiment with flattening custom data object.

I've now completed testing using the EditAttemptStep data. There's a summary of the approaches, run times, and some observations and notes in this notebook on Gitlab.

Aug 23 2023, 11:59 PM · Data Products (Sprint 00), Product-Analytics (Kanban), Metrics Platform Backlog (Metrics Platform Kanban)

Aug 18 2023

nettrom_WMF closed T327581: New Impact Module: experiment analysis as Resolved.

@KStoller-WMF : I made some changes to the summary of the findings. There's now a graph for the key finding, and I elaborated on the work we've done to investigate and mitigate the issue. I'll close this task as resolved, but you should feel free to edit the results section or reach out to me with proposed changes as you see fit.

Aug 18 2023, 10:20 PM · Product-Analytics (Kanban), Growth-Positive-Reinforcement, GrowthExperiments-ImpactModule, Growth-Team (Sprint 0 (Growth Team))
nettrom_WMF closed T327581: New Impact Module: experiment analysis, a subtask of T222310: [EPIC] Positive reinforcement: New Impact Module, as Resolved.
Aug 18 2023, 10:20 PM · Growth-Team, MW-1.40-notes (1.40.0-wmf.7; 2022-10-24), Epic, Growth-Positive-Reinforcement, GrowthExperiments-ImpactModule

Aug 14 2023

nettrom_WMF added a comment to T341387: Mentorship + Personalized Praise: Analysis.

I've completed a preliminary analysis of mentorship and made a writeup of the findings on mw:Growth/Positive reinforcement/Mentorship preliminary analysis, August 2023. The goal of this analysis was to provide some initial insights into mentorship in support of the upcoming Wikimania discussion. We therefore focus on high-level questions such as what proportion of newcomers ask questions, how quickly they do, and to understand if mentorship has an impact on the Growth KPIs. The writeup is therefore also fairly high-level and without specifics for now, but does have some supporting visualizations.

Aug 14 2023, 11:57 PM · Growth-Team, Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise

Aug 8 2023

nettrom_WMF updated the task description for T343823: Conda analytics environments breakage - conflicting dependencies between r-base and other.
Aug 8 2023, 4:10 PM · Data-Platform-SRE, Product-Analytics, Data-Engineering
nettrom_WMF created T343823: Conda analytics environments breakage - conflicting dependencies between r-base and other.
Aug 8 2023, 4:03 PM · Data-Platform-SRE, Product-Analytics, Data-Engineering

Aug 7 2023

nettrom_WMF added a comment to T279738: Superset annotation text overlaps illegibly.

@BTullis : Thank you for the update! While I no longer maintain the chart where this was an issue, I think switching the chart type to an Echarts type is a perfectly reasonable solution. I'd be happy to share this fix more widely on Slack as well so that others know about it, because I wasn't aware that there were deprecated chart types. Let me know if there are other chart types too that we should be aware of, I wasn't able to easily find a list of deprecated types.

Aug 7 2023, 5:41 PM · Data-Platform-SRE, superset.wikimedia.org
nettrom_WMF added a comment to T340702: [SPIKE] Experiment with flattening custom data object.

I'll start digging into query performance this week. In the meantime, I want to make sure that previous work is documented. The experiments and notes around working with these flattened tables can be found in this notebook on gitlab.

Aug 7 2023, 3:56 PM · Data Products (Sprint 00), Product-Analytics (Kanban), Metrics Platform Backlog (Metrics Platform Kanban)

Jul 27 2023

nettrom_WMF moved T341387: Mentorship + Personalized Praise: Analysis from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Jul 27 2023, 5:57 PM · Growth-Team, Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise
nettrom_WMF moved T341387: Mentorship + Personalized Praise: Analysis from Ready for Development to In Progress on the Growth-Team (Sprint 0 (Growth Team)) board.
Jul 27 2023, 5:57 PM · Growth-Team, Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise

Jul 24 2023

nettrom_WMF edited projects for T341387: Mentorship + Personalized Praise: Analysis, added: Product-Analytics (Kanban); removed Product-Analytics.
Jul 24 2023, 8:06 PM · Growth-Team, Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise
nettrom_WMF moved T340702: [SPIKE] Experiment with flattening custom data object from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Jul 24 2023, 5:22 PM · Data Products (Sprint 00), Product-Analytics (Kanban), Metrics Platform Backlog (Metrics Platform Kanban)
nettrom_WMF edited projects for T340702: [SPIKE] Experiment with flattening custom data object, added: Product-Analytics (Kanban); removed Product-Analytics.
Jul 24 2023, 5:22 PM · Data Products (Sprint 00), Product-Analytics (Kanban), Metrics Platform Backlog (Metrics Platform Kanban)

Jul 21 2023

nettrom_WMF updated Other Assignee for T340702: [SPIKE] Experiment with flattening custom data object, added: nettrom_WMF.

Added the Product Analytics tag, and myself as secondary assignee on this task. I've started looking at this and will continue that next week.

Jul 21 2023, 10:58 PM · Data Products (Sprint 00), Product-Analytics (Kanban), Metrics Platform Backlog (Metrics Platform Kanban)

Jul 19 2023

nettrom_WMF added a comment to T342150: [EPIC] Impact Module: Investigate activation findings from Experiment analysis.

From what I can tell we've deployed a patch to have the new impact module only switch out of deactivated state through article edits (Main namespace) per T338640, but that there was also a bug that affected it that got fixed in T341994. I'm going to restart the experiment clock in the notebook that aggregates statistics so that we can monitor whether there's a change after this patch. The aggregated leading indicators from the Levelling Up experiment will continue to exist in our database for comparison.

Jul 19 2023, 3:34 PM · Growth-Positive-Reinforcement, GrowthExperiments-ImpactModule, Growth-Team (Sprint 0 (Growth Team))

Jul 17 2023

nettrom_WMF triaged T341629: Analysis of newcomer suggested edits topic selection as Medium priority.

Adding this to the Product Analytics kanban board. I'll pick this up and grab statistics on “How many newcomers/editors have at least one topic defined?” later this week.

Jul 17 2023, 4:26 PM · Growth-Team, Product-Analytics (Kanban), User-Iflorez, GrowthExperiments-NewcomerTasks, GrowthExperiments-Homepage

Jul 7 2023

nettrom_WMF raised the priority of T341387: Mentorship + Personalized Praise: Analysis from High to Needs Triage.

I've added this to Product Analytics tag and reset the priority so the PA team can triage it there. Given the timeliness of this work in preparation for Wikimania, I think the High priority should be re-added at triage. I'll then also move it to the PA kanban board to pick it up next week.

Jul 7 2023, 9:19 PM · Growth-Team, Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise
nettrom_WMF added a project to T341387: Mentorship + Personalized Praise: Analysis: Product-Analytics.
Jul 7 2023, 9:16 PM · Growth-Team, Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise

Jun 27 2023

nettrom_WMF awarded T337258: Enable libmamba by default for conda environment solving a Yellow Medal token.
Jun 27 2023, 8:45 PM · Data-Platform-SRE, Data Engineering and Event Platform Team, Data-Engineering, Data Pipelines

Jun 26 2023

nettrom_WMF added a comment to T338640: Impact Module: New editors who have edited outside of the main namespace should see the Impact module empty state.

That way we are removing a potential confounding variable from the experiment. Any concerns with that approach? @nettrom_WMF / @RHo / @JFernandez-WMF

Jun 26 2023, 10:19 PM · MW-1.41-notes (1.41.0-wmf.17; 2023-07-11), GrowthExperiments-ImpactModule, Growth-Team (Sprint 0 (Growth Team))

Jun 21 2023

nettrom_WMF moved T337885: Assess usage of the newcomer homepage entry point from the Contributions page with view for removal from Triage to Tracking on the Product-Analytics board.
Jun 21 2023, 4:09 PM · Product-Analytics, Growth-Team

Jun 16 2023

nettrom_WMF added a comment to T335716: Section-level images: Instrumentation.

I introduced console-based event QA to @KStoller-WMF in our 1:1. We went through most of the workflow and checked the various events. From what we found the instrumentation is working correctly, there's nothing there that blocks deployment.

Jun 16 2023, 6:18 PM · Growth-Team, MW-1.41-notes (1.41.0-wmf.13; 2023-06-13), Product-Analytics, Section-Level-Image-Suggestions, Growth-Structured-Tasks

Jun 15 2023

nettrom_WMF moved T328057: Leveling Up: experiment analysis from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Jun 15 2023, 9:14 PM · User-Iflorez, Growth-Team (Sprint 0 (Growth Team)), Product-Analytics (Kanban), GrowthExperiments-LevelingUp
nettrom_WMF changed the status of T328057: Leveling Up: experiment analysis from Open to In Progress.
Jun 15 2023, 9:14 PM · User-Iflorez, Growth-Team (Sprint 0 (Growth Team)), Product-Analytics (Kanban), GrowthExperiments-LevelingUp
nettrom_WMF changed the status of T328057: Leveling Up: experiment analysis, a subtask of T317813: [EPIC] Positive Reinforcement: Leveling Up , from Open to In Progress.
Jun 15 2023, 9:14 PM · Growth-Team, GrowthExperiments-LevelingUp, Epic
nettrom_WMF edited projects for T328057: Leveling Up: experiment analysis, added: Product-Analytics (Kanban), Growth-Team (Sprint 0 (Growth Team)); removed Product-Analytics, Growth-Team.
Jun 15 2023, 9:14 PM · User-Iflorez, Growth-Team (Sprint 0 (Growth Team)), Product-Analytics (Kanban), GrowthExperiments-LevelingUp

Jun 9 2023

nettrom_WMF added a comment to T337320: [Spike] Investigate A/B test results of Growth Experiments impact module.

Yeah, at some point we stopped filtering by namespace. I'd assume that there are so few new users with a non-mainspace first edit that they can't cause the difference in activation... @nettrom_WMF is that something you can confirm or refute?

Jun 9 2023, 6:25 PM · Design-System-Team, MW-1.41-notes (1.41.0-wmf.10; 2023-05-23), Growth-Team (Sprint 0 (Growth Team))

Jun 6 2023

nettrom_WMF added a comment to T325117: Personalized Praise: Instrumentation.

@Urbanecm_WMF : From what I can tell, the notifications are being sent out by MediaWiki but not captured as events in event.mediawiki_mentor_dashboard_personalized_praise. The first query below is the Hive/Spark query of the event data, which returns a count of zero. The second I've run for each of the four pilot wiki MariaDB replicas (e.g. analytics-mysql eswiki --use-x1 on a stat* machine), and they all return a non-zero result.

Jun 6 2023, 5:01 PM · MW-1.41-notes (1.41.0-wmf.11; 2023-05-30), Product-Analytics, Growth-Team (Sprint 0 (Growth Team)), GrowthExperiments-PersonalizedPraise, GrowthExperiments-Mentorship, GrowthExperiments-MentorDashboard, Growth-Positive-Reinforcement
nettrom_WMF moved T337069: Personalized praise: Evaluate leading indicators from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Jun 6 2023, 4:10 PM · Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise, Growth-Positive-Reinforcement, Growth-Team (Sprint 0 (Growth Team))

Jun 5 2023

nettrom_WMF added a comment to T334411: Positive Reinforcement: investigate difference in mobile activation.

I've taken a first look at this by investigating two proportions. First, the proportion of newcomers with constructive article activation. Second, the proportion of those activating through Suggested Edits. While that doesn't specifically answer if users are able to find Suggested Edits and/or activate it, it should give us an indication of whether we are running into issues.

Jun 5 2023, 11:22 PM · Growth-Team (Sprint 0 (Growth Team)), Product-Analytics (Kanban), GrowthExperiments-LevelingUp
nettrom_WMF moved T334411: Positive Reinforcement: investigate difference in mobile activation from [Deprecated] Done (previously: Needs sign-off) to Doing on the Product-Analytics (Kanban) board.
Jun 5 2023, 10:56 PM · Growth-Team (Sprint 0 (Growth Team)), Product-Analytics (Kanban), GrowthExperiments-LevelingUp
KStoller-WMF awarded T334411: Positive Reinforcement: investigate difference in mobile activation a Barnstar token.
Jun 5 2023, 8:25 PM · Growth-Team (Sprint 0 (Growth Team)), Product-Analytics (Kanban), GrowthExperiments-LevelingUp
nettrom_WMF reopened T334411: Positive Reinforcement: investigate difference in mobile activation as "In Progress".

I'm reopening this after we discussed this in light of the recent deployment of the New Impact Module to additional wikis in T336203. There's a difference in how the Suggested Edits module behaves on desktop and mobile that we're interested in understanding. On desktop, users have to initialize the module before getting any suggestions, whereas on mobile the module is already initialized. Timeboxed to one hour to grab fresh data and investigate this four the pilot wikis, with an additional hour if we want to expand it to the nine other Wikipedia wikis we're currently deployed to.

Jun 5 2023, 8:12 PM · Growth-Team (Sprint 0 (Growth Team)), Product-Analytics (Kanban), GrowthExperiments-LevelingUp
nettrom_WMF reopened T334411: Positive Reinforcement: investigate difference in mobile activation, a subtask of T328055: Positive Reinforcement: evaluate Leveling Up leading indicators, as In Progress.
Jun 5 2023, 8:11 PM · Growth-Team (Sprint 0 (Growth Team)), Product-Analytics (Kanban), GrowthExperiments-LevelingUp

Jun 1 2023

nettrom_WMF added a project to T337885: Assess usage of the newcomer homepage entry point from the Contributions page with view for removal: Product-Analytics.
Jun 1 2023, 10:27 PM · Product-Analytics, Growth-Team

May 31 2023

nettrom_WMF awarded T278467: Use Hive/Spark timestamps in Refined event data a Love token.
May 31 2023, 4:34 PM · User-Iflorez, Data-Engineering, Product-Analytics
nettrom_WMF moved T337379: Make use of NewImpact Vue logger plugin for mentorship instrumentation from Triage to Tracking on the Product-Analytics board.

I added a QA step to the acceptance criteria since we'll be wanting to verify that this doesn't alter the logged data in any way. Happy to help out with that as needed, let me know! Moving this to "Tracking" on the Product Analytics board for now.

May 31 2023, 3:56 PM · Growth-Team, GrowthExperiments-Mentorship, Product-Analytics, Growth-Positive-Reinforcement
nettrom_WMF updated the task description for T337379: Make use of NewImpact Vue logger plugin for mentorship instrumentation.
May 31 2023, 3:55 PM · Growth-Team, GrowthExperiments-Mentorship, Product-Analytics, Growth-Positive-Reinforcement

May 30 2023

nettrom_WMF added a comment to T336203: Positive reinforcement: Deploy the new Impact module to all Wikipedias.

Will this be deployed to the test group (the first 10 wikis) in the same A/B test configuration that we have on the pilot wikis (50% get the new, 50% get the old), or will everyone get the new Impact module? It'll be a lot easier to measure the impact if it's an A/B test, but I'm not sure if that'll happen automagically or if it requires a lot of extra work?

May 30 2023, 4:18 PM · User-notice-archive, Growth-Team (Sprint 2 (Growth Team)), Growth-Positive-Reinforcement, GrowthExperiments-ImpactModule

May 26 2023

nettrom_WMF added a comment to T325117: Personalized Praise: Instrumentation.

Unfortunately, this is not yet implemented (currently, the post edit dialog is fully in DT's hand). I do want to implement that though. If that happens, the link would point to Special:MentorDashboard?source=personalized-praise-post-edit (or something similar), which would be captured in referer_route of the analytics/mediawiki/mentor_dashboard/visit schema. Would that be sufficient to provide you with the relevant information?

May 26 2023, 10:59 PM · MW-1.41-notes (1.41.0-wmf.11; 2023-05-30), Product-Analytics, Growth-Team (Sprint 0 (Growth Team)), GrowthExperiments-PersonalizedPraise, GrowthExperiments-Mentorship, GrowthExperiments-MentorDashboard, Growth-Positive-Reinforcement
nettrom_WMF updated the task description for T337069: Personalized praise: Evaluate leading indicators.
May 26 2023, 10:56 PM · Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise, Growth-Positive-Reinforcement, Growth-Team (Sprint 0 (Growth Team))
nettrom_WMF added a comment to T337069: Personalized praise: Evaluate leading indicators.

@nettrom_WMF Just FYI, I'm moving this out of unblocked since the remainder of the Personalized praise work was just released: T334630: Personalized praise: Deployment of the new mentor dashboard module to Growth Pilot Wikis.

Hopefully we can look at some initial leading indicators as soon as next week?

May 26 2023, 10:53 PM · Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise, Growth-Positive-Reinforcement, Growth-Team (Sprint 0 (Growth Team))