Page MenuHomePhabricator

nettrom_WMF (Morten Warncke-Wang)
Staff Data Scientist, Product Analytics

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Wednesday

  • Clear sailing ahead.

User Details

User Since
Aug 21 2018, 8:23 PM (248 w, 5 d)
Availability
Available
IRC Nick
Nettrom
LDAP User
Unknown
MediaWiki User
MWang (WMF) [ Global Accounts ]

Recent Activity

Fri, May 26

nettrom_WMF added a comment to T325117: Personalized Praise: Instrumentation.

Unfortunately, this is not yet implemented (currently, the post edit dialog is fully in DT's hand). I do want to implement that though. If that happens, the link would point to Special:MentorDashboard?source=personalized-praise-post-edit (or something similar), which would be captured in referer_route of the analytics/mediawiki/mentor_dashboard/visit schema. Would that be sufficient to provide you with the relevant information?

Fri, May 26, 10:59 PM · MW-1.41-notes (1.41.0-wmf.11; 2023-05-30), Product-Analytics, Growth-Team (Current Sprint), GrowthExperiments-PersonalizedPraise, GrowthExperiments-Mentorship, GrowthExperiments-MentorDashboard, Growth-Positive-Reinforcement
nettrom_WMF updated the task description for T337069: Personalized praise: Evaluate leading indicators.
Fri, May 26, 10:56 PM · Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise, Growth-Positive-Reinforcement, Growth-Team (Current Sprint)
nettrom_WMF added a comment to T337069: Personalized praise: Evaluate leading indicators.

@nettrom_WMF Just FYI, I'm moving this out of unblocked since the remainder of the Personalized praise work was just released: T334630: Personalized praise: Deployment of the new mentor dashboard module to Growth Pilot Wikis.

Hopefully we can look at some initial leading indicators as soon as next week?

Fri, May 26, 10:53 PM · Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise, Growth-Positive-Reinforcement, Growth-Team (Current Sprint)
nettrom_WMF edited projects for T337069: Personalized praise: Evaluate leading indicators, added: Product-Analytics (Kanban); removed Product-Analytics.
Fri, May 26, 10:51 PM · Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise, Growth-Positive-Reinforcement, Growth-Team (Current Sprint)
nettrom_WMF moved T337069: Personalized praise: Evaluate leading indicators from In Progress to Ready for Development on the Growth-Team (Current Sprint) board.
Fri, May 26, 10:51 PM · Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise, Growth-Positive-Reinforcement, Growth-Team (Current Sprint)
nettrom_WMF moved T337069: Personalized praise: Evaluate leading indicators from In Progress to Prioritized on the GrowthExperiments-Mentorship board.
Fri, May 26, 10:51 PM · Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise, Growth-Positive-Reinforcement, Growth-Team (Current Sprint)

Wed, May 24

nettrom_WMF closed T316300: Positive Reinforcement: measurement specification as Resolved.

With the publication of the results for the Levelling Up leading indicators, as well as the planned leading indicators for Personalized praise on the project page, this task can be resolved.

Wed, May 24, 5:49 PM · Growth-Positive-Reinforcement, Product-Analytics (Kanban), Growth-Team (Current Sprint)
nettrom_WMF closed T316300: Positive Reinforcement: measurement specification , a subtask of T248330: [EPIC] Growth: Positive reinforcement, as Resolved.
Wed, May 24, 5:49 PM · Growth-Positive-Reinforcement, Growth-Team, Epic
nettrom_WMF updated the task description for T316300: Positive Reinforcement: measurement specification .
Wed, May 24, 5:48 PM · Growth-Positive-Reinforcement, Product-Analytics (Kanban), Growth-Team (Current Sprint)
nettrom_WMF closed T328055: Positive Reinforcement: evaluate Leveling Up leading indicators as Resolved.

The findings for all indicators have been posted on wiki here: https://www.mediawiki.org/wiki/Growth/Positive_reinforcement#Levelling_up Closing this task as resolved.

Wed, May 24, 5:48 PM · Growth-Team (Current Sprint), Product-Analytics (Kanban), GrowthExperiments-LevelingUp
nettrom_WMF closed T328055: Positive Reinforcement: evaluate Leveling Up leading indicators, a subtask of T317813: [EPIC] Positive Reinforcement: Leveling Up , as Resolved.
Wed, May 24, 5:47 PM · GrowthExperiments-LevelingUp, Growth-Team (Current Sprint), Epic

Tue, May 23

nettrom_WMF moved T337069: Personalized praise: Evaluate leading indicators from Triage to Blocked on the Product-Analytics board.
Tue, May 23, 4:26 PM · Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise, Growth-Positive-Reinforcement, Growth-Team (Current Sprint)
nettrom_WMF created T337310: Remove notification_seen column from echo_notifications database table.
Tue, May 23, 1:31 PM · DBA, Growth-Team, Notifications

Mon, May 22

nettrom_WMF moved T335716: Section-level images: Instrumentation from Triage to Tracking on the Product-Analytics board.
Mon, May 22, 5:40 PM · Product-Analytics, Growth-Team (Current Sprint), Section-Level-Image-Suggestions, Growth-Structured-Tasks
nettrom_WMF added a project to T335716: Section-level images: Instrumentation: Product-Analytics.
Mon, May 22, 5:40 PM · Product-Analytics, Growth-Team (Current Sprint), Section-Level-Image-Suggestions, Growth-Structured-Tasks
nettrom_WMF moved T325117: Personalized Praise: Instrumentation from Triage to Tracking on the Product-Analytics board.
Mon, May 22, 5:23 PM · MW-1.41-notes (1.41.0-wmf.11; 2023-05-30), Product-Analytics, Growth-Team (Current Sprint), GrowthExperiments-PersonalizedPraise, GrowthExperiments-Mentorship, GrowthExperiments-MentorDashboard, Growth-Positive-Reinforcement
nettrom_WMF added a project to T325117: Personalized Praise: Instrumentation: Product-Analytics.
Mon, May 22, 5:23 PM · MW-1.41-notes (1.41.0-wmf.11; 2023-05-30), Product-Analytics, Growth-Team (Current Sprint), GrowthExperiments-PersonalizedPraise, GrowthExperiments-Mentorship, GrowthExperiments-MentorDashboard, Growth-Positive-Reinforcement
nettrom_WMF added a project to T337069: Personalized praise: Evaluate leading indicators: Product-Analytics.
Mon, May 22, 5:10 PM · Product-Analytics (Kanban), GrowthExperiments-Mentorship, GrowthExperiments-PersonalizedPraise, Growth-Positive-Reinforcement, Growth-Team (Current Sprint)

Tue, May 16

nettrom_WMF added a comment to T325117: Personalized Praise: Instrumentation.

@Urbanecm_WMF : I've reviewed the existing and proposed schemas and compared them to the specification. This is great work! Almost everything looks according to spec from what I can tell and will enable us to answer the questions we have. Here are the two questions I've got:

Tue, May 16, 9:59 PM · MW-1.41-notes (1.41.0-wmf.11; 2023-05-30), Product-Analytics, Growth-Team (Current Sprint), GrowthExperiments-PersonalizedPraise, GrowthExperiments-Mentorship, GrowthExperiments-MentorDashboard, Growth-Positive-Reinforcement
nettrom_WMF updated the task description for T325117: Personalized Praise: Instrumentation.
Tue, May 16, 9:40 PM · MW-1.41-notes (1.41.0-wmf.11; 2023-05-30), Product-Analytics, Growth-Team (Current Sprint), GrowthExperiments-PersonalizedPraise, GrowthExperiments-Mentorship, GrowthExperiments-MentorDashboard, Growth-Positive-Reinforcement
nettrom_WMF lowered the priority of T329907: [REQUEST] Analysis of available data on clients calling individual services through RESTbase from High to Medium.

Reducing the priority of this for now and moving it to "Needs Investigation" while I follow up with @VirginiaPoundstone on what API Platform needs.

Tue, May 16, 4:19 PM · API Platform (RESTbase Deprecation Roadmap), Product-Analytics (Kanban)
nettrom_WMF updated subscribers of T332344: Section-Level Images: leading indicators.

I've chatted with @cchen about what the Structured Data team needs so that we're not duplicating efforts, and learned that there isn't currently any overlap between us. Adding this comment mainly to document that the SD team will send out notifications about section-level images to experienced editors. If the Growth team at some point chooses to measure something like edits, we might want to make sure we take that into account.

Tue, May 16, 4:17 PM · Product-Analytics, Growth-Team (Current Sprint), Section-Level-Image-Suggestions, Growth-Structured-Tasks

Wed, May 10

nettrom_WMF updated the task description for T332344: Section-Level Images: leading indicators.
Wed, May 10, 8:19 PM · Product-Analytics, Growth-Team (Current Sprint), Section-Level-Image-Suggestions, Growth-Structured-Tasks

Mon, May 8

nettrom_WMF added a comment to T335716: Section-level images: Instrumentation.

@nettrom_WMF would you like separate schemas (where it makes sense, e.g. mediawiki/page/image-suggestions-feedback) or should we just plug in a section ID into the schema, and set it to null for the old image recommendation task type? Do you want a specific format for the section ID (ordinal, section title text etc)?

Mon, May 8, 5:20 PM · Product-Analytics, Growth-Team (Current Sprint), Section-Level-Image-Suggestions, Growth-Structured-Tasks
nettrom_WMF raised the priority of T332344: Section-Level Images: leading indicators from High to Needs Triage.

Adding the Product Analytics tag and resetting the priority so that Product Analytics can triage this tomorrow. Since this is planned for late May/early June the work needed on this at the moment is more of a planning nature, which to me fits into PA's "medium priority".

Mon, May 8, 4:58 PM · Product-Analytics, Growth-Team (Current Sprint), Section-Level-Image-Suggestions, Growth-Structured-Tasks

Fri, May 5

nettrom_WMF moved T334411: Positive Reinforcement: investigate difference in mobile activation from Doing to Needs Sign-off on the Product-Analytics (Kanban) board.

We've examined this phenomenon in several different ways looking for patterns, and not found any specific pattern that provides clues as to why this is happening. This analysis started out by looking at the three breakdowns suggested in the task, as well as by wiki since it was trivial to split the data that way too. Here's what we found:

Fri, May 5, 5:23 PM · Growth-Team (Current Sprint), Product-Analytics (Kanban), GrowthExperiments-LevelingUp

Apr 12 2023

nettrom_WMF updated subscribers of T334432: Grant Access to analytics-privatedata-users for KMorgan.

@nettrom_WMF Are you the approving party (manager) of @KMorgan-WMF?

Apr 12 2023, 3:18 PM · SRE, LDAP-Access-Requests

Apr 11 2023

nettrom_WMF closed T331495: Quick Analysis: Thank You Pages: custom account creation pages for sv, it, ja, fr, nl as Resolved.

I've gathered all the data from English Wikipedia and in the process fixed a bug in our processing so that we also correctly count mobile page views after the desktop to mobile redirect. We do not report data for the French and Dutch campaigns because of low numbers.

Apr 11 2023, 8:31 PM · Product-Analytics (Kanban), Growth-Team (Current Sprint)
nettrom_WMF closed T331495: Quick Analysis: Thank You Pages: custom account creation pages for sv, it, ja, fr, nl, a subtask of T321370: Thank You Pages: custom account creation pages for sv, it, ja, fr, nl, as Resolved.
Apr 11 2023, 8:31 PM · MW-1.40-notes (1.40.0-wmf.21; 2023-01-30), Growth-Team (Current Sprint)
nettrom_WMF moved T328057: Leveling Up: experiment analysis from Upcoming Quarter to Current Quarter on the Product-Analytics board.
Apr 11 2023, 5:14 PM · Product-Analytics, GrowthExperiments-LevelingUp, Growth-Team
nettrom_WMF created T334495: Delete the leading question mark from uri_query in the webrequest table.
Apr 11 2023, 4:56 PM · Data Pipelines, Product-Analytics

Apr 10 2023

nettrom_WMF changed the status of T334411: Positive Reinforcement: investigate difference in mobile activation from Open to In Progress.
Apr 10 2023, 9:42 PM · Growth-Team (Current Sprint), Product-Analytics (Kanban), GrowthExperiments-LevelingUp
nettrom_WMF moved T334411: Positive Reinforcement: investigate difference in mobile activation from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Apr 10 2023, 9:42 PM · Growth-Team (Current Sprint), Product-Analytics (Kanban), GrowthExperiments-LevelingUp
nettrom_WMF changed the status of T334411: Positive Reinforcement: investigate difference in mobile activation, a subtask of T328055: Positive Reinforcement: evaluate Leveling Up leading indicators, from Open to In Progress.
Apr 10 2023, 9:42 PM · Growth-Team (Current Sprint), Product-Analytics (Kanban), GrowthExperiments-LevelingUp
nettrom_WMF created T334432: Grant Access to analytics-privatedata-users for KMorgan.
Apr 10 2023, 9:01 PM · SRE, LDAP-Access-Requests
nettrom_WMF lowered the priority of T328055: Positive Reinforcement: evaluate Leveling Up leading indicators from High to Medium.

Reducing the priority of this to Medium as the majority of this work is done and not as urgent. I'll also remove the due date.

Apr 10 2023, 4:20 PM · Growth-Team (Current Sprint), Product-Analytics (Kanban), GrowthExperiments-LevelingUp
nettrom_WMF removed Due Date on T328055: Positive Reinforcement: evaluate Leveling Up leading indicators.
Apr 10 2023, 4:20 PM · Growth-Team (Current Sprint), Product-Analytics (Kanban), GrowthExperiments-LevelingUp
nettrom_WMF triaged T334411: Positive Reinforcement: investigate difference in mobile activation as High priority.
Apr 10 2023, 4:19 PM · Growth-Team (Current Sprint), Product-Analytics (Kanban), GrowthExperiments-LevelingUp
nettrom_WMF created T334411: Positive Reinforcement: investigate difference in mobile activation.
Apr 10 2023, 4:18 PM · Growth-Team (Current Sprint), Product-Analytics (Kanban), GrowthExperiments-LevelingUp
nettrom_WMF added a comment to T328055: Positive Reinforcement: evaluate Leveling Up leading indicators.

The leading indicators have been defined and I've created a notebook to aggregate data and set it up to run once a day. We also have a Superset dashboard to monitor the indicators. What remains is:

Apr 10 2023, 4:09 PM · Growth-Team (Current Sprint), Product-Analytics (Kanban), GrowthExperiments-LevelingUp

Apr 5 2023

nettrom_WMF changed the status of T331495: Quick Analysis: Thank You Pages: custom account creation pages for sv, it, ja, fr, nl from Open to In Progress.

Those seeing the Thank You page in English do get a link to create an account on the English Wikipedia e.g. in Sweden the English language Thank You page links to https://en.wikipedia.org/w/index.php?title=Special:CreateAccount&campaign=typage-nwe-sv-2023.

This seemed to make more sense for the user, and is consistent with what we previously did in Latin America. I mentioned this in Slack, but I don't think the links in T321370 got updated sorry.

Apr 5 2023, 8:44 PM · Product-Analytics (Kanban), Growth-Team (Current Sprint)
nettrom_WMF changed the status of T331495: Quick Analysis: Thank You Pages: custom account creation pages for sv, it, ja, fr, nl, a subtask of T321370: Thank You Pages: custom account creation pages for sv, it, ja, fr, nl, from Open to In Progress.
Apr 5 2023, 8:44 PM · MW-1.40-notes (1.40.0-wmf.21; 2023-01-30), Growth-Team (Current Sprint)
nettrom_WMF reassigned T331495: Quick Analysis: Thank You Pages: custom account creation pages for sv, it, ja, fr, nl from nettrom_WMF to KStoller-WMF.

Editing this to remove the posted data because the more recent table posted below is the authoritative one.

Apr 5 2023, 6:24 PM · Product-Analytics (Kanban), Growth-Team (Current Sprint)
nettrom_WMF moved T331495: Quick Analysis: Thank You Pages: custom account creation pages for sv, it, ja, fr, nl from Ready for Development to In Progress on the Growth-Team (Current Sprint) board.
Apr 5 2023, 6:09 PM · Product-Analytics (Kanban), Growth-Team (Current Sprint)

Apr 3 2023

nettrom_WMF added a comment to T333863: Grant Access to analytics-privatedata-users for trizek.

@Ottomata : as you've discovered, @Trizek-WMF is already assigned to the wmf group and is WMF staff (and also subscribed to this task).

Apr 3 2023, 8:29 PM · SRE, LDAP-Access-Requests
nettrom_WMF created T333863: Grant Access to analytics-privatedata-users for trizek.
Apr 3 2023, 5:14 PM · SRE, LDAP-Access-Requests

Mar 29 2023

nettrom_WMF added a comment to T331495: Quick Analysis: Thank You Pages: custom account creation pages for sv, it, ja, fr, nl.

To make sure I understand your point "we'll have to look at the combined metrics since there's no language data". Does this mean we won't be able to know for example English-France registrations, we'll only be able to know total France registrations (en, fr) as a portion of total France TY Page views? That seems ok.
I'm assuming we'd still be able to look at total English registrations (all countries) as a portion of total English TY Page views (all countries) too? Or do we also lose that ability.

Mar 29 2023, 5:51 PM · Product-Analytics (Kanban), Growth-Team (Current Sprint)

Mar 28 2023

nettrom_WMF added a comment to T331495: Quick Analysis: Thank You Pages: custom account creation pages for sv, it, ja, fr, nl.

@nettrom_WMF - My sense is that we did more in-depth analysis for the initial experiment, and at this point it should be OK to simplify and just look at combined metrics.
But if you or @HNordeenWMF think splitting the data by language is definitely needed, feel free to speak up.

Mar 28 2023, 4:24 PM · Product-Analytics (Kanban), Growth-Team (Current Sprint)
nettrom_WMF closed T230070: Newcomer tasks: post-deployment QA of instrumentation as Resolved.

This work has been done multiple times over since Newcomer Tasks was deployed, although not documented here. Closing as resolved.

Mar 28 2023, 2:48 PM · Growth-Team-Filtering, NewcomerTasks 1.0 , Growth-Team, GrowthExperiments-Homepage
nettrom_WMF closed T230070: Newcomer tasks: post-deployment QA of instrumentation, a subtask of T230068: Newcomer tasks: instrumentation, as Resolved.
Mar 28 2023, 2:48 PM · Product-Analytics, Epic, Growth-Team (Current Sprint), NewcomerTasks 1.0 , GrowthExperiments-Homepage

Mar 27 2023

nettrom_WMF moved T328055: Positive Reinforcement: evaluate Leveling Up leading indicators from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Mar 27 2023, 10:12 PM · Growth-Team (Current Sprint), Product-Analytics (Kanban), GrowthExperiments-LevelingUp
nettrom_WMF moved T331495: Quick Analysis: Thank You Pages: custom account creation pages for sv, it, ja, fr, nl from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Mar 27 2023, 8:57 PM · Product-Analytics (Kanban), Growth-Team (Current Sprint)
nettrom_WMF added a comment to T331495: Quick Analysis: Thank You Pages: custom account creation pages for sv, it, ja, fr, nl.

@HNordeenWMF : I'm currently working on gathering these statistics and noticed that T321370 specifies two languages for each campaign: the local language of the wiki as well as English. The language is then reflected in the uselang parameter in the URL. When grabbing the data I noticed this parameter doesn't show up (except for a few test visits), and from playing around with the Thank You page I did not see the uselang parameter in the link to account creation. I'm not sure if this is a bug or not?

Mar 27 2023, 6:26 PM · Product-Analytics (Kanban), Growth-Team (Current Sprint)

Mar 24 2023

nettrom_WMF closed T332441: Quick Analysis: impact of adding Thanks to more pages as Resolved.

This analysis has been completed. We created a slide deck presenting the findings and decided to upload that to Commons for wider accessibility: https://commons.wikimedia.org/wiki/File:T332441_-_Thanks_Usage_-_Introduction.pdf The notebooks used for data gathering and visualization are in this GitHub repository

Mar 24 2023, 7:04 PM · Product-Analytics (Kanban), Growth-Positive-Reinforcement, Growth-Team (Current Sprint)

Mar 21 2023

nettrom_WMF added a comment to T325117: Personalized Praise: Instrumentation.

I've updated the task description to reflect the following changes:

Mar 21 2023, 11:06 PM · MW-1.41-notes (1.41.0-wmf.11; 2023-05-30), Product-Analytics, Growth-Team (Current Sprint), GrowthExperiments-PersonalizedPraise, GrowthExperiments-Mentorship, GrowthExperiments-MentorDashboard, Growth-Positive-Reinforcement
nettrom_WMF updated the task description for T325117: Personalized Praise: Instrumentation.
Mar 21 2023, 11:03 PM · MW-1.41-notes (1.41.0-wmf.11; 2023-05-30), Product-Analytics, Growth-Team (Current Sprint), GrowthExperiments-PersonalizedPraise, GrowthExperiments-Mentorship, GrowthExperiments-MentorDashboard, Growth-Positive-Reinforcement
nettrom_WMF added a comment to T325117: Personalized Praise: Instrumentation.

One question I have about these appreciation messages, will they have a specific edit tag to identify them as Personalised praise? One possible scenario we might run into is that a mentor clicks "Send appreciation" and then sees something on the mentee's talk page that makes them change their mind, but perhaps also makes them add a comment. Would we be able to tell the difference? I'm also not sure how likely we are to encounter that scenario?

Good question! T322449: Personalized praise talk page message doesn't ask for a tag, so I didn't plan for one, but having one would certainly make sense. That said, I'm not sure whether the potential tag would help us to tell the difference between a praising message and a non-praising message sent via Personalized praise, since mentors will have the ability to change the message content as they please. In other words: if a mentor closes the preloaded interface and then clicks "Add a new topic" again (opening DiscussionTools for the second time), we'd be able to detect that. However, if a mentor merely empties the preloaded message and replaces it with something totally different, that's more difficult to identify

Mar 21 2023, 10:58 PM · MW-1.41-notes (1.41.0-wmf.11; 2023-05-30), Product-Analytics, Growth-Team (Current Sprint), GrowthExperiments-PersonalizedPraise, GrowthExperiments-Mentorship, GrowthExperiments-MentorDashboard, Growth-Positive-Reinforcement
nettrom_WMF added a comment to T85756: Make (redacted) log_search table available on Wiki Replicas.

Found this after my curiosity made me check if log_search is available on the Wiki replicas. A couple of fields that might be considered non-private are:

Mar 21 2023, 5:01 PM · Security, MediaWiki-General, Data-Services, DBA
nettrom_WMF awarded T85756: Make (redacted) log_search table available on Wiki Replicas a Like token.
Mar 21 2023, 4:55 PM · Security, MediaWiki-General, Data-Services, DBA
nettrom_WMF added a comment to T332621: Add log_search to monthly sqoop list.

I got curious and checked whether log_search is available on the standard replicas, and it isn't because it contains private data. T85756 exists to also make it available more widely with private data redacted. If it's sqooped it would need to be marked as private, similarly as some of the other tables we have.

Mar 21 2023, 4:47 PM · Data Pipelines, Data-Engineering-Planning, Product-Analytics

Mar 20 2023

nettrom_WMF created T332621: Add log_search to monthly sqoop list.
Mar 20 2023, 5:40 PM · Data Pipelines, Data-Engineering-Planning, Product-Analytics
nettrom_WMF moved T332441: Quick Analysis: impact of adding Thanks to more pages from Top Product Priorities to In Progress on the Growth-Team (Current Sprint) board.
Mar 20 2023, 5:15 PM · Product-Analytics (Kanban), Growth-Positive-Reinforcement, Growth-Team (Current Sprint)
nettrom_WMF moved T332441: Quick Analysis: impact of adding Thanks to more pages from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Mar 20 2023, 5:15 PM · Product-Analytics (Kanban), Growth-Positive-Reinforcement, Growth-Team (Current Sprint)
nettrom_WMF edited projects for T328055: Positive Reinforcement: evaluate Leveling Up leading indicators, added: Product-Analytics (Kanban), Growth-Team (Current Sprint); removed Product-Analytics, Growth-Team.

Moving this to the Product Analytics kanban board and the Growth Team's current sprint board as this work will need to start either this week or next.

Mar 20 2023, 4:56 PM · Growth-Team (Current Sprint), Product-Analytics (Kanban), GrowthExperiments-LevelingUp

Mar 18 2023

nettrom_WMF reopened T330379: Set up Herald rule to automatically answer some common newcomer questions as "Open".

Reopening this because this regular expression:

Hi and thank you for your interest! Please check thoroughly https://www.mediawiki.org/wiki/New_Developers (and all of its communication section!). The page covers how to get started, assigning tasks, task status, how to find a codebase, how to create patches, where to ask general development questions and where to get help with setup problems, and how to ask good questions. Thanks a lot! :)

Mar 18 2023, 12:52 AM · Phabricator, Developer-Advocacy (Jan-Mar 2023)
nettrom_WMF added a project to T332441: Quick Analysis: impact of adding Thanks to more pages: Product-Analytics (Kanban).

Adding this to the Product Analytics kanban board, keeping the High priority. The plan is to start this work on March 20 to get an initial perspective on the usage of Thanks as it relates to edits made in the most recent months, then go on from there based on what our needs are.

Mar 18 2023, 12:28 AM · Product-Analytics (Kanban), Growth-Positive-Reinforcement, Growth-Team (Current Sprint)

Mar 17 2023

nettrom_WMF added a comment to T325117: Personalized Praise: Instrumentation.

The "Send appreciation" button only opens DiscussionTools with a message prefilled (regardless of user's DiscussionTools-related preferences). Once DiscussionTools's interface loads, we do not control the workflow anymore. Any instrumentation that normally works for topics added through DiscussionTools should work for Send appreciation button as well. Checking whether mentors tend to click the button without actually sending anything is indeed a good idea.

Should we revisit instrumentation for the talk page (ie. tracking of what happens on the talk page)? I'm afraid that hooking into DiscussionTools could be fairly tricky, and I'd prefer letting DiscussionTools work mostly independently on our features.

Mar 17 2023, 6:32 PM · MW-1.41-notes (1.41.0-wmf.11; 2023-05-30), Product-Analytics, Growth-Team (Current Sprint), GrowthExperiments-PersonalizedPraise, GrowthExperiments-Mentorship, GrowthExperiments-MentorDashboard, Growth-Positive-Reinforcement
nettrom_WMF awarded T260386: Allow login to JupyterHub via CAS a Mountain of Wealth token.
Mar 17 2023, 4:06 PM · Data-Engineering, Data-Engineering-Jupyter, Infrastructure-Foundations, User-MoritzMuehlenhoff, CAS-SSO

Mar 16 2023

nettrom_WMF added a comment to T325117: Personalized Praise: Instrumentation.

@Urbanecm_WMF and @KStoller-WMF : I've completed drafting out the instrumentation in the measurement specification and copied everything over to the task description here. I deleted the "notes" section because all of the notes should now be reflected in the events that we'd like to capture.

Mar 16 2023, 9:47 PM · MW-1.41-notes (1.41.0-wmf.11; 2023-05-30), Product-Analytics, Growth-Team (Current Sprint), GrowthExperiments-PersonalizedPraise, GrowthExperiments-Mentorship, GrowthExperiments-MentorDashboard, Growth-Positive-Reinforcement
nettrom_WMF updated the task description for T325117: Personalized Praise: Instrumentation.
Mar 16 2023, 9:37 PM · MW-1.41-notes (1.41.0-wmf.11; 2023-05-30), Product-Analytics, Growth-Team (Current Sprint), GrowthExperiments-PersonalizedPraise, GrowthExperiments-Mentorship, GrowthExperiments-MentorDashboard, Growth-Positive-Reinforcement

Mar 15 2023

nettrom_WMF added a comment to T330637: Leveling up: Instrumentation for "Try a new task" dialog.

It seems there's another problem, though, because I see multiple impression events for the post-edit dialog after dismissing the try new task panel:

Mar 15 2023, 3:14 PM · MW-1.41-notes (1.41.0-wmf.1; 2023-03-20), GrowthExperiments-LevelingUp, Growth-Team (Current Sprint), Growth-Positive-Reinforcement

Mar 14 2023

nettrom_WMF added a comment to T304495: Implement database storage for Welcome Survey.

Adding myself as a subscriber in case we do decide to create a new table. That would require the Welcome Survey aggregation script to be updated to use the new data source. It should not be seen as an argument for or against any particular solution, instead it's just a change in data that I'll need to adapt to. Carry on.

Mar 14 2023, 4:13 PM · Growth-Team, Growth-Scaling, Technical-Debt, Performance Issue, GrowthExperiments

Mar 13 2023

nettrom_WMF added a comment to T330637: Leveling up: Instrumentation for "Try a new task" dialog.

I had just completed my 5th copyedit when I received this panel. The next task type I should try is "links". The edit-count-for-task-type is 4 because that is the value shown when I opened the page. I should probably revise that to show 5.

Mar 13 2023, 10:32 PM · MW-1.41-notes (1.41.0-wmf.1; 2023-03-20), GrowthExperiments-LevelingUp, Growth-Team (Current Sprint), Growth-Positive-Reinforcement
nettrom_WMF closed T330614: New Impact module's empty state on mobile: research spike as Resolved.

I've dug into this phenomenon and found as follows:

Mar 13 2023, 7:41 PM · Product-Analytics (Kanban), GrowthExperiments-ImpactModule, Growth-Team (Current Sprint)
nettrom_WMF closed T330614: New Impact module's empty state on mobile: research spike, a subtask of T327581: New Impact Module: experiment analysis, as Resolved.
Mar 13 2023, 7:41 PM · Product-Analytics (Kanban), Growth-Positive-Reinforcement, GrowthExperiments-ImpactModule, Growth-Team (Current Sprint)

Mar 10 2023

nettrom_WMF added a comment to T322587: Suggested edits module: description + create an article improvements.

@KStoller-WMF @nettrom_WMF this one does not need to be gated behind the LevelingUp feature flag, nor hidden from users in the oldimpact experiment group, is that correct? (Put another way, it can be merged and released to everyone when it's finished, correct?)

Mar 10 2023, 4:31 PM · MW-1.40-notes (1.40.0-wmf.27; 2023-03-13), GrowthExperiments-LevelingUp, Growth-Positive-Reinforcement, Growth-Team (Current Sprint)

Mar 8 2023

nettrom_WMF added a comment to T330637: Leveling up: Instrumentation for "Try a new task" dialog.

@nettrom_WMF I didn't implement the feature in this way in T322386: Leveling up: "Try a new task" dialog. The user will only be prompted to try a new task if there is at least one available task matching the new task type and their existing topic filters. I'll ask in T322386 if that's acceptable for this iteration.

Mar 8 2023, 11:34 PM · MW-1.41-notes (1.41.0-wmf.1; 2023-03-20), GrowthExperiments-LevelingUp, Growth-Team (Current Sprint), Growth-Positive-Reinforcement
nettrom_WMF added a project to T331495: Quick Analysis: Thank You Pages: custom account creation pages for sv, it, ja, fr, nl: Product-Analytics.
Mar 8 2023, 6:19 PM · Product-Analytics (Kanban), Growth-Team (Current Sprint)

Mar 6 2023

nettrom_WMF moved T330614: New Impact module's empty state on mobile: research spike from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Mar 6 2023, 10:48 PM · Product-Analytics (Kanban), GrowthExperiments-ImpactModule, Growth-Team (Current Sprint)

Mar 1 2023

nettrom_WMF added a comment to T330727: Leveling up: Instrumentation for new post-edit dialog for non-suggested edits.

@nettrom_WMF FWIW it doesn't seem like events right before navigation (such as button click events when the button is a link or JS redirect) get logged. This is not specific to this dialog (I can see that the dialog triggers the mw.track event but then EventLogging doesn't react to it) - maybe caused by the EventLogging library rewrite a while ago? From skimming the metric library code the handling of this scenario seems solid, so not sure what's going on.

I assume in this specific case it is not super important as there will be a homepagevisit event right afterwards anyway.

Mar 1 2023, 6:22 PM · MW-1.40-notes (1.40.0-wmf.26; 2023-03-06), Growth-Team (Current Sprint), GrowthExperiments-LevelingUp
nettrom_WMF added a comment to T330727: Leveling up: Instrumentation for new post-edit dialog for non-suggested edits.

@nettrom_WMF does that sound OK? Especially the type field in action_data that differentiates post-edit notices after suggested vs. non-suggested edits, so far we only had it in the impression action - do you want to stick with that and use the session ID to match further events, or put type into the action data for all events to differentiate them from normal post-edit? (Or maybe use a new context value like postedit-nonsuggested? That seems the cleanest to me.) Currently for link-click actions the action data is just the link name, without any key-value structure; we should probably change that (here and elsewhere for consistency) if we want to add another piece of information for link click events.

Mar 1 2023, 1:12 AM · MW-1.40-notes (1.40.0-wmf.26; 2023-03-06), Growth-Team (Current Sprint), GrowthExperiments-LevelingUp

Feb 28 2023

nettrom_WMF added a comment to T330766: Decommission the EditorActivation instrument.

This task came up in the Product Analytics team's board refinement meeting, and I wanted to chime in and confirm that the Growth team doesn't use this instrumentation in our analyses. Or in other words, from the perspective of the Growth team this instrumentation is safe to decommission.

Feb 28 2023, 6:11 PM · MW-1.41-notes (1.41.0-wmf.2; 2023-03-27), Patch-For-Review, Data-Engineering, Technical-Debt, MediaWiki-extensions-WikimediaEvents, Product-Analytics, Event-Platform Value Stream
nettrom_WMF claimed T330614: New Impact module's empty state on mobile: research spike.
Feb 28 2023, 4:55 PM · Product-Analytics (Kanban), GrowthExperiments-ImpactModule, Growth-Team (Current Sprint)
nettrom_WMF moved T329907: [REQUEST] Analysis of available data on clients calling individual services through RESTbase from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Feb 28 2023, 4:53 PM · API Platform (RESTbase Deprecation Roadmap), Product-Analytics (Kanban)

Feb 27 2023

nettrom_WMF moved T327581: New Impact Module: experiment analysis from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Feb 27 2023, 5:06 PM · Product-Analytics (Kanban), Growth-Positive-Reinforcement, GrowthExperiments-ImpactModule, Growth-Team (Current Sprint)
nettrom_WMF moved T327581: New Impact Module: experiment analysis from Ready for Development to In Progress on the Growth-Team (Current Sprint) board.
Feb 27 2023, 5:06 PM · Product-Analytics (Kanban), Growth-Positive-Reinforcement, GrowthExperiments-ImpactModule, Growth-Team (Current Sprint)

Feb 22 2023

nettrom_WMF closed T272365: [Scaling] Investigate if we need treatment groups on all wikis for some features as Declined.

The team decided first to stop experimenting on smaller wikis, then removed the control group altogether in T320876. No need to keep this task open.

Feb 22 2023, 6:30 PM · Growth-Team-Filtering, GrowthExperiments-Homepage, Growth-Scaling, Growth-Team
nettrom_WMF added a comment to T328060: Leveling up: Instrumentation.

The Levelling up instrumentation section in the measurement specification is now complete and ready for review and/or implementation. I'll be in charge of updating the measurement specification based on any changes that come out of the development process.

Feb 22 2023, 12:05 AM · Growth-Team (Current Sprint), GrowthExperiments-LevelingUp
nettrom_WMF added a subtask for T317813: [EPIC] Positive Reinforcement: Leveling Up : T328060: Leveling up: Instrumentation.
Feb 22 2023, 12:03 AM · GrowthExperiments-LevelingUp, Growth-Team (Current Sprint), Epic
nettrom_WMF added a parent task for T328060: Leveling up: Instrumentation: T317813: [EPIC] Positive Reinforcement: Leveling Up .
Feb 22 2023, 12:03 AM · Growth-Team (Current Sprint), GrowthExperiments-LevelingUp
nettrom_WMF renamed T328060: Leveling up: Instrumentation from Instrumentation for Leveling up to Leveling up: Instrumentation.
Feb 22 2023, 12:02 AM · Growth-Team (Current Sprint), GrowthExperiments-LevelingUp

Feb 16 2023

nettrom_WMF added a comment to T322386: Leveling up: "Try a new task" dialog.

One thing I noticed when I was just checking the mocks is that the proposal is to show the dialogue on every k edits (with k=5) as the default. So if I've made 10 add a link edits, I should see the dialogue again (provided I haven't checked "don't show me this again"). Are we still planning on doing that or have these changed to milestones that trigger the dialogue and then never again?

Feb 16 2023, 7:29 PM · MW-1.41-notes (1.41.0-wmf.4; 2023-04-10), MW-1.40-notes (1.40.0-wmf.25; 2023-02-27), GrowthExperiments-LevelingUp, Growth-Team (Current Sprint), Growth-Positive-Reinforcement

Feb 15 2023

nettrom_WMF closed T305406: Newcomer tasks: quick analysis of AND selection usage as Resolved.

We've done a first pass of this analysis by looking at users who exit the topic selection dialogue on wikis where this feature has been available. While that doesn't capture users who turn the feature on during the module initialization process, our findings suggested changes to the UX was needed (ref some of the related tasks from December and January) to improve the user experience. This conclusion wouldn't change if we extended the dataset, and so we decided to instead prioritize other work.

Feb 15 2023, 5:24 PM · Growth-Team (Current Sprint), Product-Analytics (Kanban), GrowthExperiments-NewcomerTasks
nettrom_WMF closed T305406: Newcomer tasks: quick analysis of AND selection usage, a subtask of T305399: Newcomer tasks: deploy AND selection to pilot wikis, as Resolved.
Feb 15 2023, 5:24 PM · MW-1.39-notes (1.39.0-wmf.10; 2022-05-02), GrowthExperiments-NewcomerTasks, Growth-Team (Current Sprint)

Feb 14 2023

nettrom_WMF added a comment to T328288: Leveling up: "Keep going" notification.

I dug into this as part of T322433 and found that the main limiting factor here is the threshold, and not time. The figure below exemplifies this, where the X-axis is time since first edit tagged "newcomer task" in hours, the Y-axis is the proportion of editors making a threshold, and each line is a different threshold (this figure is also on Commons as: Newcomer_task_milestones_by_hour_Feb_2023.png) It doesn't matter if we change the scale of the X-axis to minutes, hours, or days, the graph looks basically the same.

Feb 14 2023, 5:23 PM · MW-1.40-notes (1.40.0-wmf.27; 2023-03-13), GrowthExperiments-LevelingUp, Growth-Positive-Reinforcement, Growth-Team (Current Sprint)
nettrom_WMF moved T322433: Leveling up: suggested edit task completion analysis from Doing to Needs Sign-off on the Product-Analytics (Kanban) board.

This analysis is largely complete. We have an internal googledoc report, and I've gone ahead and created this on-wiki summary that goes through the main findings and has a couple of graphs to support it.

Feb 14 2023, 5:13 PM · Product-Analytics (Kanban), GrowthExperiments-LevelingUp, Growth-Positive-Reinforcement, Growth-Team (Current Sprint)

Feb 6 2023

nettrom_WMF added a comment to T328288: Leveling up: "Keep going" notification.

@KStoller-WMF @RHo : I noticed one thing about this notification when I was working on investigating baseline for this and checked the Figma designs: if we define this as written then the longest possible interval a newcomer can go between making their first suggested edit and the notification being sent is 9 x 120 hours, which is 1080 hours or 45 days. Or say a newcomer activates by making a suggested edit on the first day, then they can wait 4 days between each of the 8 subsequent edits and get the notification 5 days after that for like 37 days total.

Feb 6 2023, 10:16 PM · MW-1.40-notes (1.40.0-wmf.27; 2023-03-13), GrowthExperiments-LevelingUp, Growth-Positive-Reinforcement, Growth-Team (Current Sprint)

Jan 31 2023

nettrom_WMF added a project to T328055: Positive Reinforcement: evaluate Leveling Up leading indicators: Product-Analytics.
Jan 31 2023, 5:55 PM · Growth-Team (Current Sprint), Product-Analytics (Kanban), GrowthExperiments-LevelingUp
nettrom_WMF added a project to T328057: Leveling Up: experiment analysis: Product-Analytics.
Jan 31 2023, 5:54 PM · Product-Analytics, GrowthExperiments-LevelingUp, Growth-Team

Jan 30 2023

nettrom_WMF added a comment to T327849: GrowthExperiments: undefined is not a constructor (evaluating 'new Intl.Locale(language,languageOptions)').

@nettrom_WMF do you think we should log something in homepagemodule schema to indicate if the user's browser has Intl support?

Jan 30 2023, 6:26 PM · MW-1.40-notes (1.40.0-wmf.21; 2023-01-30), Growth-Team (Current Sprint), JavaScript, GrowthExperiments-ImpactModule