- User Since
- Aug 21 2018, 8:23 PM (248 w, 5 d)
- IRC Nick
- LDAP User
- MediaWiki User
- MWang (WMF) [ Global Accounts ]
Fri, May 26
Wed, May 24
With the publication of the results for the Levelling Up leading indicators, as well as the planned leading indicators for Personalized praise on the project page, this task can be resolved.
The findings for all indicators have been posted on wiki here: https://www.mediawiki.org/wiki/Growth/Positive_reinforcement#Levelling_up Closing this task as resolved.
Tue, May 23
Mon, May 22
Tue, May 16
@Urbanecm_WMF : I've reviewed the existing and proposed schemas and compared them to the specification. This is great work! Almost everything looks according to spec from what I can tell and will enable us to answer the questions we have. Here are the two questions I've got:
Reducing the priority of this for now and moving it to "Needs Investigation" while I follow up with @VirginiaPoundstone on what API Platform needs.
I've chatted with @cchen about what the Structured Data team needs so that we're not duplicating efforts, and learned that there isn't currently any overlap between us. Adding this comment mainly to document that the SD team will send out notifications about section-level images to experienced editors. If the Growth team at some point chooses to measure something like edits, we might want to make sure we take that into account.
Wed, May 10
Mon, May 8
Adding the Product Analytics tag and resetting the priority so that Product Analytics can triage this tomorrow. Since this is planned for late May/early June the work needed on this at the moment is more of a planning nature, which to me fits into PA's "medium priority".
Fri, May 5
We've examined this phenomenon in several different ways looking for patterns, and not found any specific pattern that provides clues as to why this is happening. This analysis started out by looking at the three breakdowns suggested in the task, as well as by wiki since it was trivial to split the data that way too. Here's what we found:
Apr 12 2023
Apr 11 2023
I've gathered all the data from English Wikipedia and in the process fixed a bug in our processing so that we also correctly count mobile page views after the desktop to mobile redirect. We do not report data for the French and Dutch campaigns because of low numbers.
Apr 10 2023
Reducing the priority of this to Medium as the majority of this work is done and not as urgent. I'll also remove the due date.
The leading indicators have been defined and I've created a notebook to aggregate data and set it up to run once a day. We also have a Superset dashboard to monitor the indicators. What remains is:
Apr 5 2023
Editing this to remove the posted data because the more recent table posted below is the authoritative one.
Apr 3 2023
@Ottomata : as you've discovered, @Trizek-WMF is already assigned to the wmf group and is WMF staff (and also subscribed to this task).
Mar 29 2023
Mar 28 2023
This work has been done multiple times over since Newcomer Tasks was deployed, although not documented here. Closing as resolved.
Mar 27 2023
@HNordeenWMF : I'm currently working on gathering these statistics and noticed that T321370 specifies two languages for each campaign: the local language of the wiki as well as English. The language is then reflected in the uselang parameter in the URL. When grabbing the data I noticed this parameter doesn't show up (except for a few test visits), and from playing around with the Thank You page I did not see the uselang parameter in the link to account creation. I'm not sure if this is a bug or not?
Mar 24 2023
This analysis has been completed. We created a slide deck presenting the findings and decided to upload that to Commons for wider accessibility: https://commons.wikimedia.org/wiki/File:T332441_-_Thanks_Usage_-_Introduction.pdf The notebooks used for data gathering and visualization are in this GitHub repository
Mar 21 2023
I've updated the task description to reflect the following changes:
Found this after my curiosity made me check if log_search is available on the Wiki replicas. A couple of fields that might be considered non-private are:
I got curious and checked whether log_search is available on the standard replicas, and it isn't because it contains private data. T85756 exists to also make it available more widely with private data redacted. If it's sqooped it would need to be marked as private, similarly as some of the other tables we have.
Mar 20 2023
Moving this to the Product Analytics kanban board and the Growth Team's current sprint board as this work will need to start either this week or next.
Mar 18 2023
Reopening this because this regular expression:
Hi and thank you for your interest! Please check thoroughly https://www.mediawiki.org/wiki/New_Developers (and all of its communication section!). The page covers how to get started, assigning tasks, task status, how to find a codebase, how to create patches, where to ask general development questions and where to get help with setup problems, and how to ask good questions. Thanks a lot! :)
Adding this to the Product Analytics kanban board, keeping the High priority. The plan is to start this work on March 20 to get an initial perspective on the usage of Thanks as it relates to edits made in the most recent months, then go on from there based on what our needs are.
Mar 17 2023
Mar 16 2023
@Urbanecm_WMF and @KStoller-WMF : I've completed drafting out the instrumentation in the measurement specification and copied everything over to the task description here. I deleted the "notes" section because all of the notes should now be reflected in the events that we'd like to capture.
Mar 15 2023
Mar 14 2023
Adding myself as a subscriber in case we do decide to create a new table. That would require the Welcome Survey aggregation script to be updated to use the new data source. It should not be seen as an argument for or against any particular solution, instead it's just a change in data that I'll need to adapt to. Carry on.
Mar 13 2023
I've dug into this phenomenon and found as follows:
Mar 10 2023
Mar 8 2023
Mar 6 2023
Mar 1 2023
Feb 28 2023
This task came up in the Product Analytics team's board refinement meeting, and I wanted to chime in and confirm that the Growth team doesn't use this instrumentation in our analyses. Or in other words, from the perspective of the Growth team this instrumentation is safe to decommission.
Feb 27 2023
Feb 22 2023
The team decided first to stop experimenting on smaller wikis, then removed the control group altogether in T320876. No need to keep this task open.
The Levelling up instrumentation section in the measurement specification is now complete and ready for review and/or implementation. I'll be in charge of updating the measurement specification based on any changes that come out of the development process.
Feb 16 2023
One thing I noticed when I was just checking the mocks is that the proposal is to show the dialogue on every k edits (with k=5) as the default. So if I've made 10 add a link edits, I should see the dialogue again (provided I haven't checked "don't show me this again"). Are we still planning on doing that or have these changed to milestones that trigger the dialogue and then never again?
Feb 15 2023
We've done a first pass of this analysis by looking at users who exit the topic selection dialogue on wikis where this feature has been available. While that doesn't capture users who turn the feature on during the module initialization process, our findings suggested changes to the UX was needed (ref some of the related tasks from December and January) to improve the user experience. This conclusion wouldn't change if we extended the dataset, and so we decided to instead prioritize other work.
Feb 14 2023
I dug into this as part of T322433 and found that the main limiting factor here is the threshold, and not time. The figure below exemplifies this, where the X-axis is time since first edit tagged "newcomer task" in hours, the Y-axis is the proportion of editors making a threshold, and each line is a different threshold (this figure is also on Commons as: Newcomer_task_milestones_by_hour_Feb_2023.png) It doesn't matter if we change the scale of the X-axis to minutes, hours, or days, the graph looks basically the same.
This analysis is largely complete. We have an internal googledoc report, and I've gone ahead and created this on-wiki summary that goes through the main findings and has a couple of graphs to support it.
Feb 6 2023
@KStoller-WMF @RHo : I noticed one thing about this notification when I was working on investigating baseline for this and checked the Figma designs: if we define this as written then the longest possible interval a newcomer can go between making their first suggested edit and the notification being sent is 9 x 120 hours, which is 1080 hours or 45 days. Or say a newcomer activates by making a suggested edit on the first day, then they can wait 4 days between each of the 8 subsequent edits and get the notification 5 days after that for like 37 days total.