User Details
- User Since
- Aug 21 2018, 8:23 PM (290 w, 6 d)
- Availability
- Available
- IRC Nick
- Nettrom
- LDAP User
- Unknown
- MediaWiki User
- MWang (WMF) [ Global Accounts ]
Tue, Mar 5
Tue, Feb 27
I agree with what's being said here. The performer values would reflect what the current user is, and if that user successfully logs in it'll reflect their logged in status. If the login attempt fails, it'll reflect their either anon or temp user status.
Feb 5 2024
Jan 30 2024
Jan 23 2024
A written summary of our findings have been published on the Positive reinforcement project page in the section "Personalized praise experiment results". The notebooks associated with this analysis have been uploaded to this Gitlab repository and carry the name of this phab task. Closing as resolved.
A written summary of our findings have been published on the Positive reinforcement project page in the section "Personalized praise experiment results". The notebooks associated with this analysis have been uploaded to this Gitlab repository and carry the name of this phab task. Closing as resolved.
Jan 22 2024
The notebooks for the previously mentioned preliminary analysis of the effects of mentorship have been added to this Gitlab repository using a naming convention that reflects that they're associated with this phab task. Resolving this parent task now, and I'll close the child tasks once their notebooks are published as well.
Out of curiosity, and in hoping to make this easier to fix, I grabbed daily aggregates for October 2023. Looks like something was deployed the week of Oct 23 as that's when the proportion increases from the expected 6–7%:
Jan 9 2024
Jan 8 2024
Closing this task as declined for now. If the need for additional data analysis comes up, feel free to file another request.
Jan 4 2024
I've collected data and calculated the statistics for both of the campaigns using the date/time ranges listed in the task description.
Jan 3 2024
I've tested creating new environments on two stat machines with different OSes, and both with and without adding the conda-meta/pinned file. From what I can tell, the conda-forge R 4.2.3 package no longer results in a broken environment because it works just fine without the pin-file. Looking at the file listing on conda-forge, there have been several new versions of the package released since this bug was opened. Based on documentation I've found from googling it should be possible to install a specific build version, but it looks like the Mamba solver doesn't support it properly yet. It doesn't make sense to try this with the classic solver because that wasn't able to resolve the dependencies when I tried that back in August.
Dec 12 2023
Dec 6 2023
I've collected data to understand the effects of this campaign after about one week. During the data gathering for this update, I discovered a data quality issue in one of the underlying datasets that rendered us unable to correctly identify reverts (this has been filed as T352899). I was able to work around this by modifying the query that collects editing data.
Nov 29 2023
I've run the numbers on the currently available data^1 to get a sense of where we're at. Overall, we get the following counts and proportions of page views, "unique visitors" (based on combinations of IPs and user-agents for page views of the account creation page), registrations, and activations.
Nov 28 2023
Nov 15 2023
Nov 14 2023
Nov 10 2023
Nov 7 2023
Oct 24 2023
Oct 10 2023
Oct 2 2023
Sep 25 2023
Sep 19 2023
Sep 18 2023
This is no longer relevant, closing as invalid.
Sep 8 2023
I pulled data from Growth's KPI Grafana board from 2023-07-31 to 2023-08-28 (available here) for Section-Level and Article-Level suggestions. This timeframe was chosen because it should not be as much affected by the June/July slump in activity that we often see on the wikis. The end date is limited by the team shutting off image suggestions in late August (see T345188 for more information). This data range covers four whole weeks of data. While this dataset does not allow us to separate it by platform (desktop and mobile web), nor does it allow us more fine-grained user filtering, it was easily available and provides us with a reasonably good picture that's sufficient for this kind of analysis at this time.
Aug 29 2023
Aug 28 2023
Moving this out of the various doing/in progress columns as I'll instead be focusing on Levelling Up for a bit.
Aug 24 2023
Reducing the priority on this to Medium since we've completed the time-sensitive part of this.
Aug 23 2023
I've now completed testing using the EditAttemptStep data. There's a summary of the approaches, run times, and some observations and notes in this notebook on Gitlab.
Aug 18 2023
@KStoller-WMF : I made some changes to the summary of the findings. There's now a graph for the key finding, and I elaborated on the work we've done to investigate and mitigate the issue. I'll close this task as resolved, but you should feel free to edit the results section or reach out to me with proposed changes as you see fit.
Aug 14 2023
I've completed a preliminary analysis of mentorship and made a writeup of the findings on mw:Growth/Positive reinforcement/Mentorship preliminary analysis, August 2023. The goal of this analysis was to provide some initial insights into mentorship in support of the upcoming Wikimania discussion. We therefore focus on high-level questions such as what proportion of newcomers ask questions, how quickly they do, and to understand if mentorship has an impact on the Growth KPIs. The writeup is therefore also fairly high-level and without specifics for now, but does have some supporting visualizations.
Aug 8 2023
Aug 7 2023
@BTullis : Thank you for the update! While I no longer maintain the chart where this was an issue, I think switching the chart type to an Echarts type is a perfectly reasonable solution. I'd be happy to share this fix more widely on Slack as well so that others know about it, because I wasn't aware that there were deprecated chart types. Let me know if there are other chart types too that we should be aware of, I wasn't able to easily find a list of deprecated types.
I'll start digging into query performance this week. In the meantime, I want to make sure that previous work is documented. The experiments and notes around working with these flattened tables can be found in this notebook on gitlab.
Jul 27 2023
Jul 24 2023
Jul 21 2023
Added the Product Analytics tag, and myself as secondary assignee on this task. I've started looking at this and will continue that next week.
Jul 19 2023
From what I can tell we've deployed a patch to have the new impact module only switch out of deactivated state through article edits (Main namespace) per T338640, but that there was also a bug that affected it that got fixed in T341994. I'm going to restart the experiment clock in the notebook that aggregates statistics so that we can monitor whether there's a change after this patch. The aggregated leading indicators from the Levelling Up experiment will continue to exist in our database for comparison.
Jul 17 2023
Adding this to the Product Analytics kanban board. I'll pick this up and grab statistics on “How many newcomers/editors have at least one topic defined?” later this week.
Jul 7 2023
I've added this to Product Analytics tag and reset the priority so the PA team can triage it there. Given the timeliness of this work in preparation for Wikimania, I think the High priority should be re-added at triage. I'll then also move it to the PA kanban board to pick it up next week.
Jun 27 2023
Jun 26 2023
Jun 21 2023
Jun 16 2023
I introduced console-based event QA to @KStoller-WMF in our 1:1. We went through most of the workflow and checked the various events. From what we found the instrumentation is working correctly, there's nothing there that blocks deployment.
Jun 15 2023
Jun 9 2023
Jun 6 2023
@Urbanecm_WMF : From what I can tell, the notifications are being sent out by MediaWiki but not captured as events in event.mediawiki_mentor_dashboard_personalized_praise. The first query below is the Hive/Spark query of the event data, which returns a count of zero. The second I've run for each of the four pilot wiki MariaDB replicas (e.g. analytics-mysql eswiki --use-x1 on a stat* machine), and they all return a non-zero result.
Jun 5 2023
I've taken a first look at this by investigating two proportions. First, the proportion of newcomers with constructive article activation. Second, the proportion of those activating through Suggested Edits. While that doesn't specifically answer if users are able to find Suggested Edits and/or activate it, it should give us an indication of whether we are running into issues.
I'm reopening this after we discussed this in light of the recent deployment of the New Impact Module to additional wikis in T336203. There's a difference in how the Suggested Edits module behaves on desktop and mobile that we're interested in understanding. On desktop, users have to initialize the module before getting any suggestions, whereas on mobile the module is already initialized. Timeboxed to one hour to grab fresh data and investigate this four the pilot wikis, with an additional hour if we want to expand it to the nine other Wikipedia wikis we're currently deployed to.
Jun 1 2023
May 31 2023
I added a QA step to the acceptance criteria since we'll be wanting to verify that this doesn't alter the logged data in any way. Happy to help out with that as needed, let me know! Moving this to "Tracking" on the Product Analytics board for now.
May 30 2023
Will this be deployed to the test group (the first 10 wikis) in the same A/B test configuration that we have on the pilot wikis (50% get the new, 50% get the old), or will everyone get the new Impact module? It'll be a lot easier to measure the impact if it's an A/B test, but I'm not sure if that'll happen automagically or if it requires a lot of extra work?