@mewoph : Can we add the reference to client_ip to https://gerrit.wikimedia.org/r/c/schemas/event/secondary/+/704402 so we also get that done and only have one version change to the schema? Based on the code for VirtualPageview that @Ottomata mentions above, all that should be needed is to add - $ref: /fragment/http/client_ip/1.0.0# to the referenced fragments.

In T287404#7247286, @mewoph wrote:

Hi @nettrom_WMF, would these additions make sense?

• close with active_interface=recommendedlinktoolbar_dialog for when the user clicks outside the link inspector to close it
• impression with active_interface=recommendedlinktoolbar_dialog for re-opening the link inspector
• reopen_dialog_click for clicking on the robot button --- for this, would machinesuggestions_mode make more sense for active_interface since recommendedlinktoolbar_dialog isn't shown?

In T287404#7240322, @mewoph wrote:

• For instrumentation, do we need new events for these new actions? Are there any other things that need to be instrumented?
  • Closing the link inspector
  • Clicking on the robot button
  • Re-opening the link inspector

In T287210#7231273, @Ottomata wrote:

@nettrom_WMF @MMiller_WMF Do either EchoMail or EchoInteraction need client_ip and/or geocoded data?

This work is completed and the notebook is on GitHub: https://github.com/nettrom/Growth-welcomesurvey-2018/blob/master/T275172_survey_aggregation.ipynb

The analysis has been updated to categorize users depending on whether they registered before or after deployment of Add a Link, in order to make comparisons between Add a Link and the unstructured link task easier, as well as to make it clearer that existing users were getting Add a Link almost exclusively. This could otherwise have created a lot of confusion, as prolific experienced users would make Add a Link look overwhelmingly good.

From what I've been able to find, this is the first time this has been requested, and so I'm unsure what exactly to ask for and how to do this. I'm hoping someone from Data Engineering can help us out. I suspect @Ottomata is the right person to tag first, so adding him here.

This looks good to me! Having it as a separate event should work great (the other option I'd suggest would be to use action_data to make it possible to distinguish them, but I don't think that's necessary, let's go with the current plan!)

Looks good to me! I +1'ed the schema patch, and if a +2 is needed at some point just let me know and I can do that too.

In T286746#7227261, @toberto wrote:

@nettrom_WMF Hi Morten! I've signed the doc and completed info up above. One question: is the approver my manager or someone else? Thanks!

In T279105#7227232, @MPhamWMF wrote:

@nettrom_WMF , thanks! I'm currently getting a Presto error that says access denied and that I should contact chart owners for assistance. Have tried reloading a few times but didn't seem to help. Any advice?

@MPhamWMF : We now have a test dashboard for user satisfaction (item number 2), it's available here: https://superset.wikimedia.org/r/628

I love that we have active_interface, and I think editmode_confirmation_dialog makes great sense to use for this!

I've added @cchen as the second assignee to this task. She'll be picking up this task as we transition my work from Search and Structured Data. So far that means she'll start working on item number 3.

Closing this task as resolved to not keep this around as an eternal phab task that Marshall and I can play Pong with by sending back and forth. We might want to make changes to the notebook to support Add a Link more specifically, or other things as the team deploys new features. If so, we'll discuss and create a new task for that and prioritize as needed.

The Growth features were deployed to the Spanish Wikipedia on 2021-04-21 (ref: T278235#7025299). I'd like to use MediaWiki history for this because it's generally accepted as the canonical dataset for user and editing data, and because that's what I've used for NEWTEA revisited so I have a data gathering pipeline that's easy to adapt. Based on the key results and other metrics in T286796, we don't really need editing data from August, so we can use the July snapshot of MediaWiki history (which will be available in early August).

In T286523#7216092, @Tgr wrote:

I guess it might not be possible if there was only one signup per wiki, but it would be nice to see a mobile / desktop split too, as the UI is quite different so it's theoretically possible we only broke one of them.

This work has now been completed. The query that was run is below, which for simplicity doesn't exclude known test accounts. I also ran a query to manually inspect each account and removed one test account from the results. We then got the following table with number of signups:

Case in point, I'm fitting one of these models for NEWTEA revisited (T270786) and as of yesterday it's running time is 338 hours. Here's a screenshot of htop, running time is the 11th column (TIME+). While some of this might be contributed to the model specification or the data, I wouldn't be surprised to see 24–48 hours to completion because that's what it took in previous analysis.

I've updated the task description with a link to the notebook so it's easier to do this if questions related to this get asked and prioritized in the future. For now, closing as resolved.

@mewoph – These events all look correct to me! I also reviewed the schema patch and +1'ed it. Let me know if a +2 is needed there, as I'd be happy to do that as well.

In T272664#7188281, @mewoph wrote:

For the impression event, should we send one when the new task is shown? If so, should we use the existing postedit-impression event or a new one? I was looking at the data sent for postedit-impression and it looks like only the information about the task that was just saved is sent so upon refresh (not the information about the next task).

In T285728#7184754, @RHo wrote:

Pinging @MMiller_WMF and @nettrom_WMF - should this be something we keep track of?

In T272664#7182737, @MMiller_WMF wrote:

@nettrom_WMF -- I wanted to ping you again about how you think we should instrument the click on this button. Would you like there to be an explicit event? What should it contain?

The analysis has been done and shared with the team. I uploaded the notebook for the analysis to GitHub. Next steps are sharing these more widely, which I'm leaving in the capable hands of @MMiller_WMF, but I can help where needed. The Growth team had many follow up questions, and we need to create a new task with those questions so they can be prioritized and answered.

In T285169#7176214, @CBogen wrote:

In T285169#7175918, @egardner wrote:

Here's a somewhat related question: should we add instrumentation to track when users click the "switch to special:search" link? Could be useful to know how many users are still reaching for the non-MediaSearch interface.

This would be useful, but I wonder if we can already gather than info just based on usage data for the Special:Search interface, without needing to specifically know whether users are switching over. What do you think @nettrom_WMF ?

In T284481#7173617, @MMiller_WMF wrote:

@kostajh -- yes, I think we should deploy with the A/B test like we did in our pilot wikis, where half of the users get the structured task and the other half get the unstructured task. @nettrom_WMF, do you agree with that? Would having that additional data from these larger wikis help us see the differences that "add a link" makes?

This is a task that would be interesting to do. At the same time, it's unclear when or if it'll be a priority for the Growth team. Should the need to understand more about how newcomers search for how-to information come up at some point in the future, I suggest creating a new task with a fresh set of questions based on the team's needs at that time.

This is a task that would be interesting to do. At the same time, it's unclear when or if it'll be a priority for the Growth team. Should the need to understand more about usage of Echo come up at some point in the future, I suggest creating a new task with a fresh set of questions based on the team's needs at that time.

We've changed the start module to no longer have a CTA to create a user page, so I'm declining this.

The dashboards that used this data are no longer in use, and there has been no feedback suggesting a need for the data. I see no need to keep this task around.

As obvious from the earlier discussion in this task, completing this is complicated. It's unclear at this point that this is still within the scope of the Growth team's work. In summary: declining this.

Tested this in Firefox on my phone and it redirects just fine to the mobile site there, so it looks like this only happens if you pull up the mobile site on a non-mobile browser because then the redirect doesn't happen. Closing this as declined.

@MMiller_WMF : I set up a time for us to meet early next week to chat about this project so I don't miss anything important.

@jwang : I fiddled with the active editors query a bit and created a notebook that combines it with the Growth dataset of registrations from October/November 2020 to count the number of them that edits every month from Nov 2020 through May 2021, split by whether they had the Growth features or not. You can find that notebook here. Let me know what questions you might have about this!

Closing as resolved, thanks!

I've drafted a slide deck and handed it off to Marshall for review. Once we've resolved any outstanding issues, I'll write up a summary and resolve this task.

Moving this out of Product Analytics' "tracking" column as the edit tags have been deployed. We'd now like to get some example charts in Superset based on edits_hourly.

In T284119#7132671, @Tgr wrote:

In T284119#7132419, @Ragesoss wrote:

I think it makes sense to add geForceVariant=control to all the account creation links on both Wiki Education Dashboard and Programs & Events Dashboard, as this won't matter for languages that aren't currently doing testing and it will keep new users from en.wiki editathons and such out of the experiment group over the next few months as well.

@nettrom_WMF any concerns about that?

@kostajh @Tgr : From what I know, the way we use EditAttemptStep means that we set the is_oversample field to true whenever we're logging a session that wouldn't normally be logged. Also as far as I know other people who work with that data know about this and will use that field to exclude oversampled sessions, but I'll mention it in our team meeting tomorrow just in case.

Adding Product Analytics to the project tags so we can triage this in board refinement tomorrow.

@MPhamWMF : I've been working on putting together a notebook to aggregate user engagement metrics (full text query sessions with a dwell time of > 10 seconds), so we have a working example of using data from SearchSatisfaction. From digging into data from that schema, it appears to only be instrumented on desktop. Could you check in with the engineers on the team about whether that's the case? I want to make sure I'm not looking for non-existent data.

The notebook has been sent to Marshall for review, moving to "Needs Sign-off".

I've added window-open-from-command to our data gathering, taking into account that it represents multiple ways of working with media (both inserting new media and editing existing media). It's represented as the "other" path in the dashboard. The dashboard's description of the paths has also been updated. The usage of search in this path is low, and we end up seeing a lot of variance in the proportion of completed edits in this path as a result.

Thanks, @DAbad ! The Product Analytics team will prioritize this request in our next board refinement meeting (June 1).

The answer is that the user can open the media dialogue for example by double clicking on an image, or dragging and dropping an image into the editor. Both of those approaches result in a window-open-from-command event. I noticed that these events are more frequent on the phone platform, compared to the desktop. Either way, moving this back into "Doing". I'll then update the notebook to capture these and pull in some historic data.

I'm closing this as resolved as we're also tracking this work in T260254

Moving this back to "Needs investigation" while I ask the web folks how window-open-from-command works.

EditorJourney was shut off in T252391. While the Welcome Survey is now on lots of wikis and we can ask lots of interesting questions to the data in it, our time is limited. If insights into WS is desired, we might want to look into collaborating with a research institution, or elevating the priority of that kind of work (e.g. as an OKR).

This has been done, I think. Or even though it might not have been, it's no longer relevant. Closing as resolved.

Declining this as we're doing a similar analysis for NEWTEA Revisited in T270786.

Declining this as we're doing a similar analysis for NEWTEA Revisited in T270786.

Moving this to In Progress on the Growth sprint board, and onto the Product Analytics team's kanban board and into Doing. Because I'm doing this.

Similarly as in T282260, I tested this out with R 3.6.1 too. The installation fails with the same error message that's reported.

I ssh'ed to stat1006 and created a new stacked conda environment and activated it. Running R (/usr/lib/anaconda-wmf/bin/R specifically, which is 3.6.1 as listed above) and then running install.packages('nloptr') successfully installs the package for me. The library file it grabs is https://cran.rstudio.com/src/contrib/nloptr_1.2.2.2.tar.gz

In T278177#7061224, @kostajh wrote:

The "Learn more" link appears if it is set in MediaWiki:NewcomerTasks.json. Some wikis will have this, others may not. For example, cswiki has a "learnmore" link defined for the references task type but not for the new link-recommendation one https://cs.wikipedia.org/wiki/MediaWiki:NewcomerTasks.json

In T278177#7051361, @kostajh wrote:

In T278177#7046405, @nettrom_WMF wrote:

1. Onboarding, user clicks the "Learn more" link. I'd expect that to be captured as a action="link_click" but what is action_data set to? And in what interfaces is that available? There's some documentation of how action_data is set but not for the onboarding steps.

So, action=link_click and active_interface is onboarding_step_1, and the additional action_data is whether the "Don't show again" box is checked or unchecked. Does that sound OK?