Page MenuHomePhabricator

nettrom_WMF (Morten Warncke-Wang)
Staff Data Scientist, Product Analytics

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Thursday

  • Clear sailing ahead.

User Details

User Since
Aug 21 2018, 8:23 PM (112 w, 6 d)
Availability
Available
IRC Nick
Nettrom
LDAP User
Unknown
MediaWiki User
MWang (WMF) [ Global Accounts ]

Recent Activity

Yesterday

nettrom_WMF moved T265768: Statistics on media usage across Wikipedias from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Mon, Oct 19, 4:23 PM · Product-Analytics (Kanban), Structured-Data-Backlog
nettrom_WMF triaged T265768: Statistics on media usage across Wikipedias as Medium priority.
Mon, Oct 19, 4:23 PM · Product-Analytics (Kanban), Structured-Data-Backlog
nettrom_WMF triaged T265773: Statistics on play rates for audio and video files as Medium priority.
Mon, Oct 19, 4:22 PM · Structured-Data-Backlog, Product-Analytics
nettrom_WMF triaged T265774: Statistics on image clicks from Wikipedia articles across time as Medium priority.
Mon, Oct 19, 4:22 PM · Structured-Data-Backlog, Product-Analytics
nettrom_WMF moved T265773: Statistics on play rates for audio and video files from Triage to Needs Investigation on the Product-Analytics board.
Mon, Oct 19, 4:21 PM · Structured-Data-Backlog, Product-Analytics
nettrom_WMF moved T265774: Statistics on image clicks from Wikipedia articles across time from Triage to Needs Investigation on the Product-Analytics board.
Mon, Oct 19, 4:20 PM · Structured-Data-Backlog, Product-Analytics

Fri, Oct 16

nettrom_WMF added a comment to T265774: Statistics on image clicks from Wikipedia articles across time.

This statistic was mentioned in the Technology Department's Quarter in Review for Q4 of FY 19/20. Looking further, I found out that it comes from the Understanding Engagement with Images in Wikipedia research project. More detailed statistics can be found on the First Round of Analysis page, which I'll dig into further. Looks like T250154 is the parent task for this work.

Fri, Oct 16, 10:37 PM · Structured-Data-Backlog, Product-Analytics
nettrom_WMF added a project to T261343: Dashboard of multimedia usage on the Wikipedias: Epic.

Created subtasks for all five points, changing this to an epic and moving it to the Epics column on the Product Analytics board.

Fri, Oct 16, 10:28 PM · Epic, Structured-Data-Backlog, Product-Analytics
nettrom_WMF updated the task description for T261343: Dashboard of multimedia usage on the Wikipedias.
Fri, Oct 16, 10:27 PM · Epic, Structured-Data-Backlog, Product-Analytics
nettrom_WMF created T265774: Statistics on image clicks from Wikipedia articles across time.
Fri, Oct 16, 10:24 PM · Structured-Data-Backlog, Product-Analytics
nettrom_WMF added a comment to T265773: Statistics on play rates for audio and video files.

There's the MediaViewer schema, and there's data from it in the Data Lake. An investigation would be needed to understand what data is actually logged and whether that can answer this.

Fri, Oct 16, 10:23 PM · Structured-Data-Backlog, Product-Analytics
nettrom_WMF created T265773: Statistics on play rates for audio and video files.
Fri, Oct 16, 10:21 PM · Structured-Data-Backlog, Product-Analytics
nettrom_WMF updated subscribers of T265772: Statistics on dwell time and multimedia interaction with Wikipedia articles.

As far as I know, there is not any live instrumentation that would allow us to measure this. The SearchSatisfaction schema measures dwell time, but requires the user to reach a page through an on-wiki search, and we know that's not representative of how visitors reach us.

Fri, Oct 16, 10:20 PM · Structured-Data-Backlog, Product-Analytics
nettrom_WMF created T265772: Statistics on dwell time and multimedia interaction with Wikipedia articles.
Fri, Oct 16, 10:12 PM · Structured-Data-Backlog, Product-Analytics
nettrom_WMF added a comment to T265771: Measure how multimedia content is added to Wikipedia articles.

Based on my conversations with @cchen and @mpopov it looks like this will not be straightforward to do any time soon. If we're interested in understanding this based on existing edits we'll need to extract and process diffs between revisions.

Fri, Oct 16, 10:09 PM · Structured-Data-Backlog, Product-Analytics
nettrom_WMF created T265771: Measure how multimedia content is added to Wikipedia articles.
Fri, Oct 16, 10:05 PM · Structured-Data-Backlog, Product-Analytics
nettrom_WMF updated subscribers of T265768: Statistics on media usage across Wikipedias.

I've previously discussed something similar with @jwang in relation to T247417. We can do this on a monthly basis by using the sqooped tables in wmf_raw in the Data Lake. We'll left join mediawiki_imagelinks twice, first with the mediawiki_page table to identify local files, second with mediawiki_page table to identify files used from Commons. If a file isn't found in either of those it should be redlink, and we can mark it as such.

Fri, Oct 16, 10:03 PM · Product-Analytics (Kanban), Structured-Data-Backlog
nettrom_WMF created T265768: Statistics on media usage across Wikipedias.
Fri, Oct 16, 9:57 PM · Product-Analytics (Kanban), Structured-Data-Backlog
nettrom_WMF added a comment to T265101: Instrument event logging for VE's image search.

I agree with @MNeisler that using the VisualEditorFeatureUse schema makes sense since we're asking questions about user behaviour around features in VE specifically.

Fri, Oct 16, 9:02 PM · Product-Infrastructure-Data, Structured-Data-Backlog (Current Work), Editing-team (Tracking), Editing-Team-Request, SDAW-MediaSearch, VisualEditor
nettrom_WMF added a parent task for T259308: Measure usage of image search in Visual Editor: T265761: Update Media Search measurement specification with Visual Editor measurements.
Fri, Oct 16, 8:32 PM · Product-Analytics (Kanban), SDAW-MediaSearch (MediaSearch-Beta), Structured-Data-Backlog
nettrom_WMF added a parent task for T260254: Measure usage of Media Search integration in Visual Editor: T265761: Update Media Search measurement specification with Visual Editor measurements.
Fri, Oct 16, 8:32 PM · Product-Analytics, Structured-Data-Backlog, SDAW-MediaSearch (MediaSearch-ReleaseCandidate)
nettrom_WMF added subtasks for T265761: Update Media Search measurement specification with Visual Editor measurements: T260254: Measure usage of Media Search integration in Visual Editor, T259308: Measure usage of image search in Visual Editor.
Fri, Oct 16, 8:32 PM · SDAW-MediaSearch, Product-Analytics (Kanban), Structured-Data-Backlog
nettrom_WMF created T265761: Update Media Search measurement specification with Visual Editor measurements.
Fri, Oct 16, 8:31 PM · SDAW-MediaSearch, Product-Analytics (Kanban), Structured-Data-Backlog
nettrom_WMF added a comment to T263875: Develop a new schema for MediaSearch analytics or adapt an existing one.

Also, I think storing previous and current state of the filters is a great way to do it! Perhaps particularly if we switch to a map type for storing additional action parameters/values. The only other alternative I was going to suggest was having a combination of value and is_default fields (similar to how PrefUpdate does it), where is_default is true if the value is set back to whatever the default is, and false otherwise. Looking at it again, I think storing the previous and current state is a better option.

Fri, Oct 16, 6:44 PM · Analytics-Radar, Patch-For-Review, SDAW-MediaSearch (MediaSearch-Beta), Product-Analytics, Structured-Data-Backlog (Current Work), Structured Data Engineering
nettrom_WMF added a comment to T263875: Develop a new schema for MediaSearch analytics or adapt an existing one.

@egardner : Thanks for the updates and work so far. Thanks also for your patience while I work on getting feedback to you on this, I met with @mpopov last week and discussed a lot of things around this schema and should've relayed information to you sooner, sorry!

Fri, Oct 16, 6:35 PM · Analytics-Radar, Patch-For-Review, SDAW-MediaSearch (MediaSearch-Beta), Product-Analytics, Structured-Data-Backlog (Current Work), Structured Data Engineering

Tue, Oct 13

nettrom_WMF added a comment to T252391: Reimage one memcached shard to Buster.

Hmm, I spoke too soon. We rely on the wgWMEUnderstandingFirstDay being set in order to oversample in Schema:EditAttemptStep (in WikimediEvents's shouldSchemaEditAttemptStepOversample()), so we need to detangle the configuration value from that method before we can switch off EditorJourney logging. It shouldn't be that complicated -- I think instead of checking to see if wgWMEUnderstandingFirstDay is true, we instead want to see if GrowthExperiments extension is enabled, because we want to oversample edit attempts for all GrowthExperiments users regardless of whether they are opted-in to the Homepage experiment. @nettrom_WMF does that sound right to you?

Tue, Oct 13, 4:51 PM · User-jijiki, Growth-Team (Current Sprint), User-Elukey, Patch-For-Review, Operations, serviceops

Fri, Oct 9

nettrom_WMF added a comment to T250049: Drop data from Prefupdate schema that is older than 90 days.

@Milimetric : It looks like there's no data in event_sanitized.prefupdate for 2020-09-19 through 2020-09-21, and it looks like there's partial data on 2020-09-22. Would it be possible to re-sanitize that date range, or will we need to wait for the re-sanitization script to stop by?

Fri, Oct 9, 10:39 PM · Analytics-Kanban, audits-data-retention, Analytics, Product-Analytics, Privacy Engineering, Privacy, Security
nettrom_WMF moved T255517: Newcomer tasks: reporting notebook after schema changes from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Fri, Oct 9, 8:14 PM · Product-Analytics (Kanban), NewcomerTasks 1.2, Growth-Team (Current Sprint)
nettrom_WMF added a comment to T216668: Welcome survey: investigate Vietnamese abandonment rate.

BTW, I came back to this because of T252391, and noticed that when looking at the two-year registration rate on Vietnamese[1] it looks like the time period where we ran our Welcome Survey A/B test had substantially higher registration rates than expected. If we decide to run another experiment, we should consider fitting a time-series model to the data and use it to predict number of registrations in order to understand if registrations are outside what's expected.

Fri, Oct 9, 6:52 PM · Product-Analytics, Growth-Team, CommRel-Specialists-Support (Jan-Mar-2019)
nettrom_WMF added a comment to T252391: Reimage one memcached shard to Buster.

@kostajh : Thanks for picking this up and pinging me about it. I think we should switch off EditorJourney since we're not actively using the data in any ongoing analysis.

Fri, Oct 9, 6:47 PM · User-jijiki, Growth-Team (Current Sprint), User-Elukey, Patch-For-Review, Operations, serviceops
nettrom_WMF added a comment to T250049: Drop data from Prefupdate schema that is older than 90 days.

@Milimetric : Not a problem, definitely understand that this would be a non-standard request! I've reached out to the PA team and will report back, probably some time on Tuesday.

Fri, Oct 9, 5:52 PM · Analytics-Kanban, audits-data-retention, Analytics, Product-Analytics, Privacy Engineering, Privacy, Security
nettrom_WMF added a comment to T250049: Drop data from Prefupdate schema that is older than 90 days.

@Milimetric : I inspected the sanitized data by looking at the event structs of random partitions and aggregating some random months across various years from 2017 onwards, and in all cases the sanitized data looks correct to me.

Fri, Oct 9, 5:16 PM · Analytics-Kanban, audits-data-retention, Analytics, Product-Analytics, Privacy Engineering, Privacy, Security

Wed, Oct 7

nettrom_WMF moved T262421: [Morten] Review "Schema Migration Audit" document from Doing to Needs Sign-off on the Product-Analytics (Kanban) board.

@mpopov : Thanks for your patience while I work on juggling tasks and finding time to come back to this. I've discussed the schemas with the SD team and we found that the MultimediaViewer and UploadWizard schemas could be marked for deprecation. As I didn't have edit permission of the googledoc, I left a couple of comments to that effect. I think this concludes everything, handing it to you for sign-off!

Wed, Oct 7, 7:02 PM · Product-Analytics (Kanban)

Tue, Oct 6

nettrom_WMF added a comment to T263875: Develop a new schema for MediaSearch analytics or adapt an existing one.

If there is a better/standard way to capture some of these things I'm happy to re-work the schema (but specific guidance would be helpful).

Tue, Oct 6, 10:08 PM · Analytics-Radar, Patch-For-Review, SDAW-MediaSearch (MediaSearch-Beta), Product-Analytics, Structured-Data-Backlog (Current Work), Structured Data Engineering
nettrom_WMF moved T230174: Newcomer tasks: experiment analysis from Needs Review to Needs Sign-off on the Product-Analytics (Kanban) board.
Tue, Oct 6, 5:07 PM · Product-Analytics (Kanban), Growth-Team (Current Sprint), NewcomerTasks 1.0 , GrowthExperiments-Homepage
nettrom_WMF moved T259308: Measure usage of image search in Visual Editor from Needs Review to Blocked on the Product-Analytics (Kanban) board.
Tue, Oct 6, 5:07 PM · Product-Analytics (Kanban), SDAW-MediaSearch (MediaSearch-Beta), Structured-Data-Backlog
nettrom_WMF moved T261759: Analyze Media Search A/B test from Needs Review to Needs Sign-off on the Product-Analytics (Kanban) board.
Tue, Oct 6, 5:06 PM · Product-Analytics (Kanban), Structured-Data-Backlog, SDAW-MediaSearch (MediaSearch-Alpha)

Mon, Oct 5

nettrom_WMF moved T259308: Measure usage of image search in Visual Editor from Next 2 weeks to Needs Review on the Product-Analytics (Kanban) board.

I've dug into this a bit to get an understanding of what data is available through the VisualEditorFeatureUse schema. I also met with @MNeisler on the Product Analytics team to get a check on whether my understanding of the data was correct, and it appears to be.

Mon, Oct 5, 9:26 PM · Product-Analytics (Kanban), SDAW-MediaSearch (MediaSearch-Beta), Structured-Data-Backlog
nettrom_WMF edited projects for T259308: Measure usage of image search in Visual Editor, added: Product-Analytics (Kanban); removed Product-Analytics.
Mon, Oct 5, 8:53 PM · Product-Analytics (Kanban), SDAW-MediaSearch (MediaSearch-Beta), Structured-Data-Backlog
nettrom_WMF added a comment to T262271: Activate mediasearch profile without requiring an explicit flag.

However I think we need to make sure VE is properly instrumented and get some baselines from T259308 before we make the switch. @nettrom_WMF, do you have any sense of a timeline on that task?

Mon, Oct 5, 6:07 PM · SDAW-MediaSearch (MediaSearch-ReleaseCandidate), Patch-For-Review, Structured-Data-Backlog (Current Work)

Thu, Oct 1

nettrom_WMF added a comment to T255028: Move the stat1004-6-7 hosts to Debian Buster.

With these new upgrades happening, I wanted to move my Jupyter notebooks from stat1008 to stat1006 as stat1008 has been very busy lately. After rsync'ing my files, I started reinstalling my R libraries and had them error out because one of them wasn't available for R v3.3. That surprised me, because Debian Buster ships with R v3.5 (as can be found on stat1005 and stat1008).

Thu, Oct 1, 9:29 PM · Analytics-Kanban, Analytics-Clusters
nettrom_WMF added a comment to T263875: Develop a new schema for MediaSearch analytics or adapt an existing one.

This is awesome work so far! I've read through this task, its parent task, and the proposed patch and updated the measurement specification to reflect the set of questions mentioned by @CBogen in T263875#6495409. From what I can tell, the proposed schema allows us to answer our current set of questions.

Thu, Oct 1, 4:18 PM · Analytics-Radar, Patch-For-Review, SDAW-MediaSearch (MediaSearch-Beta), Product-Analytics, Structured-Data-Backlog (Current Work), Structured Data Engineering

Tue, Sep 29

nettrom_WMF added a comment to T262421: [Morten] Review "Schema Migration Audit" document.

@mpopov : Ah, feel free to reopen this if you want me to ping the SD team and have them come back to me with a list of schemas.

Tue, Sep 29, 9:54 PM · Product-Analytics (Kanban)
nettrom_WMF moved T261759: Analyze Media Search A/B test from Doing to Needs Review on the Product-Analytics (Kanban) board.

A huge thanks to @mpopov for doing a lot of work on this, improving the data processing code and figuring out ways massage the data from SearchSatisfaction to pull out the insights!

Tue, Sep 29, 9:32 PM · Product-Analytics (Kanban), Structured-Data-Backlog, SDAW-MediaSearch (MediaSearch-Alpha)
nettrom_WMF closed T262421: [Morten] Review "Schema Migration Audit" document, a subtask of T261794: [REQUEST] Event Schema Audit Review, as Resolved.
Tue, Sep 29, 8:03 PM · Product-Analytics (Kanban), Better Use Of Data, Product-Infrastructure-Data
nettrom_WMF closed T262421: [Morten] Review "Schema Migration Audit" document as Resolved.

I've gone through the spreadsheet and added information for all known Growth-related schemas. Looks like the Multimedia team already went through and marked theirs as well. Don't think this needs any peer review, so closing it as resolved.

Tue, Sep 29, 8:03 PM · Product-Analytics (Kanban)

Thu, Sep 24

nettrom_WMF moved T261759: Analyze Media Search A/B test from Needs Review to Doing on the Product-Analytics (Kanban) board.

We're unsure if the finding is trustworthy. I'm moving this back to "Doing" to dig further into this.

Thu, Sep 24, 4:11 PM · Product-Analytics (Kanban), Structured-Data-Backlog, SDAW-MediaSearch (MediaSearch-Alpha)

Wed, Sep 23

nettrom_WMF moved T262421: [Morten] Review "Schema Migration Audit" document from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Wed, Sep 23, 11:12 PM · Product-Analytics (Kanban)
nettrom_WMF moved T261759: Analyze Media Search A/B test from Doing to Needs Review on the Product-Analytics (Kanban) board.

The analysis has been done and can be found in this Jupyter/R notebook. We find a slight preference for the control condition (legacy search) over Media Search.

Wed, Sep 23, 10:48 PM · Product-Analytics (Kanban), Structured-Data-Backlog, SDAW-MediaSearch (MediaSearch-Alpha)
nettrom_WMF added a comment to T255722: Check and update reporting related to "automated" traffic change for product teams.

@cchen : Thanks for the ping, I've confirmed none of the teams I work with use pageview data in reports and updated the task description to reflect this.

Wed, Sep 23, 10:38 PM · Product-Analytics (Kanban)
nettrom_WMF updated the task description for T255722: Check and update reporting related to "automated" traffic change for product teams.
Wed, Sep 23, 10:36 PM · Product-Analytics (Kanban)
nettrom_WMF closed T258723: Baseline metrics of legacy search on Commons as Resolved.

Closing this as resolved, but feel free to reopen this if I misinterpreted something and there are still unanswered questions.

Wed, Sep 23, 8:56 PM · Product-Analytics (Kanban), SDAW-MediaSearch (MediaSearch-Alpha), Structured-Data-Backlog
nettrom_WMF closed T258723: Baseline metrics of legacy search on Commons, a subtask of T258229: Build dashboards for search activity on MediaSearch on Commons, as Resolved.
Wed, Sep 23, 8:56 PM · SDAW-MediaSearch (MediaSearch-Beta), Product-Analytics, Structured-Data-Backlog
nettrom_WMF added a comment to T263603: Pageview and editing stats for Fundraising Report.

Commenting here because I'd be curious to know if we have other sources we'd use for this: For #1, using the edit counts for all Wikipedias on wikistats[1], it looks like we're pretty consistently in the 15–16M range now (I see March–May as COVID-related outliers). So I'd suggest August 2020 is a reasonable estimate for activity. With 15,927,411 edits in 31 days in August, and assuming the normal 86400 seconds/day, we get 5.95 edits per second to all Wikipedias.

Wed, Sep 23, 5:28 PM · Product-Analytics (Kanban)

Mon, Sep 21

nettrom_WMF moved T261759: Analyze Media Search A/B test from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Mon, Sep 21, 5:14 PM · Product-Analytics (Kanban), Structured-Data-Backlog, SDAW-MediaSearch (MediaSearch-Alpha)
nettrom_WMF closed T255501: Newcomer tasks: update schema whitelist for Guidance as Resolved.

I've verified that this change has been deployed. The NewcomerTask schema is available in a sanitized version, and the changes to the HelpPanel schema are also in the sanitized data for that schema. I've not verified that the tokens are no longer hashed, but we can do that when we update the reporting notebook.

Mon, Sep 21, 4:36 PM · Analytics-Radar, Growth-Team (Current Sprint), Product-Analytics (Kanban)
nettrom_WMF closed T255501: Newcomer tasks: update schema whitelist for Guidance, a subtask of T246919: Newcomer tasks: guidance instrumentation, as Resolved.
Mon, Sep 21, 4:36 PM · MW-1.35-notes (1.35.0-wmf.36; 2020-06-09), NewcomerTasks 1.2, Growth-Team (Current Sprint)

Sep 14 2020

nettrom_WMF added a comment to T262831: [REQUEST] Sample of Active Users for Community Insights Survey: Second sample pull.

Hi @Rmaung! We'll review and triage this request at our next board refinement meeting on Tuesday, September 15.

Sep 14 2020, 5:29 PM · Product-Analytics (Kanban)
nettrom_WMF changed Due Date from Jun 26 2020, 7:00 PM to Fri, Sep 25, 7:00 PM on T230174: Newcomer tasks: experiment analysis.
Sep 14 2020, 4:26 PM · Product-Analytics (Kanban), Growth-Team (Current Sprint), NewcomerTasks 1.0 , GrowthExperiments-Homepage

Sep 10 2020

nettrom_WMF moved T258419: Survey users about mediasearch on commons from Triage to Tracking on the Product-Analytics board.
Sep 10 2020, 8:19 PM · Product-Analytics, Patch-For-Review, SDAW-MediaSearch (MediaSearch-Beta), Surveys, Structured-Data-Backlog (Current Work), Structured Data Engineering
nettrom_WMF added a project to T258419: Survey users about mediasearch on commons: Product-Analytics.
Sep 10 2020, 8:19 PM · Product-Analytics, Patch-For-Review, SDAW-MediaSearch (MediaSearch-Beta), Surveys, Structured-Data-Backlog (Current Work), Structured Data Engineering

Sep 9 2020

nettrom_WMF moved T261759: Analyze Media Search A/B test from Doing to Next 2 weeks on the Product-Analytics (Kanban) board.

Moving this out of "Doing" as we've discovered that the data gathering had a bug leading to us being unable to determine which algorithm produced a clicked result when interleaving occurred. Will pick up the analysis again once the second iteration of the test has been completed. And yes, we'll QA the data after relaunch to make sure it's working correctly.

Sep 9 2020, 10:07 PM · Product-Analytics (Kanban), Structured-Data-Backlog, SDAW-MediaSearch (MediaSearch-Alpha)

Sep 8 2020

nettrom_WMF moved T261759: Analyze Media Search A/B test from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Sep 8 2020, 8:54 PM · Product-Analytics (Kanban), Structured-Data-Backlog, SDAW-MediaSearch (MediaSearch-Alpha)
nettrom_WMF added a comment to T250049: Drop data from Prefupdate schema that is older than 90 days.

@Milimetric : I've gone through the various subtasks and changes we made before the tracking list was implemented and not found any cause for concern. We've got the properties we want to track listed and that covers all current analysis needs. So applying the list to old data should be fine.

Sep 8 2020, 4:55 PM · Analytics-Kanban, audits-data-retention, Analytics, Product-Analytics, Privacy Engineering, Privacy, Security

Aug 25 2020

mpopov awarded T261165: wmfdata-r cannot query MariaDB a Stroopwafel token.
Aug 25 2020, 1:48 PM · Product-Analytics (Kanban)

Aug 24 2020

nettrom_WMF added a comment to T258723: Baseline metrics of legacy search on Commons.

@CBogen : the number of full-text searches does not include any autocomplete searches. It does include full-text searches that originate from an autocomplete search (e.g. the user clicks on the "contains …" part, or hits enter) because identifying those to separate them out is tricky.

Aug 24 2020, 10:53 PM · Product-Analytics (Kanban), SDAW-MediaSearch (MediaSearch-Alpha), Structured-Data-Backlog
nettrom_WMF moved T261165: wmfdata-r cannot query MariaDB from Next 2 weeks to Needs Review on the Product-Analytics (Kanban) board.

Patch uploaded, moving to the review column for @mpopov to review.

Aug 24 2020, 9:22 PM · Product-Analytics (Kanban)
nettrom_WMF edited projects for T261165: wmfdata-r cannot query MariaDB, added: Product-Analytics (Kanban); removed Product-Analytics.
Aug 24 2020, 9:21 PM · Product-Analytics (Kanban)
nettrom_WMF created T261165: wmfdata-r cannot query MariaDB.
Aug 24 2020, 9:13 PM · Product-Analytics (Kanban)

Aug 21 2020

nettrom_WMF added a comment to T260867: PrefUpdate captures user preference modifications at registration.

One question came up when discussing this task in the Growth team: are we seeing this pattern for user preferences from other extensions besides GrowthExperiments?

Aug 21 2020, 5:48 PM · MW-1.36-notes (1.36.0-wmf.12; 2020-10-05; NEVER DEPLOYED), Product-Infrastructure-Data, Analytics-Radar, Product-Analytics, Growth-Team

Aug 20 2020

nettrom_WMF created T260867: PrefUpdate captures user preference modifications at registration.
Aug 20 2020, 3:23 AM · MW-1.36-notes (1.36.0-wmf.12; 2020-10-05; NEVER DEPLOYED), Product-Infrastructure-Data, Analytics-Radar, Product-Analytics, Growth-Team

Aug 18 2020

nettrom_WMF moved T252829: Add a link: background data analyses from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Aug 18 2020, 9:08 PM · Product-Analytics (Kanban), Growth-Team (Current Sprint), Growth-Structured-Tasks
nettrom_WMF awarded T260706: Update/repair Search A/B Test autoreporter a Like token.
Aug 18 2020, 9:02 PM · Structured-Data-Backlog, Discovery-Search, Discovery-Analysis, Product-Analytics
nettrom_WMF added a comment to T260706: Update/repair Search A/B Test autoreporter.

Adding my support to having this tool available! As I've been working with the Structured Data team to determine what metrics we want to use to measure the impact of upcoming tests, it's become more and more clear to me that what we're doing is generally what the Discovery team were doing a couple of years ago. The hewiki report created by the tool that Mikhail linked in the descriptions contains a lot of the metrics we've been discussing with the SD team (as well as a lot of additional ones). The tool also analyzes interleaved A/B tests, something the team are planning on doing. Having all of that readily available to enable iterating on experiments with streamlined analysis would take a lot of work out of it!

Aug 18 2020, 9:01 PM · Structured-Data-Backlog, Discovery-Search, Discovery-Analysis, Product-Analytics

Aug 17 2020

nettrom_WMF moved T258723: Baseline metrics of legacy search on Commons from Doing to Needs Review on the Product-Analytics (Kanban) board.

A first pass on calculating these baselines has now been done. The numbers and the calculations can be found in this Jupyter notebook. It uses the past 7 days as the source of the data and was run on 2020-08-16, meaning it reflects the week from 2020-08-09 through 2020-08-15. It does make some assumptions and shortcuts, and I'm happy to discuss those and modify the code as we see fit.

Aug 17 2020, 10:05 PM · Product-Analytics (Kanban), SDAW-MediaSearch (MediaSearch-Alpha), Structured-Data-Backlog
nettrom_WMF closed T260097: Data request for Legal as Resolved.
Aug 17 2020, 8:37 PM · Product-Analytics (Kanban)
nettrom_WMF added a project to T259768: Scale: analyze Russian welcome survey: Product-Analytics.
Aug 17 2020, 4:27 PM · Product-Analytics, Russian-Sites, Growth-Team
nettrom_WMF closed T238319: Deploy Help Panel to Ukrainian, Hungarian, Armenian Wikipedias, a subtask of T242897: Deployments to Ukrainian, Hungarian, Armenian Wikipedias, as Resolved.
Aug 17 2020, 4:21 PM · CommRel-Specialists-Support (Jan-Mar-2020), Growth-Team
nettrom_WMF closed T238319: Deploy Help Panel to Ukrainian, Hungarian, Armenian Wikipedias as Resolved.

We've not seen any indication that there are data quality issues after our deployment, and we do not have the capacity to prioritize this work to dig in further. Closing this task as resolved since the Help Panel has been deployed.

Aug 17 2020, 4:21 PM · Growth-Team (Current Sprint), Product-Analytics, GrowthExperiments
nettrom_WMF closed T238320: Deploy Newcomer Homepage to Ukrainian, Hungarian, Armenian Wikipedias, a subtask of T242897: Deployments to Ukrainian, Hungarian, Armenian Wikipedias, as Resolved.
Aug 17 2020, 4:21 PM · CommRel-Specialists-Support (Jan-Mar-2020), Growth-Team
nettrom_WMF closed T238320: Deploy Newcomer Homepage to Ukrainian, Hungarian, Armenian Wikipedias as Resolved.

We've not seen any indication that there are data quality issues after our deployment, and we do not have the capacity to prioritize this work to dig in further. Closing this task as resolved since the Homepage has been deployed.

Aug 17 2020, 4:21 PM · Growth-Team (Current Sprint), Product-Analytics, GrowthExperiments
nettrom_WMF closed T233066: Deploy Newcomer Homepage to Basque Wikipedia, a subtask of T232060: Deploy Growth features to Basque Wikipedia, as Resolved.
Aug 17 2020, 4:19 PM · GrowthExperiments, Growth-Team (Current Sprint)
nettrom_WMF closed T233066: Deploy Newcomer Homepage to Basque Wikipedia as Resolved.

We've not seen any indication that there are data quality issues after our deployment on Basque Wikipedia, and we do not have the capacity to prioritize this work to dig in further. Closing this task as resolved since the Homepage has been deployed.

Aug 17 2020, 4:19 PM · Product-Analytics, GrowthExperiments, Growth-Team (Current Sprint)
nettrom_WMF closed T233065: Deploy Help Panel to Basque Wikipedia as Resolved.

We've not seen any indication that there are data quality issues after our deployment on Basque Wikipedia, and we do not have the capacity to prioritize this work to dig in further. Closing this task as resolved since the Help Panel has been deployed.

Aug 17 2020, 4:19 PM · Product-Analytics, GrowthExperiments, Growth-Team (Current Sprint)
nettrom_WMF closed T233065: Deploy Help Panel to Basque Wikipedia, a subtask of T232060: Deploy Growth features to Basque Wikipedia, as Resolved.
Aug 17 2020, 4:19 PM · GrowthExperiments, Growth-Team (Current Sprint)

Aug 13 2020

nettrom_WMF added a comment to T259748: Review identifiers schema fragment (Morten).

Yeah, I think the descriptions and clarifications on the wikitech page are great, nice work!

Aug 13 2020, 6:18 PM · Product-Analytics (Kanban)

Aug 12 2020

nettrom_WMF moved T259748: Review identifiers schema fragment (Morten) from Doing to Needs Review on the Product-Analytics (Kanban) board.

As mentioned, I also left a comment in Gerrit. I see that the descriptions on Wikitech provide more information than the ones in the repository, which makes sense to me. Moving this to the review column.

Aug 12 2020, 10:12 PM · Product-Analytics (Kanban)
nettrom_WMF moved T260097: Data request for Legal from Doing to Needs Review on the Product-Analytics (Kanban) board.

Initial analysis done, awaiting review and possibly follow-up questions from Legal.

Aug 12 2020, 9:48 PM · Product-Analytics (Kanban)
nettrom_WMF added a comment to T259748: Review identifiers schema fragment (Morten).

Reviewing the descriptions: I made this edit as I think fields are supposed to be required, not events.

Aug 12 2020, 5:30 PM · Product-Analytics (Kanban)

Aug 10 2020

nettrom_WMF moved T249666: Growth: validate that data is purged after 270 days from Next 2 weeks to Needs Review on the Product-Analytics (Kanban) board.
Aug 10 2020, 6:22 PM · Growth-Team (Current Sprint), Product-Analytics (Kanban), Analytics
nettrom_WMF moved T249666: Growth: validate that data is purged after 270 days from Incoming to Needs PM Review on the Growth-Team (Current Sprint) board.

I ran SHOW PARTITIONS event_sanitized.homepagemodule on 2020-08-07 and again today (2020-08-10). After the first run, I compared the available partitions to the date specified in T244312 (2019-11-05) and noticed that the first available partition on 2020-08-07 was 2019-11-11, indicating that data was starting to be purged.

Aug 10 2020, 6:22 PM · Growth-Team (Current Sprint), Product-Analytics (Kanban), Analytics
nettrom_WMF edited projects for T249666: Growth: validate that data is purged after 270 days, added: Growth-Team (Current Sprint); removed Growth-Team.
Aug 10 2020, 6:14 PM · Growth-Team (Current Sprint), Product-Analytics (Kanban), Analytics
nettrom_WMF edited projects for T249666: Growth: validate that data is purged after 270 days, added: Product-Analytics (Kanban); removed Product-Analytics.
Aug 10 2020, 6:14 PM · Growth-Team (Current Sprint), Product-Analytics (Kanban), Analytics
nettrom_WMF moved T230174: Newcomer tasks: experiment analysis from Doing to Needs Review on the Product-Analytics (Kanban) board.
Aug 10 2020, 5:02 PM · Product-Analytics (Kanban), Growth-Team (Current Sprint), NewcomerTasks 1.0 , GrowthExperiments-Homepage
nettrom_WMF added a comment to T259944: NULL-values for useragent column in event.searchsatisfaction.

Thanks for taking care of this so quickly, @Ottomata, very much appreciated!

Aug 10 2020, 4:54 PM · Analytics-Kanban, Product-Analytics, Analytics

Aug 7 2020

nettrom_WMF created T259944: NULL-values for useragent column in event.searchsatisfaction.
Aug 7 2020, 10:52 PM · Analytics-Kanban, Product-Analytics, Analytics
nettrom_WMF added a comment to T259913: Welcome survey: remove question about mentor program.

I don't have any objections to removing the question. As @RHo points out, we have the mentor module. Users in the control group have it too, if they turn the Homepage on. We've also analyzed the answers to this question a few times and gotten a good sense of how popular that choice is, so I'm not sure we can learn that much more.

Aug 7 2020, 8:17 PM · MW-1.36-notes (1.36.0-wmf.5; 2020-08-18), Growth-Team (Current Sprint), GrowthExperiments
nettrom_WMF added a comment to T259860: [mobile] HomepageModule does not record link-click events for Impact module .

EventLogging on the beta cluster appears to be broken enough that it's near impossible to verify events there, unfortunately. From what I could tell by checking the JS console in my browser, link-click events on desktop trigger an event, but I didn't see any of those on mobile.

Aug 7 2020, 4:44 PM · MW-1.36-notes (1.36.0-wmf.5; 2020-08-18), Growth-Team (Current Sprint), GrowthExperiments-Homepage

Aug 4 2020

nettrom_WMF moved T258229: Build dashboards for search activity on MediaSearch on Commons from Needs Investigation to Current Quarter on the Product-Analytics board.
Aug 4 2020, 5:20 PM · SDAW-MediaSearch (MediaSearch-Beta), Product-Analytics, Structured-Data-Backlog
nettrom_WMF moved T258723: Baseline metrics of legacy search on Commons from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Aug 4 2020, 5:06 PM · Product-Analytics (Kanban), SDAW-MediaSearch (MediaSearch-Alpha), Structured-Data-Backlog