mpopov (Mikhail Popov)
Data Analyst

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Saturday

  • Clear sailing ahead.

User Details

User Since
Jul 27 2015, 4:15 PM (86 w, 2 d)
Availability
Available
IRC Nick
bearloga
LDAP User
Bearloga
MediaWiki User
MPopov (WMF)

Data Scientist at Discovery | Highlighted Works

Recent Activity

Today

mpopov moved T160624: Search dashboard, suspicious fulltext api usage from In progress to Done on the Discovery-Analysis (Current work) board.

Everything is counted correctly by this script which uses the patched search request definition. Unfortunately we were used the pre-patch version until 2017-01-01 that undercounted API calls because it was not counting the 100M+ generator calls. I'm sorry that I didn't switch to using the latest refinery JARs sooner.

Thu, Mar 23, 3:06 AM · Discovery-Analysis (Current work), Discovery
mpopov claimed T160624: Search dashboard, suspicious fulltext api usage.

Taking a look…

Thu, Mar 23, 1:10 AM · Discovery-Analysis (Current work), Discovery

Tue, Mar 21

mpopov added a comment to T156300: analysis of results from A/B/C test for displaying sister projects in search results.

Revision uploaded. Thanks again, @ksmith!

Tue, Mar 21, 9:43 PM · Discovery-Analysis (Current work), Discovery

Mon, Mar 20

mpopov added a comment to T145445: Unit testing (& CI) for dashboard data retrieval scripts.

Note for future self: assertr 2.0 just released on CRAN. From http://www.onthelambda.com/2017/03/20/data-validation-with-the-assertr-package/:

Mon, Mar 20, 5:55 PM · Discovery-Analysis

Fri, Mar 17

mpopov closed T160772: Add self-config checking & correcting to golden as "Resolved".
Fri, Mar 17, 6:36 PM · Patch-For-Review, Discovery-Analysis
mpopov created T160772: Add self-config checking & correcting to golden.
Fri, Mar 17, 5:37 PM · Patch-For-Review, Discovery-Analysis
mpopov added a comment to T156300: analysis of results from A/B/C test for displaying sister projects in search results.

@ksmith I'll see if I can address some/all of those in a revision. Thanks!

Fri, Mar 17, 3:49 PM · Discovery-Analysis (Current work), Discovery

Thu, Mar 16

mpopov removed a project from T112170: Model user behavior and detect when reality heavily deviated from expectation: Patch-For-Review.
Thu, Mar 16, 8:05 PM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
mpopov moved T112170: Model user behavior and detect when reality heavily deviated from expectation from Backlog to In progress on the Discovery-Analysis (Current work) board.
Thu, Mar 16, 8:05 PM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
mpopov moved T156300: analysis of results from A/B/C test for displaying sister projects in search results from Needs review to Done on the Discovery-Analysis (Current work) board.

Updated the report following @debt's feedback. PDF version up on Commons: https://commons.wikimedia.org/wiki/File:A_Test_Of_Cross-wiki_Search_-_Helping_Users_Discover_Content_On_Wikipedia%E2%80%99s_Sister_Projects.pdf

Thu, Mar 16, 5:51 PM · Discovery-Analysis (Current work), Discovery

Wed, Mar 15

mpopov moved T156300: analysis of results from A/B/C test for displaying sister projects in search results from In progress to Needs review on the Discovery-Analysis (Current work) board.
Wed, Mar 15, 11:29 PM · Discovery-Analysis (Current work), Discovery
mpopov added a comment to T156300: analysis of results from A/B/C test for displaying sister projects in search results.

@debt, @TJones, @chelsyx, @ksmith thank you all for your feedback! Hopefully I've addressed it all in the second draft (web: https://wikimedia-research.github.io/Discovery-Search-Test-CrosswikiSidebar/, PDF: https://github.com/wikimedia-research/Discovery-Search-Test-CrosswikiSidebar/blob/master/docs/index.pdf) which now includes additional analysis per Kevin's curiosity :)

Wed, Mar 15, 11:26 PM · Discovery-Analysis (Current work), Discovery

Tue, Mar 14

mpopov added a comment to T160004: [A/B Test] Add sister project search results in a right sidebar (test #2).

For a one week period (Mar 5, 2017 00:00:00 to Mar 12, 2017 00:00:00) we collected the following number of fulltext search sessions at the default 1:200 sampling:

wikisessions
arwiki590
dewiki4776
frwiki2165
ruwiki2227

@mpopov Ideally how many sessions would we like to record per bucket for this second test? From there i can figure out what the sampling should be set at.

Tue, Mar 14, 8:38 PM · MW-1.29-release (WMF-deploy-2017-03-14_(1.29.0-wmf.16)), Patch-For-Review, Discovery-Search (Current work), Discovery
mpopov edited the description of T160463: Update event logging for sister project search results.
Tue, Mar 14, 8:23 PM · MW-1.29-release (WMF-deploy-2017-03-14_(1.29.0-wmf.16)), Patch-For-Review, Discovery-Search (Current work), Discovery
mpopov added a comment to T153856: Add CI to all wikimedia/discovery repositories that are active.

Linking to the lintr package here for future reference: https://github.com/jimhester/lintr has instructions for configuring linters and adding it to Travis CI, which we could adapt to Jenkins when we get to it :)

Tue, Mar 14, 5:53 PM · Discovery-Analysis, Discovery, Release-Engineering-Team, Continuous-Integration-Config
mpopov closed T160211: Typo in position data for multimedia iw-result as "Resolved".

Jan fixed it in https://gerrit.wikimedia.org/r/#/c/342200/1/resources/src/mediawiki.special/mediawiki.special.search.commonsInterwikiWidget.js@31 :)

Tue, Mar 14, 5:47 PM · Discovery-Search (Current work), Discovery

Fri, Mar 10

mpopov added a comment to T160211: Typo in position data for multimedia iw-result.

whoops - one too many 's''s...

@mpopov - I'm not sure we can fix this before the next A/B test starts...can we update how we look for this data or is it simply not collected at all because of the typo?

Fri, Mar 10, 11:24 PM · Discovery-Search (Current work), Discovery
mpopov created T160211: Typo in position data for multimedia iw-result.
Fri, Mar 10, 10:45 PM · Discovery-Search (Current work), Discovery

Thu, Mar 9

mpopov moved T112170: Model user behavior and detect when reality heavily deviated from expectation from Stalled/Waiting to Backlog on the Discovery-Analysis (Current work) board.
Thu, Mar 9, 9:03 PM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
mpopov moved T156300: analysis of results from A/B/C test for displaying sister projects in search results from Needs review to In progress on the Discovery-Analysis (Current work) board.
Thu, Mar 9, 9:03 PM · Discovery-Analysis (Current work), Discovery
mpopov moved T154722: External traffic dashboard - update with notation for Safari internal referrers from Needs review to Done on the Discovery-Analysis (Current work) board.

Deployed to production: http://discovery.wmflabs.org/external/#traffic_summary

Thu, Mar 9, 12:15 AM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
mpopov moved T152617: Wikipedia portal: update eventlogging for sister project clickthroughs from Needs review to Done on the Discovery-Analysis (Current work) board.
Thu, Mar 9, 12:14 AM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
mpopov added a comment to T152617: Wikipedia portal: update eventlogging for sister project clickthroughs.

Deployed to production: http://discovery.wmflabs.org/portal/#sister_projects

Thu, Mar 9, 12:14 AM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
mpopov moved T154634: Wikipedia.org: add app links to Portal dashboard from Needs review to Done on the Discovery-Analysis (Current work) board.

Deployed to production: http://discovery.wmflabs.org/portal/#app_links

Thu, Mar 9, 12:13 AM · Patch-For-Review, Wikimedia-Portals, Discovery-Analysis (Current work), Discovery
mpopov moved T153936: Create usage stats for WDQS LDF endpoint from Needs review to Done on the Discovery-Analysis (Current work) board.

Deployed to production: http://discovery.wmflabs.org/wdqs/#endpoint_usage

Thu, Mar 9, 12:11 AM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
mpopov moved T150915: [Dashboard] Migrate golden to Reportupdater infrastructure from Needs review to Done on the Discovery-Analysis (Current work) board.

stat1002:/a/discovery/golden has been updated

Thu, Mar 9, 12:03 AM · Patch-For-Review, Discovery-Analysis (Current work), Discovery

Wed, Mar 8

mpopov edited the description of T150915: [Dashboard] Migrate golden to Reportupdater infrastructure.
Wed, Mar 8, 11:49 PM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
mpopov moved T156300: analysis of results from A/B/C test for displaying sister projects in search results from In progress to Needs review on the Discovery-Analysis (Current work) board.

First draft:

Wed, Mar 8, 1:50 AM · Discovery-Analysis (Current work), Discovery

Wed, Mar 1

mpopov added a comment to T153856: Add CI to all wikimedia/discovery repositories that are active.

@Aklapper @debt: I'm OK with re-opening this ticket if @greg or someone can help us set it up. If we can get ortiz and wmf (which already have unit tests) to do CI checks on Gerrit, Chelsy and I can look into the other ones like polloi :)

Wed, Mar 1, 4:51 PM · Discovery-Analysis, Discovery, Release-Engineering-Team, Continuous-Integration-Config

Tue, Feb 28

mpopov moved T159169: Add Chelsy and Mikhail as admins to Discovery's Analysis repos on Diffusion from Current work to Tracking on the Discovery-Analysis board.
Tue, Feb 28, 9:08 PM · Project-Admins, Discovery-Analysis, Repository-Admins
mpopov moved T159169: Add Chelsy and Mikhail as admins to Discovery's Analysis repos on Diffusion from Tracking to Current work on the Discovery-Analysis board.
Tue, Feb 28, 9:08 PM · Project-Admins, Discovery-Analysis, Repository-Admins
mpopov moved T143197: Add a citolytics query prefix that exposes the relatedness information from the elasticsearch index from Needs triage to Tracking on the Discovery-Analysis board.
Tue, Feb 28, 9:06 PM · Discovery-Analysis, Discovery-Search, CirrusSearch, Discovery, Goal

Mon, Feb 27

mpopov edited the description of T156523: Investigate adding user-friendly testing functionality to Reportupdater.
Mon, Feb 27, 7:30 PM · Analytics
mpopov created T159169: Add Chelsy and Mikhail as admins to Discovery's Analysis repos on Diffusion.
Mon, Feb 27, 7:23 PM · Project-Admins, Discovery-Analysis, Repository-Admins
mpopov renamed T139487: Get 'sparklyr' working on stats1002 from "10%: Get 'sparklyr' working" to "Get 'sparklyr' working on stats1002".
Mon, Feb 27, 6:48 PM · Analytics, Discovery-Analysis
mpopov closed T112289: Update wmf::mysql_read() as "Resolved".

This has been done:

Mon, Feb 27, 6:42 PM · Discovery-Analysis (Current work), Discovery

Tue, Feb 21

mpopov moved T157796: Visits/searches from Safari 10 location bar search suggestions from In progress to Stalled/Waiting on the Discovery-Analysis (Current work) board.

Getting mixed answers from my slow approach of using feature + digging for HTTP requests. Namely, Spotlight Suggestions within Safari web search seem to use Related Page + More Like API (thanks to @dcausse for his help), but Spotlight Suggestions from Spotlight search on Mac don't seem to make API calls. I just reached out to someone I know who works at Apple to possibly get me in touch with someone there who worked/works on that feature/team so that I could ask them about it directly. This ticket is paused until I hear back about that.

Tue, Feb 21, 8:02 PM · Discovery-Analysis (Current work), Browser-Support-Apple-Safari

Feb 16 2017

mpopov claimed T157796: Visits/searches from Safari 10 location bar search suggestions.

Hm... seeing something that might indicate they're using our API :) looking deeper into it…

Feb 16 2017, 5:25 PM · Discovery-Analysis (Current work), Browser-Support-Apple-Safari
mpopov claimed T156300: analysis of results from A/B/C test for displaying sister projects in search results.
Feb 16 2017, 12:01 AM · Discovery-Analysis (Current work), Discovery

Feb 15 2017

mpopov moved T156512: Check that data coming from cross-wiki test is valid from In progress to Needs review on the Discovery-Analysis (Current work) board.

Much better (read as "easier to learn something from") numbers post-patch ^_^

Feb 15 2017, 9:11 PM · MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, Discovery-Analysis (Current work), CirrusSearch, Discovery
mpopov added a comment to T157796: Visits/searches from Safari 10 location bar search suggestions.

We actually discussed this last year and I remember someone saying that Apple keeps a cached copy of Wikipedia (not sure if just English or if there are other languages) and that's what they use for Spotlight Suggestions in Safari. Which actually makes a lot of sense seeing as otherwise they would have to forward the user's search query to us and that would surely violate their privacy policy (as opposed to when the user explicitly makes a Google search from Safari). We can take a look but I doubt they're using our API. Last I heard even Amazon Kindle doesn't -_-

Feb 15 2017, 12:43 AM · Discovery-Analysis (Current work), Browser-Support-Apple-Safari
mpopov added a comment to T154722: External traffic dashboard - update with notation for Safari internal referrers.

I'm not actually seeing a problem. If you use Pivot (URL) to look at Safari & Mobile Safari pageviews and break it down by referrer class, we're not seeing a mis-categorization:

Feb 15 2017, 12:29 AM · Patch-For-Review, Discovery-Analysis (Current work), Discovery

Feb 14 2017

mpopov added a comment to T156512: Check that data coming from cross-wiki test is valid.

@EBernhardson: I have a single 'ssclick' event right now and it has a NULL event_position so I can't say there's definitively a problem but I do want to make this note for myself to check later tomorrow after the sampling rate patch deploys.

Feb 14 2017, 1:45 AM · MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, Discovery-Analysis (Current work), CirrusSearch, Discovery
mpopov added a comment to T156512: Check that data coming from cross-wiki test is valid.

@EBernhardson: could you please up the sampling rate to 1 in 50 across all 4? Here are the current numbers with the 1 in 200 rate:

Feb 14 2017, 12:52 AM · MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, Discovery-Analysis (Current work), CirrusSearch, Discovery

Feb 13 2017

mpopov moved T156512: Check that data coming from cross-wiki test is valid from Backlog to In progress on the Discovery-Analysis (Current work) board.
Feb 13 2017, 6:44 PM · MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, Discovery-Analysis (Current work), CirrusSearch, Discovery
mpopov claimed T156512: Check that data coming from cross-wiki test is valid.
Feb 13 2017, 6:44 PM · MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, Discovery-Analysis (Current work), CirrusSearch, Discovery

Feb 8 2017

mpopov added a comment to T154634: Wikipedia.org: add app links to Portal dashboard.

@debt: the dark axes is a known bug that we can't reproduce / don't know what causes it :\ if you click on a different dashboard and then go back to the sister projects one, it should be white

Feb 8 2017, 7:50 PM · Patch-For-Review, Wikimedia-Portals, Discovery-Analysis (Current work), Discovery
mpopov added a comment to T152617: Wikipedia portal: update eventlogging for sister project clickthroughs.

@debt: the dark axes is a known bug that we can't reproduce / don't know what causes it :\ if you click on a different dashboard and then go back to the sister projects one, it should be white

Feb 8 2017, 7:49 PM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
debt awarded T153936: Create usage stats for WDQS LDF endpoint a Like token.
Feb 8 2017, 4:29 PM · Patch-For-Review, Discovery-Analysis (Current work), Discovery

Feb 7 2017

mpopov moved T154634: Wikipedia.org: add app links to Portal dashboard from Stalled/Waiting to Needs review on the Discovery-Analysis (Current work) board.

Up on beta: http://discovery-beta.wmflabs.org/portal/#app_links

Feb 7 2017, 11:38 PM · Patch-For-Review, Wikimedia-Portals, Discovery-Analysis (Current work), Discovery
mpopov moved T152617: Wikipedia portal: update eventlogging for sister project clickthroughs from Stalled/Waiting to Needs review on the Discovery-Analysis (Current work) board.

Up on beta: http://discovery-beta.wmflabs.org/portal/#sister_projects

Feb 7 2017, 11:36 PM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
mpopov moved T153936: Create usage stats for WDQS LDF endpoint from Stalled/Waiting to Needs review on the Discovery-Analysis (Current work) board.

Up on beta: http://discovery-beta.wmflabs.org/wdqs/#endpoint_usage

Feb 7 2017, 11:34 PM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
mpopov added a comment to T149806: [A/B/C Test] Add cross-wiki search results in a right sidebar.

@EBernhardson: nevermind what I wrote earlier :) here's the actual answer, from the task description: "This test is expected to last at least a week and will be run on Persian, Italian, Catalan and Polish Wikipedias (their selection was based on community input)."

Feb 7 2017, 9:06 PM · MW-1.29-release (WMF-deploy-2017-02-28_(1.29.0-wmf.14)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), MW-1.29-release-notes, MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-01-31_(1.29.0-wmf.10)), Patch-For-Review, Discovery-Search (Current work), CirrusSearch, Discovery
mpopov added a comment to T149806: [A/B/C Test] Add cross-wiki search results in a right sidebar.

I'm OK with the same sampling on those wikis and OK with the current sampling rates we have in the description.

Feb 7 2017, 7:41 PM · MW-1.29-release (WMF-deploy-2017-02-28_(1.29.0-wmf.14)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), MW-1.29-release-notes, MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-01-31_(1.29.0-wmf.10)), Patch-For-Review, Discovery-Search (Current work), CirrusSearch, Discovery
mpopov added a comment to T149806: [A/B/C Test] Add cross-wiki search results in a right sidebar.

I'm not seeing in the task description, exactly which wiki's are we enabling the test on at first? Only enwiki?

Feb 7 2017, 7:29 PM · MW-1.29-release (WMF-deploy-2017-02-28_(1.29.0-wmf.14)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), MW-1.29-release-notes, MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-01-31_(1.29.0-wmf.10)), Patch-For-Review, Discovery-Search (Current work), CirrusSearch, Discovery
mpopov added a comment to T156686: Search metrics: add sister projects to dashboard.

@debt: I think we discussed this and concluded that we will not be proceeding on this and that Olga's team will be using Pivot, correct?

Feb 7 2017, 1:14 AM · Discovery-Analysis (Current work), Discovery
mpopov moved T150915: [Dashboard] Migrate golden to Reportupdater infrastructure from In progress to Needs review on the Discovery-Analysis (Current work) board.
Feb 7 2017, 1:13 AM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
mpopov edited the description of T150915: [Dashboard] Migrate golden to Reportupdater infrastructure.
Feb 7 2017, 1:13 AM · Patch-For-Review, Discovery-Analysis (Current work), Discovery

Feb 3 2017

mpopov added a comment to T149806: [A/B/C Test] Add cross-wiki search results in a right sidebar.

@mpopov with chelsy out, do you want to review the data collection here and ensure we are collecting everything necessary? Links above will opt you into the test on sistersearch and show eventlogging as described.

Feb 3 2017, 1:10 AM · MW-1.29-release (WMF-deploy-2017-02-28_(1.29.0-wmf.14)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), MW-1.29-release-notes, MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-01-31_(1.29.0-wmf.10)), Patch-For-Review, Discovery-Search (Current work), CirrusSearch, Discovery

Feb 2 2017

mpopov edited the description of T150915: [Dashboard] Migrate golden to Reportupdater infrastructure.
Feb 2 2017, 10:23 PM · Patch-For-Review, Discovery-Analysis (Current work), Discovery

Jan 31 2017

mpopov added a comment to T151832: Add Maps tile usage counts as a Data Cube in Pivot.

Moving to "radar", do ping us if you need more work from us. To confirm: we keep this data on druid or you guys do not think you would use it and we can delete it?

Jan 31 2017, 10:02 PM · Analytics-Kanban, Patch-For-Review, Discovery-Analysis (Current work), Discovery

Jan 27 2017

mpopov created T156523: Investigate adding user-friendly testing functionality to Reportupdater.
Jan 27 2017, 8:18 PM · Analytics
mpopov edited parent tasks for T156300: analysis of results from A/B/C test for displaying sister projects in search results, added: T156512: Check that data coming from cross-wiki test is valid; removed: T149806: [A/B/C Test] Add cross-wiki search results in a right sidebar.
Jan 27 2017, 6:34 PM · Discovery-Analysis (Current work), Discovery
mpopov removed a subtask for T149806: [A/B/C Test] Add cross-wiki search results in a right sidebar: T156300: analysis of results from A/B/C test for displaying sister projects in search results.
Jan 27 2017, 6:34 PM · MW-1.29-release (WMF-deploy-2017-02-28_(1.29.0-wmf.14)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), MW-1.29-release-notes, MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-01-31_(1.29.0-wmf.10)), Patch-For-Review, Discovery-Search (Current work), CirrusSearch, Discovery
mpopov added a subtask for T156512: Check that data coming from cross-wiki test is valid: T156300: analysis of results from A/B/C test for displaying sister projects in search results.
Jan 27 2017, 6:34 PM · MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, Discovery-Analysis (Current work), CirrusSearch, Discovery
mpopov created T156512: Check that data coming from cross-wiki test is valid.
Jan 27 2017, 6:33 PM · MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, Discovery-Analysis (Current work), CirrusSearch, Discovery
mpopov placed T156300: analysis of results from A/B/C test for displaying sister projects in search results up for grabs.
Jan 27 2017, 6:24 PM · Discovery-Analysis (Current work), Discovery
mpopov claimed T156300: analysis of results from A/B/C test for displaying sister projects in search results.
Jan 27 2017, 6:21 PM · Discovery-Analysis (Current work), Discovery
mpopov added a comment to T149806: [A/B/C Test] Add cross-wiki search results in a right sidebar.

Another clarification, in the spec we state:

Eventlogging needs to capture:
* if results were shown and from which wiki projects
...

I think we decided that this information would be obtained by joining the EventLogging data against the CirrusSearchRequestSet data. The link between the two is already collected, and the CirrusSearchRequestSet logs already contain the list of wiki's queried and the # of results returned for each query.

Jan 27 2017, 6:19 PM · MW-1.29-release (WMF-deploy-2017-02-28_(1.29.0-wmf.14)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), MW-1.29-release-notes, MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-01-31_(1.29.0-wmf.10)), Patch-For-Review, Discovery-Search (Current work), CirrusSearch, Discovery

Jan 26 2017

mpopov added a comment to T149806: [A/B/C Test] Add cross-wiki search results in a right sidebar.

@mpopov Should we continue using the existing 'click' event, or perhaps use a new 'iwclick' even or some such (maybe ssclick for sister search, so as not to confuse with textcat interwiki)? I suppose my worry is that the meaning of 'position' changes, depending on if we are talking about interwiki results or regular results. As such it may also be usefull to distinguish that with a separate event name?

Jan 26 2017, 11:52 PM · MW-1.29-release (WMF-deploy-2017-02-28_(1.29.0-wmf.14)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), MW-1.29-release-notes, MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-01-31_(1.29.0-wmf.10)), Patch-For-Review, Discovery-Search (Current work), CirrusSearch, Discovery
mpopov moved T154634: Wikipedia.org: add app links to Portal dashboard from Backlog to Stalled/Waiting on the Discovery-Analysis (Current work) board.
Jan 26 2017, 8:33 PM · Patch-For-Review, Wikimedia-Portals, Discovery-Analysis (Current work), Discovery
mpopov moved T153936: Create usage stats for WDQS LDF endpoint from Backlog to Stalled/Waiting on the Discovery-Analysis (Current work) board.
Jan 26 2017, 8:33 PM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
mpopov moved T152728: Analyze access log to see whether we need to add filetype: aliases from Needs review to Done on the Discovery-Analysis (Current work) board.

Good job, @chelsyx!

Jan 26 2017, 7:59 PM · Discovery-Analysis (Current work), Discovery-Search, Discovery, CirrusSearch
mpopov awarded T152728: Analyze access log to see whether we need to add filetype: aliases a 100 token.
Jan 26 2017, 7:58 PM · Discovery-Analysis (Current work), Discovery-Search, Discovery, CirrusSearch

Jan 24 2017

mpopov claimed T154634: Wikipedia.org: add app links to Portal dashboard.
Jan 24 2017, 11:07 PM · Patch-For-Review, Wikimedia-Portals, Discovery-Analysis (Current work), Discovery

Jan 23 2017

mpopov committed R1821:af94de9d5a31: Support more MySQL config filenames (authored by mpopov).
Support more MySQL config filenames
Jan 23 2017, 8:19 PM

Jan 18 2017

mpopov claimed T153936: Create usage stats for WDQS LDF endpoint.
Jan 18 2017, 11:17 PM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
mpopov committed R1821:98769c609481: Add more Hive error/log filters (authored by mpopov).
Add more Hive error/log filters
Jan 18 2017, 9:00 PM
mpopov added a parent task for T154634: Wikipedia.org: add app links to Portal dashboard: T150915: [Dashboard] Migrate golden to Reportupdater infrastructure.
Jan 18 2017, 6:42 PM · Patch-For-Review, Wikimedia-Portals, Discovery-Analysis (Current work), Discovery
mpopov added a subtask for T150915: [Dashboard] Migrate golden to Reportupdater infrastructure: T154634: Wikipedia.org: add app links to Portal dashboard.
Jan 18 2017, 6:42 PM · Patch-For-Review, Discovery-Analysis (Current work), Discovery

Jan 17 2017

mpopov added a comment to T151832: Add Maps tile usage counts as a Data Cube in Pivot.

Okie dokie, thank you @Nuria! I will see if I can replicate your Druid-loading steps to get December's counts into Druid/Pivot.

Jan 17 2017, 9:36 PM · Analytics-Kanban, Patch-For-Review, Discovery-Analysis (Current work), Discovery

Jan 9 2017

mpopov edited the description of T150915: [Dashboard] Migrate golden to Reportupdater infrastructure.
Jan 9 2017, 7:08 PM · Patch-For-Review, Discovery-Analysis (Current work), Discovery

Jan 6 2017

mpopov removed a project from T154717: Maps Dashboard - add notation for increased tile usage: Patch-For-Review.

Noted in production: http://discovery.wmflabs.org/maps/#tiles_summary

Jan 6 2017, 7:46 PM · Discovery-Analysis (Current work), Discovery
mpopov added a comment to T154717: Maps Dashboard - add notation for increased tile usage.

Yup, that checks out:

Jan 6 2017, 7:33 PM · Discovery-Analysis (Current work), Discovery
mpopov updated subscribers of T149440: [EPIC] Detect bots from searches (and learn C++/Rcpp).

@TJones, @chelsyx, and I discussed this today and we're starting to think it would be more useful to instead classify users as "typical" vs "atypical", rather than "bot" vs "person" because there may be real people using search in really weird and highly specific ways that would still negatively impact our metrics and make it difficult for us to gauge how we're serving regular users.

Jan 6 2017, 1:58 AM · Epic, Discovery, Discovery-Analysis (Current work)
mpopov moved T112170: Model user behavior and detect when reality heavily deviated from expectation from In progress to Stalled/Waiting on the Discovery-Analysis (Current work) board.
Jan 6 2017, 1:55 AM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
mpopov moved T152617: Wikipedia portal: update eventlogging for sister project clickthroughs from In progress to Stalled/Waiting on the Discovery-Analysis (Current work) board.
Jan 6 2017, 1:55 AM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
mpopov moved T150915: [Dashboard] Migrate golden to Reportupdater infrastructure from Needs review to In progress on the Discovery-Analysis (Current work) board.
Jan 6 2017, 1:55 AM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
mpopov moved T153715: Portal dashboard heading "No. Events by Country" could be clearer from Needs review to Done on the Discovery-Analysis (Current work) board.
Jan 6 2017, 1:55 AM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
mpopov moved T153716: Portal dashboard: Mexico shouldn't be listed as being in South America from Needs review to Done on the Discovery-Analysis (Current work) board.
Jan 6 2017, 1:55 AM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
mpopov moved T138107: [EPIC] Wikipedia Portal Dashboard: expand "other" countries data display from Needs review to Done on the Discovery-Analysis (Current work) board.
Jan 6 2017, 1:55 AM · Patch-For-Review, Discovery-Analysis (Current work), Wikimedia-Portals, Discovery, Epic
mpopov added a comment to T138107: [EPIC] Wikipedia Portal Dashboard: expand "other" countries data display.

On beta: http://discovery-beta.wmflabs.org/portal/#all_country

Jan 6 2017, 1:54 AM · Patch-For-Review, Discovery-Analysis (Current work), Wikimedia-Portals, Discovery, Epic
mpopov added a comment to T153715: Portal dashboard heading "No. Events by Country" could be clearer.

On beta: http://discovery-beta.wmflabs.org/portal/#all_country

Jan 6 2017, 1:54 AM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
mpopov added a comment to T153716: Portal dashboard: Mexico shouldn't be listed as being in South America.

On beta: http://discovery-beta.wmflabs.org/portal/#all_country

Jan 6 2017, 1:54 AM · Patch-For-Review, Discovery-Analysis (Current work), Discovery
mpopov awarded T147500: Analyze results of the second BM25 test a Like token.
Jan 6 2017, 1:52 AM · Discovery-Analysis (Current work), Discovery

Jan 5 2017

mpopov added a comment to T147500: Analyze results of the second BM25 test.

@chelsyx: Looks great! Only minor change I'd suggest is to elevate the percentages on Figs 16 & 17 a little bit so they're not right on top of the bars, and then it's done! :D

Jan 5 2017, 11:02 PM · Discovery-Analysis (Current work), Discovery

Jan 4 2017

mpopov added a comment to T146807: Wikipedia/Wikimedia apps availability test: analyze results.

@debt: November & December totals, as requested:

Jan 4 2017, 10:29 PM · Patch-For-Review, Discovery-Analysis (Current work), Wikimedia-Portals, Discovery-Portal-Sprint, Discovery
mpopov added a comment to T150215: [Dashboard][Search] Sparklines for KPIs.

Deployed! :D Good job, @chelsyx!~

Jan 4 2017, 7:56 PM · Patch-For-Review, Discovery-Analysis (Current work), Discovery

Jan 3 2017

mpopov moved T153887: Search Metrics Dashboard: Investigate recent spike in user engagement from In progress to Done on the Discovery-Analysis (Current work) board.

Thanks @chelsyx for looking into it!

Jan 3 2017, 10:17 PM · Discovery-Analysis (Current work), Discovery
mpopov added a comment to T153856: Add CI to all wikimedia/discovery repositories that are active.

The dashboard repositories are R/Shiny applications (e.g. http://discovery.wmflabs.org/metrics/) that we can't apply CI to.

Jan 3 2017, 10:07 PM · Discovery-Analysis, Discovery, Release-Engineering-Team, Continuous-Integration-Config