@Dbrant: which release of the app will have the fix?

In T187239#4033569, @Tbayer wrote:

Great work! Are we going to document findings about each schema on that schema's documentation page (or associated talk page) too?

In T187239#4025260, @Dbrant wrote:

/* Funnel.SAMPLE_LOG_ALL sampling by default, right? Also channel info is missing :( Phab task incoming... */
public InstallReferrerFunnel(WikipediaApp app) {
    super(app, SCHEMA_NAME, REV_ID);
}

Correct; all of these events are sent for all users. (Will reply in separate task regarding channel info.)

Update: we decided to change this task to be about extending all schemas to include a client timestamp.

Forgot to comment here but Android monthly metrics for Feb 2018 have been updated as well.

@Charlotte: I'm going to share the spreadsheet with you and Dmitry. Once you review, can you please comment on this ticket so we know whether to move it into our Done column? ta!

Looks good! I don't think CI stuff is that important here, so shall we move it into Done?

Waiting for systems users with private data access to become available.

Shall we move this into the "Done" column as the report looks to have been reviewed?

I think I need to redo some/most/all of this once the sampling bug (T186682) fix is deployed and we've collected enough data from users of the new version.

In T187239#4025260, @Dbrant wrote:

To clarify, the sampling logic in the app works like this:

• When the app is installed, a unique appInstallId is generated and saved, which is a random UUID. (this will identify this "user" across different schemas)
• To decide whether a certain funnel's events are sent or not, we take the last digits of the appInstallId, and if those digits equal zero, modulo the sample rate (e.g. SAMPLE_LOG_100), then the events from that funnel will be sent. Otherwise the funnel will be silent.

That's basically it; there's no other random selection at work. This has the following implications:

• If the appInstallId = 0 mod 100, then all funnels with SAMPLE_LOG_100 will be enabled, as well as all funnels with SAMPLE_LOG_10.
• If the appInstallId = 0 mod 10, then all funnels with SAMPLE_LOG_10 will be enabled, but not necessarily funnels with SAMPLE_LOG_100.
• Funnels with SAMPLE_LOG_ALL are always enabled, since appInstallId mod 1 is always 0.

And there's data! Thank you!

In T187104#4013380, @mforns wrote:

Hi @mpopov!

Is there a special login I'll need to use?

I enter piwik.wikimedia.org with my Wikitech credentials.

In T187104#4012317, @Nuria wrote:

You just have to add tracking code to your site

In T188557#4012125, @Dbrant wrote:

Yes indeed, there were some old versions of the app (from around Dec 2016) that were erroneously populating the wiki field with the app version, but is this actually happening with any recent versions of the app?

I never do cross-wiki joins, so no objections from me either. BUT we do have Maps prevalence metrics calculated on a per-wiki basis so we will need to know which wikis will be on which hosts.

@Tbayer: I'm trying to figure out the -100 in ROUND(100*SUM(IF(month = 3 AND day <= 28, view_count, null)) / SUM(IF(month = 2, view_count, null)) -100,1) and no success 😕

@Charlotte: since the user can switch between modes multiple times in any time period, are we interested in (1) % of users who have tried out the two modes† or (2) at a particular snapshot in time, what's the breakdown of people using each theme?

@Dbrant @Sharvaniharan @cooltey: I'm trying to figure out the different sampling configurations as part of this audit. Can one of you please review my guesses? Thanks!

Make battery usage more efficient

@Sharvaniharan: hi! What's the intention behind this and what data did you have in mind?

In T184095#3987158, @JAllemandou wrote:

Some ideas of improvement:

• Use parquet file format instead of default hive format
• Store data daily instead of hourly (~10Mb per hour in hive format, meaning ~250Mb per day, plus parquet compaction --> Should be good)
• Add request_count instead of keeping distincts
• Prevent errors in happending data using OVERWRITE
• Enforce number of files hive output (default is way too many, therefore small, therefore inefficient)

Updated version of the code below:

@Nuria @JAllemandou: I ran a query over the weekend because I need a processed subset of webrequests from Dec & Jan to work with. Everything was fine and last one I remember working right was 2017-12-05, but I just checked in and at some point I just kept getting:

Note to future self: will need to join MobileWikiAppReadingLists events with wmf.webrequest data in a way that makes it easier to do T184094 also.

What sources/referrers are bringing people to the app?

@MNeisler good job! I like how you wrote your findings. Some initial feedback about the visualizations:

Done with DAU/MAU/stickiness part: https://github.com/wikimedia-research/App-Android-Baseline_Metrics/tree/master/T184089#app-stickiness

@Nuria: my blog post is ready to be published but I'm holding off until I add the piwik stuff :) if there's a chance you can help with this sometime this week that would be awesome and appreciated!

In T186682#3964824, @Nuria wrote:

Going forward let's please make a practice for developers to do basic vetting of metrics. Example: notice that in this case to see the sampling oddities it was enough to add the uniques in both sources (a simple addition, no stats). Let's not fire and forget metrics but rather follow through a bit to make sure things add up.

For future self: https://github.com/slc-rug/2018-02-power-simulation/blob/master/PowerSimulation-handout.pdf & https://www.youtube.com/watch?v=Oj7dGg2sraQ

@Tbayer Done! Thank you so much for the suggestions! And yep, the benchmarks are calculated from popular free Books & Reference apps, not the whole Play Store like I previously thought.

@Charlotte & @JKatzWMF: okay, here's my progress so far: https://github.com/wikimedia-research/App-Android-Baseline_Metrics/tree/master/T184089 Am I on the right track or nah?

In T185131#3930983, @Andrew wrote:

This is ok, but keep in mind that Trusty is deprecated throughout WMF infrastructure (including on WMCS VMs). Don't get too attached to these VMs -- I'd advise moving all of your systems to Stretch as soon as you're able.

Changed the name back because I want to keep these up as a learning tool for anyone who missed the workshop and future WMF employees.

In T174465#3908880, @MoritzMuehlenhoff wrote:

This is solely for T174110 or are we anticipating other use cases?

Static version: https://bearloga.github.io/wmf-allhands18/ and here's the interactive app (which has a quiz): http://dataviz-literacy.wmflabs.org/

Thank you very much, @Andrew!

In T185131#3907289, @Andrew wrote:

I will create these VMs for you. What shall I call them?

I don't think limiting a survey to a predetermined maximum of n responses is the way to go for the reasons @EBernhardson mentioned. The limiting factor should only be the start and end times/dates, ideally so that all timezones are equally represented.

In T172581#3888536, @Nuria wrote:

Call me crazy but i bet if we ask google for this data they will be happy to give it to us w/o having to setup web scraping/downloads
Again, call me crazy but i bet this data could be made public by google 100% such you do not need authentication to query it , we woudl be able to do it and so will be any interested party. seems that it would require a few conversations but little actual hands-on work

In T184019#3869965 & T184019#3870041, @EBernhardson wrote:

@mpopov I wasn't quite sure from https://wikimedia-research.github.io/Discovery-Search-Adhoc-RelevanceSurveys/#responses_required , is 40 to 70 responses the number of impressions (yes+no+dismiss+timeout), the number of clicks (yes+no+dismiss), or the number of yes+no? I think it was yes+no+dismiss, but it might have been yes+no+dismiss+timeout?

Closer reading of the report:

the model is very accurate with at least 40 yes/no/unsure/dismiss responses and the most accurate with at least 70 responses

I think is saying that we are not considering timeouts here, which means with an ~30% response rate to get 70 responses we need 210 impressions?

Just tried the second link (API results for "sistema parlamentario con sede de gobierno" on enwiki (including results from eswiki)) and got the following:

Added Python version into the production instructions for @EBernhardson's convenience :) https://github.com/wikimedia-research/Discovery-Search-Adhoc-RelevanceSurveys/tree/master/production#predicting-rank

In T175048#3851422, @TJones wrote:

Thanks! I'm not sure what I was expecting, but it is interesting to see. It seems to like giving scores of 0.5, but a lot of models end up with a sort of "default" score they like best. I am surprised that it doesn't show any scores above 0.75. Should we map scores from a 0-0.75 range, rather then 0-1? Or, based on the low end of the trend line, maybe even 0.25-075?

Alrighty, here ya go! It's not as pretty as you were probably expecting!

@EBernhardson @chelsyx do you see any errors?