In T194961#4221017, @LGoto wrote:

Hi @mpopov Is this for you?

@EBjune @RobH: BTW @JKatzWMF and I are going to be changing which properties are tracked in GSC (namely getting rid of all but 1-2 HTTP variants in favor of HTTPS) as well as which project classes and languages are tracked. We expect to be done with this "sprint cleaning" endeavor sometime in the next two weeks so I'll post an update here when it's ready.

1. Part 2

Declining for now but open to re-opening in the future if the parent task is re-opened and this work is needed.

@debt: closing for now but if you think this is work that needs to be done feel free to re-open

@debt: closing for now but if you think this is work that needs to be done feel free to re-open

Pinging @RStallman-legalteam & @JbuattiWMF to confirm that Go Fish Digital have signed the NDAs so that Ops can proceed with adding their public SSH key to the list of allowed keys.

In T193694#4180609, @jmatazzoni wrote:

Thanks @mpopov! As you note, not all current mapframe wikipedias are accounted for in your first stats page (missing are Arabic, Bulgarian, Czech, Spanish, Kannada, Latvian, Portuguese, English).

But here is the bigger issue: we are about to release mapframe to 277 more Wikipedias—essentially all wikipedias except nine flagged revision wikis. We need to be able to track usage on these as well. What do you suggest? How should they be added in?

Will your stats page be able to scale up to measure hundreds more? Is the general "Wikipedia" figure already accounting for all wikipedias programmatically, or does it just add up the 11 you list on the page? What about the spreadsheet: what will happen if we start loading hundreds of wikis? Should we pick some representative wikis we want to measure?

@Catrope: I updated the repository with instructions: https://github.com/wikimedia-research/Discovery-Interactive-Adhoc-Usage#re-run-instructions

Sooooo…most of this has actually already been done. We have per-wiki daily stats beginning on 2017-09-14 over at:

@Deskana: Progress update: I have 4 days of data (~12GB gzipped) and right now I have a script that's verifying ~20K IP addresses to determine which ones are legit and which ones spoofed the UA and pretended to be one of those crawlers. As you might expect, that part is taking some time.

In T184092#3998152, @mpopov wrote:

@Charlotte: since the user can switch between modes multiple times in any time period, are we interested in (1) % of users who have tried out the two modes† or (2) at a particular snapshot in time, what's the breakdown of people using each theme?

@Charlotte: did you mean to make this iOS ticket part of the Android baseline analytics epic? Also, iOS team is long way away from having data that can be presented; should we replace the parent task with T192819?

@debt: same as with T183028 can you please update the description if this is still something that will need to be worked on? It looks like this is supposed to happen after T176428 (if that ever happens?) but that's not an A/B test, so what is the requested deliverable?

@debt: can you please update the description if this is still something that will need to be worked on? It looks like this is supposed to happen after T178006 (if that ever happens?) but that's not an A/B test, so what is the requested deliverable?

In T193052#4159509, @chelsyx wrote:

FAQs of Baiduspider (in English, include UA): http://help.baidu.com/question?prod_id=99&class=0&id=3001
How to identify Baiduspider (in Chinese, let me know if you can't understand it with google translate): https://ziyuan.baidu.com/college/articleinfo?id=1002

@chelsyx Can you please help me? I’ve been able to find documentation (UserAgent strings and instructions for verifying) on Google’s, Bing’s, and Yandex’s crawlers but the best I’ve been able to find for Baidu (in English) is this blog post by a third party from 2011: https://chineseseoshifu.com/blog/new-baidu-user-agent-baiduspider.html so I suspect any official documentation about what Baiduspider’s UA looks like these days (or how to verify) would be on Baidu's website and in Chinese.

@Deskana: Googlebot Mobile does not seem to be a thing anymore according to its lack of presence on https://support.google.com/webmasters/answer/1061943

In T191859#4156112, @Tbayer wrote:

Method 1 has the disadvantage that we would be able to find out username given crossDeviceID, which is not the case for Method 2.

How is that not the case for Method 2?

In T191859#4155885, @Nuria wrote:

@mpopov FYI that adding things to X-Analytics does not work automagically, we strongly recommend team to track interactions using events.
Want to clarify With @chelsyx and @Fjalapeno that events do not have to be send from client side, the server can also send them. EL has a mediawiki server side client too.

@APalmer_WMF @Fjalapeno @Jhernandez: can y'all please take a look at the updated description and let us know if you have any questions or concerns.

Done

In T191859#4152718, @Tgr wrote:

All these new proposals sound a bit overcomplicated. Why not just use X-Analytics? There is already a purge mechanism for raw webrequest data, right?

0.7.0 on CRAN as of 2018-03-21. Re-installed the package on prod & beta

Majority of this is done. Deb can ping Chelsy or me if she has any more questions or issues accessing the data.

Thanks, everyone!

Good job, @MNeisler!