Wed, Nov 7
@JMinor I just checked the errors for v6.1 in the eventerror table:
We still need to add a source field to identify the source from which the search interface was invoked: the top of feed, the search tab, or from the article view, as stated in this task description. @NHarateh_WMF can you take a look at it? Thanks!
Fri, Nov 2
The bug in MobileWikiAppiOSSearch and the bug in MobileWikiAppiOSUserHistory (T207427) should be fixed in v6.1 (see https://github.com/wikimedia/wikipedia-ios/pull/2700 and https://github.com/wikimedia/wikipedia-ios/pull/2703), which was released yesterday. We should start to see less of these errors as users starting to upgrade their apps to v6.1
Thu, Nov 1
Thanks @Ottomata ! This was fixed in v6.1, which was released yesterday. We should start to see less of these errors as users starting to upgrade their apps to v6.1
Tue, Oct 30
Hi @atgo, since the online campaign was targeting Madhya Pradesh only while the TV campaign was broadcasting nationwide, I have to deal with them separately. And given that I will be OOO this Thur & Fri (sorry I forgot about it last week), I will probably finish the analysis next week.
Mon, Oct 29
Hi all, I closed this one as it has been fixed in T207424
Mon, Oct 22
Also, do we know the page to which the video was taking viewers in facebook? It will be worthy it to look at pageviews for just that one page and plot what we see.
According to T185584, the links we used for the campaign are:
All the changes have been merged. Please check https://discovery.wmflabs.org/metrics/ and let me know if there is any questions.
Fri, Oct 19
Thank you @NHarateh_WMF !
Thu, Oct 18
@NHarateh_WMF Can you fix the name of these fields?
"app_install_ID" -> "app_install_id"
"ts" -> "event_dt"
"type_of_search" -> "search_type"
Tue, Oct 16
@atgo Got you! I will get you the PVs from Madhya Pradesh by languages from March to May, and compare the number year-over-year.
Mon, Oct 15
Thanks for the quick response @atgo!
Hi @atgo ! It's unclear to me what questions you're asking here:
Oct 10 2018
Closing this ticket as it has been done. Please feel free to re-open it if needed.
Oct 3 2018
Thank you @herron !
Oct 2 2018
Hi @nettrom_WMF , sorry we don't have the number. Looks like ServerSideAccountCreation schema and the logging table is the way to go!
Thanks everyone! The patch above gave me access to analytics-search-users. Can I be added to statistics-privatedata-users as well? Thanks!
Hi @herron , I was just checking @mpopov (username: bearloga) 's groups and these three are the ones that he has but I don't, and we're working on similar stuff. @mpopov do you know anything about the stats role?
Sep 28 2018
Sep 27 2018
Sep 26 2018
Thanks @NHarateh_WMF !
Sep 25 2018
Hi @NHarateh_WMF , I tested in xcode but didn't see any search event posted in the console (I saw other events posted), can you check? Thanks!
Sep 24 2018
The main purpose of this ticket is to provide a foreground/background tag so that we can count the foreground uniques using WMF-Last-Access-Global cookie. However, as pointed out in T202664#4609355, centralize cookie management is much harder than an eventlogging solution. Since we are not going to use the cookie for uniques counting, this tag is no longer needed.
Sep 22 2018
Sep 21 2018
Sep 20 2018
Sep 19 2018
Hi @Nuria, @NHarateh_WMF told me that the iOS app can handle cookies -- WMF-Last-Access and WMF-Last-Access-Global in the X-Analytics header are the cookies we get from the domains we access. @mpopov can confirm whether this is the case in Android.
Sep 18 2018
Thanks @Nuria ! This is going to be very helpful to the iOS app team!
As to the task of this ticket -- productionize per-country daily & monthly active app user stats, I suggest we leave this ticket as "Stalled" and come back to modify the query when T186828#4596238 is done. As pointed out by @JAllemandou in T186828#4530420, we only need to store per-country daily and monthly moving forward.
Hi @Nuria we noticed that since August 10th, the SPARQL usage number is very small (see query in T204415#4590108), which is much less than what we saw in logstash: https://logstash.wikimedia.org/goto/74e376f55fcdc3b93e4a7232cfa5203a
Do you know any incident of webrequest that might cause this?
Sep 17 2018
Hi @Smalyshev , the dashboard is updating. But since August 10th, the SPARQL usage number is very small (even 0 for certain days) and the LDF usage number is 0. Did we change the URI of the endpoint?
Sep 7 2018
@JMinor Thanks for the list of the product questions!
Aug 27 2018
I ran the two queries of option 1 ("new") and 4 ("old") in T186828#4505784 for both iOS and Android (see https://people.wikimedia.org/~chelsyx/Apps%20Unique%20Devices%20-%20Query%20Comparison.html for more details):
Aug 23 2018
It is irrelevant only if one doesn't care about the definition of "unique app user" ;) I think the industry standard here is rather to only count active users in the sense of people intentionally using the app, as opposed to it running in the background while they use their device for something else, or merely being installed on the device. (The last one would be more like Google's "installs on active devices".) Ultimately that seems like a question for the product owners, @Charlotte and @JMinor.
That said, I think there might be a pragmatic argument for 1. and 4. over 3. (i.e. counting non-pageview requests as activity too) if the difference isn't too large - has anyone tried to calculate it (beyond the related observations for iOS at T193917#4237597) ? To be sure, I think @chelsyx is right to call out the effect of these background requests for reading lists and feed, both features that didn't exist when the currently used query (4.) was first implemented back in 2015. I wonder if they have the same impact on both platforms - regarding Android, perhaps @Dbrant knows more about how often background requests are likely to happen on days where the user hasn't opened the app.
@Tbayer I have to point out that none of the solution we have currently is perfect -- we are merely looking for something that's workable and relatively reasonable:
- For option 2, which is the count of users who intentionally used the app, the closest we can get is to count the unique ID in MobileWikiAppDailyStats. But for Android, the sampling rate is 1:100. Also remember this event is sent every 24h based on local timezone, not our system timezone (UTC), so the real count we get would be off if comparing to webrequest table.
- For option 3, which is the count of any devices that has at least one pageview, I don't understand why we want to exclude those who open the app without reading an article. It doesn't seem right to me...
- For option 4, the query doesn't make sense any more given that app could fetch info in the background, plus those parsing and string comparisons are computationally expensive. The only value I can see is to compare with historical data.
- Also need to mention that for any of the 4 options, we are counting opt-in users only. Given that the opt-in rate on iOS seems very small, the unique device count is compromised anyway...
- As to Google's "installs on active devices", which is the count of device having the app installed, iTunes doesn't report this number, but there is a way to estimate it: https://www.linkedin.com/pulse/how-estimate-active-installs-ios-sharif-aboulnaga/
Aug 16 2018
Aug 15 2018
Aug 9 2018
Hi @Nuria, could you review the change https://gerrit.wikimedia.org/r/451566 when you have a chance? I've tested the job under my account, see oozie job 0048633-180705103628398-oozie-oozi-C and 0049964-180705103628398-oozie-oozi-C
Aug 8 2018
Yes, I tested again on 22.214.171.1241 and I didn't see the problem as stated in T196933 when testing on 126.96.36.1991.
Aug 3 2018
That's awesome! Thanks @Milimetric !
@Milimetric I mean the latter one. I'm wondering if we save the original user agent strings of each revision somewhere else, so that we can backfill the ios app edit and android app edit tags for those revisions which only have mobile app edit tag in the change_tag table.
Aug 2 2018
@Ramsey-WMF JK said you are actively looking into the search clickthrough rate on Commons, and this bug is making the rate lower than it should be. How soon do you want it to get fixed?
A related ask (but probably should sit in a new ticket): Is it possible to backfill the tag? We started to include tags ios app edit and android app edit from June 29 (T194424), but it would be great if we can backfill these tags for older revisions.
Aug 1 2018
Thanks for the context @Tbayer !
Jul 31 2018
Jul 27 2018
@JKatzWMF I'm closing this ticket as it is done. Feel free to reopen it if needed.
Jul 26 2018
Jul 25 2018
@Jdlrobson This schema is still active. I changed the talk page.
Jul 24 2018
Verified. Thanks all!