Fri, Sep 21
Thu, Sep 20
Wed, Sep 19
Hi @Nuria, @NHarateh_WMF told me that the iOS app can handle cookies -- WMF-Last-Access and WMF-Last-Access-Global in the X-Analytics header are the cookies we get from the domains we access. @mpopov can confirm whether this is the case in Android.
Tue, Sep 18
Thanks @Nuria ! This is going to be very helpful to the iOS app team!
As to the task of this ticket -- productionize per-country daily & monthly active app user stats, I suggest we leave this ticket as "Stalled" and come back to modify the query when T186828#4596238 is done. As pointed out by @JAllemandou in T186828#4530420, we only need to store per-country daily and monthly moving forward.
Hi @Nuria we noticed that since August 10th, the SPARQL usage number is very small (see query in T204415#4590108), which is much less than what we saw in logstash: https://logstash.wikimedia.org/goto/74e376f55fcdc3b93e4a7232cfa5203a
Do you know any incident of webrequest that might cause this?
Mon, Sep 17
Hi @Smalyshev , the dashboard is updating. But since August 10th, the SPARQL usage number is very small (even 0 for certain days) and the LDF usage number is 0. Did we change the URI of the endpoint?
Fri, Sep 7
@JMinor Thanks for the list of the product questions!
Mon, Aug 27
I ran the two queries of option 1 ("new") and 4 ("old") in T186828#4505784 for both iOS and Android (see https://people.wikimedia.org/~chelsyx/Apps%20Unique%20Devices%20-%20Query%20Comparison.html for more details):
Aug 23 2018
It is irrelevant only if one doesn't care about the definition of "unique app user" ;) I think the industry standard here is rather to only count active users in the sense of people intentionally using the app, as opposed to it running in the background while they use their device for something else, or merely being installed on the device. (The last one would be more like Google's "installs on active devices".) Ultimately that seems like a question for the product owners, @Charlotte and @JMinor.
That said, I think there might be a pragmatic argument for 1. and 4. over 3. (i.e. counting non-pageview requests as activity too) if the difference isn't too large - has anyone tried to calculate it (beyond the related observations for iOS at T193917#4237597) ? To be sure, I think @chelsyx is right to call out the effect of these background requests for reading lists and feed, both features that didn't exist when the currently used query (4.) was first implemented back in 2015. I wonder if they have the same impact on both platforms - regarding Android, perhaps @Dbrant knows more about how often background requests are likely to happen on days where the user hasn't opened the app.
@Tbayer I have to point out that none of the solution we have currently is perfect -- we are merely looking for something that's workable and relatively reasonable:
- For option 2, which is the count of users who intentionally used the app, the closest we can get is to count the unique ID in MobileWikiAppDailyStats. But for Android, the sampling rate is 1:100. Also remember this event is sent every 24h based on local timezone, not our system timezone (UTC), so the real count we get would be off if comparing to webrequest table.
- For option 3, which is the count of any devices that has at least one pageview, I don't understand why we want to exclude those who open the app without reading an article. It doesn't seem right to me...
- For option 4, the query doesn't make sense any more given that app could fetch info in the background, plus those parsing and string comparisons are computationally expensive. The only value I can see is to compare with historical data.
- Also need to mention that for any of the 4 options, we are counting opt-in users only. Given that the opt-in rate on iOS seems very small, the unique device count is compromised anyway...
- As to Google's "installs on active devices", which is the count of device having the app installed, iTunes doesn't report this number, but there is a way to estimate it: https://www.linkedin.com/pulse/how-estimate-active-installs-ios-sharif-aboulnaga/
Aug 16 2018
Aug 15 2018
Aug 9 2018
Hi @Nuria, could you review the change https://gerrit.wikimedia.org/r/451566 when you have a chance? I've tested the job under my account, see oozie job 0048633-180705103628398-oozie-oozi-C and 0049964-180705103628398-oozie-oozi-C
Aug 8 2018
Yes, I tested again on 126.96.36.1991 and I didn't see the problem as stated in T196933 when testing on 188.8.131.521.
Aug 3 2018
That's awesome! Thanks @Milimetric !
@Milimetric I mean the latter one. I'm wondering if we save the original user agent strings of each revision somewhere else, so that we can backfill the ios app edit and android app edit tags for those revisions which only have mobile app edit tag in the change_tag table.
Aug 2 2018
@Ramsey-WMF JK said you are actively looking into the search clickthrough rate on Commons, and this bug is making the rate lower than it should be. How soon do you want it to get fixed?
A related ask (but probably should sit in a new ticket): Is it possible to backfill the tag? We started to include tags ios app edit and android app edit from June 29 (T194424), but it would be great if we can backfill these tags for older revisions.
Aug 1 2018
Thanks for the context @Tbayer !
Jul 31 2018
Jul 27 2018
@JKatzWMF I'm closing this ticket as it is done. Feel free to reopen it if needed.
Jul 26 2018
Jul 25 2018
@Jdlrobson This schema is still active. I changed the talk page.
Jul 24 2018
Verified. Thanks all!
Jul 19 2018
Jul 17 2018
Verified. Thanks all!
Verified. Thanks all!
Jul 14 2018
Jul 12 2018
@fdans Yes we can close this task as resolved.
@Nuria I've checked the l field in CentralNoticeBannerHistory is what I want. Thanks!
Jul 11 2018
@mforns If the l field in https://meta.wikimedia.org/wiki/Schema:CentralNoticeBannerHistory works, I guess it's fine?
Thank you very much @mforns! The patch looks good to me!
Thanks @mforns ! The latest revision 18126357 of MobileWikiAppiOSUserHistory, which includes the feed_enabled_list field, is still under development and we haven't send any data to it yet. Any comments and suggestions are welcome!
Jul 10 2018
Jul 6 2018
@JMinor Thanks for the investigation and explanation!
Jul 3 2018
I think the graph in T130432 has changed since then. I executed the provided query and on 2017-04-20 there is a significant x8 jump up, and in 1 year since then it has grown around 75% (don't think this is the natural growth of iOS app usage). Not sure which percentage would it be, but definitely higher.
Yeah that's true. I don't have a better way to estimate the percentage of app users who agree to share their usage report with us and the graph in T130432 is the best approximate I can think of. @Tbayer may have a better idea.
Jun 21 2018
Jun 19 2018
Jun 18 2018
Jun 16 2018
Jun 14 2018
I'm so sorry for the late response. Things has been a bit crazy on my end...
Jun 13 2018
Jun 12 2018
Jun 11 2018
@JoeWalsh I just checked the data from build 1420, out of 122 users, only 1 record had empty app ID and session ID, and its IP address showed it's from our SF office:
I guess it happened when Monte or someone test the build last Friday (cuz I was not in the office), but it's probably hard to reproduce.
Anyway, it's only 1 record, I think you fix it. YAY!!! Thanks so much!