| • Charlotte | |
| Jan 10 2018, 7:08 PM |
| F13157992: prop_view_page_randomizer.png | |
| Feb 7 2018, 1:08 AM |
| F13157994: median_time_per_user.png | |
| Feb 7 2018, 1:08 AM |
| F13157978: prop_use_randomizer.png | |
| Feb 7 2018, 1:08 AM |
| F13157957: median_pv_from_randomizer.png | |
| Feb 7 2018, 1:08 AM |
| F13157975: prop_pv_from_random.png | |
| Feb 7 2018, 1:08 AM |
| F13158009: feed_card_ctr.png | |
| Feb 7 2018, 1:08 AM |
| F12644832: median_time_per_user.png | |
| Jan 18 2018, 9:41 PM |
| F12644830: median_pv_from_randomizer.png | |
| Jan 18 2018, 9:41 PM |
For the quarterly update, we'd like to be able to answer this question:
Did the Android release on 8 December 2017 increase:
If the release moved any of those metrics we'd love to know.
@Charlotte Just a heads up. hey we just saw this, and are not sure it can be delivered by Tuesday.
Hey @Charlotte !
The first graph shows the daily number of users interacting with the Randomizer feature. The MobileWikiAppRandomizer table started to collect data from Dec 1 2017. There is only 1 user on Dec 1, 4, 5 respectively, and 30, 22, 197 users on Dec 6,7,8 respectively. I'm not sure whether it is because very few users interacted with the Randomizer feature before 12/8, or the EL table failed to collect data for the first few days. @Dbrant may be able to answer.
SELECT LEFT(timestamp, 8) AS date, COUNT(DISTINCT event_appInstallID) AS n_users FROM MobileWikiAppRandomizer_17490463 WHERE INSTR(userAgent, 'Android') > 0 AND userAgent LIKE '%-r-%' GROUP BY LEFT(timestamp, 8);
The second graph shows the daily number of users clicking on feed cards from Oct 1, 2017. Users are sample at a 1:100 rate to be tracked.
SELECT LEFT(timestamp, 8) AS date, COUNT(DISTINCT event_appInstallID) AS n_users FROM MobileWikiAppFeed_16432467 WHERE event_action = 'cardClicked' AND INSTR(userAgent, 'Android') > 0 AND userAgent LIKE '%-r-%' AND LEFT(timestamp, 6) >= 201710 GROUP BY LEFT(timestamp, 8);
Here is a chart of the daily active users since Nov 1 (part of the apps core metrics I'm tracking routinely as part of this spreadsheet, reporting them in the quarterly core metrics presentation).
It doesn't look like that metric changed much due to the rollout. (There are always seasonal effects at this time of the year, with a large increase in the days before christmas and a slight drop beforehand, cf. last year.) We could look at DAU per version too, but a solid assessment of the effect would probably have required an A/B test.
Thanks @Tbayer for the DAU!
@JKatzWMF and @Charlotte , as we discussed in email, I calculated additional metrics for Randomizer and Feed cards:
Results:
Median pageviews from Randomizer drop to around 1-2, after the new release:
Proportion of pageviews from randomizer has not change a lot:
Proportion of users that interact with the randomizer is around 0.28%. On Jan 26, the number of users interact with randomizer drop drastically. The reason is unknown and further investigation is needed.
The proportion of users and sessions that clickthrough to articles from Randomizer increase.
Median time spent on Randomizer increase slightly.
Session-wise and user-wise clickthrough rate on feed cards decrease to around 8%.
SG9tZVBoYWJyaWNhdG9yCk5vIG1lc3NhZ2VzLiBObyBub3RpZmljYXRpb25zLgoKICAgIFNlYXJjaAoKQ3JlYXRlIFRhc2sKTWFuaXBoZXN0ClQxOTcyODEKRml4IGZhaWxpbmcgd2VicmVxdWVzdCBob3VycyAodXBsb2FkIGFuZCB0ZXh0IDIwMTgtMDYtMTQtMTEpCk9wZW4sIE5lZWRzIFRyaWFnZVB1YmxpYwoKICAgIEVkaXQgVGFzawogICAgRWRpdCBSZWxhdGVkIFRhc2tzLi4uCiAgICBFZGl0IFJlbGF0ZWQgT2JqZWN0cy4uLgogICAgUHJvdGVjdCBhcyBzZWN1cml0eSBpc3N1ZQoKICAgIE11dGUgTm90aWZpY2F0aW9ucwogICAgQXdhcmQgVG9rZW4KICAgIEZsYWcgRm9yIExhdGVyCgpUYWdzCgogICAgQW5hbHl0aWNzLUthbmJhbiAoSW4gUHJvZ3Jlc3MpCgpTdWJzY3JpYmVycwpBa2xhcHBlciwgSkFsbGVtYW5kb3UKQXNzaWduZWQgVG8KSkFsbGVtYW5kb3UKQXV0aG9yZWQgQnkKSkFsbGVtYW5kb3UsIEZyaSwgSnVuIDE1CkRlc2NyaXB0aW9uCgpPb3ppZSBqb2JzIGhhdmUgYmVlbiBmYWlsaW5nIGF0IGxlYXN0IGEgZmV3IHRpbWVzIGVhY2guIE1vcmUgaW52ZXN0aWdhdGlvbiBuZWVkZWQuCkpBbGxlbWFuZG91IGNyZWF0ZWQgdGhpcyB0YXNrLkZyaSwgSnVuIDE1LCA3OjIxIEFNCkhlcmFsZCBhZGRlZCBhIHN1YnNjcmliZXI6IEFrbGFwcGVyLiC3IFZpZXcgSGVyYWxkIFRyYW5zY3JpcHRGcmksIEp1biAxNSwgNzoyMSBBTQpKQWxsZW1hbmRvdSBjbGFpbWVkIHRoaXMgdGFzay5GcmksIEp1biAxNSwgNzoyMiBBTQpKQWxsZW1hbmRvdSB1cGRhdGVkIHRoZSB0YXNrIGRlc2NyaXB0aW9uLiAoU2hvdyBEZXRhaWxzKQpKQWxsZW1hbmRvdSBhZGRlZCBhIHByb2plY3Q6IEFuYWx5dGljcy1LYW5iYW4uCkpBbGxlbWFuZG91IG1vdmVkIHRoaXMgdGFzayBmcm9tIE5leHQgVXAgdG8gSW4gUHJvZ3Jlc3Mgb24gdGhlIEFuYWx5dGljcy1LYW5iYW4gYm9hcmQuCkNoYW5nZSBTdWJzY3JpYmVycwpDaGFuZ2UgUHJpb3JpdHkKQXNzaWduIC8gQ2xhaW0KTW92ZSBvbiBXb3JrYm9hcmQKQ2hhbmdlIFByb2plY3QgVGFncwpBbmFseXRpY3MtS2FuYmFuCtcKU2VjdXJpdHkK1wpXaWtpbWVkaWEtVkUtQ2FtcGFpZ25zIChTMi0yMDE4KQrXClNjYXAK1wpTY2FwIChTY2FwMy1BZG9wdGlvbi1QaGFzZTIpCtcKQWJ1c2VGaWx0ZXIK1wpEYXRhLXJlbGVhc2UK1wpIYXNodGFncwrXCkxhYnNEQi1BdWRpdG9yCtcKTGFkaWVzLVRoYXQtRk9TUy1NZWRpYVdpa2kK1wpMYW5ndWFnZS0yMDE4LUFwci1KdW5lCtcKTGFuZ3VhZ2UtMjAxOC1KYW4tTWFyCtcKSEhWTQrXCkhBV2VsY29tZQrXCkJvbGQKSXRhbGljcwpNb25vc3BhY2VkCkxpbmsKQnVsbGV0ZWQgTGlzdApOdW1iZXJlZCBMaXN0CkNvZGUgQmxvY2sKUXVvdGUKVGFibGUKVXBsb2FkIEZpbGUKTWVtZQpQcmV2aWV3CkhlbHAKRnVsbHNjcmVlbiBNb2RlClBpbiBGb3JtIE9uIFNjcmVlbgoyMzg0ODJuMzc1IGFkZGVkIHByb2plY3RzOiBTZWN1cml0eSwgV2lraW1lZGlhLVZFLUNhbXBhaWducyAoUzItMjAxOCksIFNjYXAgKFNjYXAzLUFkb3B0aW9uLVBoYXNlMiksIEFidXNlRmlsdGVyLCBEYXRhLXJlbGVhc2UsIEhhc2h0YWdzLCBMYWJzREItQXVkaXRvciwgTGFkaWVzLVRoYXQtRk9TUy1NZWRpYVdpa2ksIExhbmd1YWdlLTIwMTgtQXByLUp1bmUsIExhbmd1YWdlLTIwMTgtSmFuLU1hciwgSEhWTSwgSEFXZWxjb21lLlBSRVZJRVcKMjM4NDgybjM3NSBtb3ZlZCB0aGlzIHRhc2sgZnJvbSBJbiBQcm9ncmVzcyB0byBJbiBDb2RlIFJldmlldyBvbiB0aGUgQW5hbHl0aWNzLUthbmJhbiBib2FyZC4KMjM4NDgybjM3NSByZW1vdmVkIEpBbGxlbWFuZG91IGFzIHRoZSBhc3NpZ25lZSBvZiB0aGlzIHRhc2suCjIzODQ4Mm4zNzUgdHJpYWdlZCB0aGlzIHRhc2sgYXMgTG93ZXN0IHByaW9yaXR5LgoyMzg0ODJuMzc1IHJlbW92ZWQgc3Vic2NyaWJlcnM6IEFrbGFwcGVyLCBKQWxsZW1hbmRvdS4KQ29udGVudCBsaWNlbnNlZCB1bmRlciBDcmVhdGl2ZSBDb21tb25zIEF0dHJpYnV0aW9uLVNoYXJlQWxpa2UgMy4wIChDQy1CWS1TQSkgdW5sZXNzIG90aGVyd2lzZSBub3RlZDsgY29kZSBsaWNlbnNlZCB1bmRlciBHTlUgR2VuZXJhbCBQdWJsaWMgTGljZW5zZSAoR1BMKSBvciBvdGhlciBvcGVuIHNvdXJjZSBsaWNlbnNlcy4gQnkgdXNpbmcgdGhpcyBzaXRlLCB5b3UgYWdyZWUgdG8gdGhlIFRlcm1zIG9mIFVzZSwgUHJpdmFjeSBQb2xpY3ksIGFuZCBDb2RlIG9mIENvbmR1Y3QuILcgV2lraW1lZGlhIEZvdW5kYXRpb24gtyBQcml2YWN5IFBvbGljeSC3IENvZGUgb2YgQ29uZHVjdCC3IFRlcm1zIG9mIFVzZSC3IERpc2NsYWltZXIgtyBDQy1CWS1TQSC3IEdQTApZb3VyIGJyb3dzZXIgdGltZXpvbmUgc2V0dGluZyBkaWZmZXJzIGZyb20gdGhlIHRpbWV6b25lIHNldHRpbmcgaW4geW91ciBwcm9maWxlLCBjbGljayB0byByZWNvbmNpbGUu