Page MenuHomePhabricator
Create Task
Maniphest T120292

MobileWebSectionUsage schema changes to measure impact of section collapsing
Closed, ResolvedPublic2 Estimated Story Points
Actions

Description

Top priority for sprint - we should aim to get this in the 14th Dec deployment train

The data collected in T118041 could be interpreted by skeptics as saying:

  1. Users do not know that they can open content (bad UX)
  2. People are coming from search and unable to find the content they want e.g. via find in page

To explore this and provide answers we will run an A/B test in stable using a slight schema change. (note since we are already running on a 1% sample, only 0.05% will load the mobile site with sections open by default.

This will allow us to draw some conclusions such as:

  • If users do not scroll headings into view => people are satisfied with lead section only to get the information they need
  • Is toggling bad or good for users - e.g. if users see more sections when the sections toggled open it suggests we are disrupting the reading experience.

(plus more!)

  • Update schema revision to 14957482
  • The new schema has a "scrolled-into-view" event. This should be triggered when the top of the heading is scrolled into the user's viewport.
  • The new schema has a "isTestA" field - this should be true for 50% of the bucketed users.
  • When isTestA is true, sections should not be collapsed by default on the mobile site.

Details

SubjectRepoBranchLines +/-
Enable the A/B test to measure impact of collapsing contentoperations/mediawiki-configmaster+5 -0
Allow measuring impact of section collapsing in Schema:MobileWebSectionUsagemediawiki/extensions/MobileFrontendmaster+92 -14
Customize query in gerrit

Related Objects
Search...

Event Timeline

Jdlrobson created this task.Dec 3 2015, 9:58 PM
Jdlrobson raised the priority of this task from to High.
Jdlrobson updated the task description. (Show Details)
Jdlrobson added projects: Reading Web Sprint 62 - DJ-Jazzy-Jeff-and-the-Fresh-Sprints, MobileFrontend.
Jdlrobson moved this task to Needs Analysis on the Reading Web Sprint 62 - DJ-Jazzy-Jeff-and-the-Fresh-Sprints board.
Jdlrobson subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 3 2015, 9:58 PM
Jdlrobson renamed this task from EventLogging schema changes to MobileSectionUsage schema changes to measure impact of section collapsing.Dec 3 2015, 9:58 PM
Jdlrobson set Security to None.
Jdlrobson edited a custom field.
Jdlrobson updated the task description. (Show Details)Dec 3 2015, 10:05 PM
• Tbayer mentioned this in T118041: Analysis data in MobileWebSectionUsage.Dec 4 2015, 10:17 PM
• Tbayer subscribed.Dec 4 2015, 10:27 PM

Regarding the sample rate (JonR already suggested this when we drafted this on Wednesday, and I went a bit back and forth on it, but on reflection):

I think we can take this occasion to decrease the overall sample rate from 1:100 to 1:1000, in the interest of Datensparsamkeit and faster queries. The only downside I can see is that it will get harder or impossible to analyze section openings for specific pages. But we already have a full month of 1:100 sampled date for that now, and if we ever want to do that more thoroughly, we could create a separate schema without session IDs.

• KLans_WMF edited a custom field.Dec 7 2015, 5:25 PM
• KLans_WMF moved this task from Needs Analysis to To Do on the Reading Web Sprint 62 - DJ-Jazzy-Jeff-and-the-Fresh-Sprints board.
• bmansurov renamed this task from MobileSectionUsage schema changes to measure impact of section collapsing to MobileWebSectionUsage schema changes to measure impact of section collapsing.Dec 9 2015, 2:09 AM
• bmansurov claimed this task.
• bmansurov updated the task description. (Show Details)
• bmansurov moved this task from To Do to Doing on the Reading Web Sprint 62 - DJ-Jazzy-Jeff-and-the-Fresh-Sprints board.
gerritbot subscribed.Dec 9 2015, 3:51 AM

Change 257820 had a related patch set uploaded (by Bmansurov):
Allow measuring impact of section collapsing in Schema:MobileWebSectionUsage

https://gerrit.wikimedia.org/r/257820

gerritbot added a project: Patch-For-Review.Dec 9 2015, 3:51 AM
• bmansurov added a comment.Dec 9 2015, 3:51 AM

A couple of questions:

  • Should we send the "scrolled-into-view" event for each section of the page or just the first heading? Currently, I'm sending it when the user scrolls the first heading into the viewport or if it's already scrolled into the viewport.
  • Have we agreed to change the sampling rate to 1:1000?

Btw, I updated the schema to make the isTestA field optional (as other events don't use it).

• bmansurov moved this task from Doing to Code Review on the Reading Web Sprint 62 - DJ-Jazzy-Jeff-and-the-Fresh-Sprints board.Dec 9 2015, 3:51 AM
• Tbayer added a comment.Dec 9 2015, 6:15 AM
In T120292#1864549, @bmansurov wrote:

A couple of questions:

  • Should we send the "scrolled-into-view" event for each section of the page or just the first heading? Currently, I'm sending it when the user scrolls the first heading into the viewport or if it's already scrolled into the viewport.

As we discussed it last week, it's meant to fire for each section that comes into view (the wording "top of the heading" in the task description was meant to pinpoint the exact condition; I guess what matters most here is that the user has been able to read the content of the preceding section). We want to find out, among other things, whether collapsing sections means that content is hidden from readers that they would otherwise read - for each section, not just the first one after the lede.

• bmansurov moved this task from Code Review to Doing on the Reading Web Sprint 62 - DJ-Jazzy-Jeff-and-the-Fresh-Sprints board.Dec 9 2015, 10:05 AM
• bmansurov moved this task from Doing to Code Review on the Reading Web Sprint 62 - DJ-Jazzy-Jeff-and-the-Fresh-Sprints board.Dec 9 2015, 10:47 AM
Jdlrobson moved this task from Code Review to -1 (Needs More Work) on the Reading Web Sprint 62 - DJ-Jazzy-Jeff-and-the-Fresh-Sprints board.Dec 9 2015, 5:24 PM
Jdlrobson moved this task from -1 (Needs More Work) to Code Review on the Reading Web Sprint 62 - DJ-Jazzy-Jeff-and-the-Fresh-Sprints board.Dec 9 2015, 10:06 PM
gerritbot added a comment.Dec 9 2015, 10:43 PM

Change 257820 merged by jenkins-bot:
Allow measuring impact of section collapsing in Schema:MobileWebSectionUsage

https://gerrit.wikimedia.org/r/257820

gerritbot added a comment.Dec 9 2015, 10:43 PM

Change 258053 had a related patch set uploaded (by Jdlrobson):
Enable the A/B test to measure impact of collapsing content

https://gerrit.wikimedia.org/r/258053

ReleaseTaggerBot added a project: MW-1.27-release (WMF-deploy-2015-12-15_(1.27.0-wmf.9)).Dec 9 2015, 11:00 PM
Jdlrobson moved this task from Code Review to Ready for Signoff on the Reading Web Sprint 62 - DJ-Jazzy-Jeff-and-the-Fresh-Sprints board.Dec 9 2015, 11:18 PM
Jdlrobson moved this task from Ready for Signoff to Code Review on the Reading Web Sprint 62 - DJ-Jazzy-Jeff-and-the-Fresh-Sprints board.
gerritbot added a comment.Dec 10 2015, 12:16 AM

Change 258053 merged by jenkins-bot:
Enable the A/B test to measure impact of collapsing content

https://gerrit.wikimedia.org/r/258053

Jdlrobson closed this task as Resolved.Dec 10 2015, 1:12 AM
Jdlrobson moved this task from Code Review to Ready for Signoff on the Reading Web Sprint 62 - DJ-Jazzy-Jeff-and-the-Fresh-Sprints board.
• Tbayer added a comment.Dec 10 2015, 11:37 AM

PS, one clarification question (because I'm not certain from looking at the code here) :
How does the instrumentation handle repeat appearances of a section, i.e. when it is logged once and then remains visible after another scroll action, or is scrolled out of and into the viewport again? From the perspective of the research question being asked here (whether a section has been seen or not), it might be preferable to only trigger one event, i.e. ignore subsequent appearances. I expect that by relying on the session ID, we can still filter out repetitions afterwards while analyzing the data, but it would require extra effort.

phuedx subscribed.Dec 15 2015, 2:37 PM

@Tbayer:

How does the instrumentation handle repeat appearances of a section, i.e. when it is logged once and then remains visible after another scroll action, or is scrolled out of and into the viewport again?

Looking at the code, we stop tracking the scrolling into view of a section once we have logged the event.

@Jdlrobson: Was this verified during sign off?

Jdlrobson added a comment.Dec 17 2015, 1:25 AM

@phuedx @Tbayer a quick skim through the result of select event_eventName, event_sessionId, event_pageId, event_section from MobileWebSectionUsage_15038458 where event_eventName = 'scrolled-into-view' order by event_sessionId, event_pageId, event_section

I cannot see any events where sectionId is the same for a given event_pageId and event_sessionId
Not sure if there is an sql query to reinforce this manual check.

• Tbayer added a comment.Dec 17 2015, 1:39 PM

(The new schema is live now, and @Jdlrobson suggested to do some sanity checks before it is deployed to enwiki too and deployments are frozen. Three such checks are below.)

A. Check that a particular section is only "scrolled-into-view" once for each page and session:

In T120292#1886687, @Jdlrobson wrote:

@phuedx @Tbayer a quick skim through the result of select event_eventName, event_sessionId, event_pageId, event_section from MobileWebSectionUsage_15038458 where event_eventName = 'scrolled-into-view' order by event_sessionId, event_pageId, event_section

I cannot see any events where sectionId is the same for a given event_pageId and event_sessionId
Not sure if there is an sql query to reinforce this manual check.

We can simply group by sectionId, event_pageId, event_sessionId and then use COUNT(*). It turns out that unfortunately multiple events do happen, albeit rarely so - less than 0.4% of 'scrolled-into-view' events are duplicates .[1] Do we have an idea under what circumstances this could occur?

B. Assume that if a section is scrolled into view, all previous sections have been scrolled into view too (probably disregarding some edge cases with some large scroll steps that skip over some sections, e.g. when the user uses "find in page"). Then the frequency of scroll events should be monotonously decreasing by section numbers. This turns out to be almost entirely true[2] - good enough I would say, although I would feel a bit more comfortable if someone familiar with the code can confirm that it's indeed possible to skip over sections (i.e. their "scrolled-into-view" events) when using "find in page" (or scrolling fast).

C. According to the schema description, the isTestA property "is only used with the 'scrolled-into-view' event." That's consistent with the data so far [3]. However, I'm not sure why it is not also attached to "entered" events. How else can we establish a baseline of users who open an article but do not either scroll or open sections? (for queries analogous to e.g. T118041#1847031 )

[1]
mysql:research@analytics-store.eqiad.wmnet [(none)]> SELECT event_sessionId, event_pageId, event_section, COUNT(*) AS freq FROM log.MobileWebSectionUsage_15038458 WHERE event_eventName = 'scrolled-into-view' GROUP BY event_sessionId, event_pageId, event_section ORDER BY freq DESC LIMIT 130;
+------------------+--------------+---------------+------+
| event_sessionId  | event_pageId | event_section | freq |
+------------------+--------------+---------------+------+
| e72205faae7c9957 |       151157 |             0 |    3 |
| 56eb450b928a3235 |      1500013 |             0 |    3 |
| 6262baf35c90f87f |         5705 |             0 |    2 |
| d26d287500a9f7a9 |         1920 |             0 |    2 |
| 2e98907e0a1c8522 |         6919 |            10 |    2 |
| 7173d772c558f2d2 |      1033972 |             4 |    2 |
| 1998baa0cb14da1e |        15311 |             9 |    2 |
| 50f97eaf59ce133f |     10392268 |             0 |    2 |
| c9b68662b8a5ee30 |       293160 |             6 |    2 |
| eba716b139389ae6 |      5212996 |             1 |    2 |
| 0ff4dcf49244041d |       423256 |             0 |    2 |
| 271cbb8cc93e5783 |        18163 |             0 |    2 |
| e6b1c97656543fe0 |         2853 |            40 |    2 |
| b9f8cdcd1f511223 |         7984 |             5 |    2 |
| 6262baf35c90f87f |         5705 |             2 |    2 |
| 1998baa0cb14da1e |        15311 |             6 |    2 |
| 54ca597d4b521233 |        39359 |             1 |    2 |
| 1998baa0cb14da1e |        15311 |             4 |    2 |
| 4ad25424a5eb8424 |      2049466 |             0 |    2 |
| e21c5cccf9c2d3c5 |         3694 |             2 |    2 |
| d189e284ea1800ab |         2147 |             1 |    2 |
| 1998baa0cb14da1e |        15311 |             2 |    2 |
| b1ffe5fa888e8f2b |        10302 |             2 |    2 |
| 8dee885064f71f57 |       147165 |             6 |    2 |
| 4ad25424a5eb8424 |      2049466 |             1 |    2 |
| 271cbb8cc93e5783 |        18163 |             4 |    2 |
| c9b68662b8a5ee30 |       293160 |             8 |    2 |
| b9f8cdcd1f511223 |         7984 |             2 |    2 |
| 8dee885064f71f57 |       147165 |             5 |    2 |
| 7173d772c558f2d2 |      1033972 |             2 |    2 |
| b9f8cdcd1f511223 |         7984 |             4 |    2 |
| b9f8cdcd1f511223 |         7984 |             3 |    2 |
| 4ad25424a5eb8424 |      2049466 |             2 |    2 |
| 1998baa0cb14da1e |        15311 |             0 |    2 |
| 814991359a0c8534 |          933 |             4 |    2 |
| c9b68662b8a5ee30 |       293160 |             7 |    2 |
| 77a6015114896d0e |        41453 |             1 |    2 |
| 7173d772c558f2d2 |      1033972 |             0 |    2 |
| 1998baa0cb14da1e |        15311 |            15 |    2 |
| bf3dd6cf97d03061 |       256977 |             0 |    2 |
| 1998baa0cb14da1e |        15311 |             1 |    2 |
| 814991359a0c8534 |          933 |             3 |    2 |
| b1ffe5fa888e8f2b |        10302 |             6 |    2 |
| c9b68662b8a5ee30 |       293160 |             1 |    2 |
| bdf93f98cc22af78 |        81148 |             1 |    2 |
| 8dee885064f71f57 |       147165 |             1 |    2 |
| 271cbb8cc93e5783 |        18163 |             2 |    2 |
| d189e284ea1800ab |         2147 |             0 |    2 |
| e21c5cccf9c2d3c5 |         3694 |             0 |    2 |
| 5fbd1057001c0631 |          164 |             4 |    2 |
| 8dee885064f71f57 |       147165 |             7 |    2 |
| b9f8cdcd1f511223 |         7984 |             0 |    2 |
| 4ad25424a5eb8424 |      2049466 |             5 |    2 |
| 54ca597d4b521233 |        39359 |             2 |    2 |
| 40d1461cdb87e1d8 |       248582 |             5 |    2 |
| 40d1461cdb87e1d8 |       248582 |            10 |    2 |
| 77a6015114896d0e |        41453 |             2 |    2 |
| b9f8cdcd1f511223 |         7984 |             1 |    2 |
| 1998baa0cb14da1e |        15311 |            14 |    2 |
| bdf93f98cc22af78 |        81148 |             0 |    2 |
| 0ff4dcf49244041d |       423256 |             1 |    2 |
| ecd1c4eec96258d3 |       315185 |             1 |    2 |
| 7ee741748ba35cbf |        15258 |             0 |    2 |
| 40d1461cdb87e1d8 |       248582 |             2 |    2 |
| 271cbb8cc93e5783 |        18163 |             3 |    2 |
| eba716b139389ae6 |      5212996 |             2 |    2 |
| b1ffe5fa888e8f2b |        10302 |             5 |    2 |
| 090be1476196019d |        38504 |             0 |    2 |
| c9b68662b8a5ee30 |       293160 |             3 |    2 |
| 814991359a0c8534 |          933 |             1 |    2 |
| 5fbd1057001c0631 |          164 |             3 |    2 |
| 1998baa0cb14da1e |        15311 |            13 |    2 |
| 814991359a0c8534 |          933 |             2 |    2 |
| b1ffe5fa888e8f2b |        10302 |             1 |    2 |
| 96f99f1b125a8fb8 |       699211 |             1 |    2 |
| b1ffe5fa888e8f2b |        10302 |             3 |    2 |
| 814991359a0c8534 |          933 |             5 |    2 |
| 1998baa0cb14da1e |        15311 |             7 |    2 |
| 96f99f1b125a8fb8 |       699211 |             0 |    2 |
| 3138f8cb75a6e8d4 |        46148 |             0 |    2 |
| 1edf1eb23d2a3adc |         9782 |             0 |    2 |
| 5fbd1057001c0631 |          164 |             2 |    2 |
| e677d56a4a837f58 |       188080 |             0 |    2 |
| 8dee885064f71f57 |       147165 |             0 |    2 |
| 40d1461cdb87e1d8 |       248582 |             1 |    2 |
| 1998baa0cb14da1e |        15311 |            11 |    2 |
| 7173d772c558f2d2 |      1033972 |             3 |    2 |
| 6262baf35c90f87f |         5705 |             3 |    2 |
| 814991359a0c8534 |          933 |             0 |    2 |
| 8709bfc58ec18a4f |       536841 |             0 |    2 |
| 7173d772c558f2d2 |      1033972 |             1 |    2 |
| 4b52021a9e3b8235 |       219094 |             0 |    2 |
| c9b68662b8a5ee30 |       293160 |             0 |    2 |
| 1998baa0cb14da1e |        15311 |             3 |    2 |
| 1998baa0cb14da1e |        15311 |             8 |    2 |
| b1ffe5fa888e8f2b |        10302 |             0 |    2 |
| 5fbd1057001c0631 |          164 |             0 |    2 |
| e677d56a4a837f58 |       188080 |             1 |    2 |
| 1998baa0cb14da1e |        15311 |            12 |    2 |
| d1a1d7e1954f2cd7 |        22550 |             0 |    2 |
| bdf93f98cc22af78 |        81148 |             2 |    2 |
| c9b68662b8a5ee30 |       293160 |             5 |    2 |
| 5fbd1057001c0631 |          164 |             5 |    2 |
| 271cbb8cc93e5783 |        18163 |             1 |    2 |
| 69195465728d3caf |         4519 |             0 |    2 |
| c9b68662b8a5ee30 |       293160 |             4 |    2 |
| acbbe484df37f389 |        68279 |             2 |    2 |
| c9b68662b8a5ee30 |       293160 |             2 |    2 |
| 4ad25424a5eb8424 |      2049466 |             3 |    2 |
| 5fbd1057001c0631 |          164 |             1 |    2 |
| 6db95735d51b1c40 |         4532 |             0 |    2 |
| 65e0c93bcd3027fc |         3639 |             0 |    2 |
| 96f99f1b125a8fb8 |       699211 |             2 |    2 |
| 6262baf35c90f87f |         5705 |             1 |    2 |
| 1998baa0cb14da1e |        15311 |            10 |    2 |
| eba716b139389ae6 |      5212996 |             0 |    2 |
| b1ffe5fa888e8f2b |        10302 |             4 |    2 |
| 54ca597d4b521233 |        39359 |             0 |    2 |
| 4ad25424a5eb8424 |      2049466 |             4 |    2 |
| e21c5cccf9c2d3c5 |         3694 |             1 |    2 |
| 77a6015114896d0e |        41453 |             0 |    2 |
| 1998baa0cb14da1e |        15311 |             5 |    2 |
| d3cb29421a842341 |       202067 |             5 |    1 |
| 95165247c6242d10 |       535420 |             0 |    1 |
| b16d56036272c6d9 |         4974 |             8 |    1 |
| 04353f07a25c87f1 |         2584 |             3 |    1 |
| de3b8e51f7107278 |          614 |             5 |    1 |
| c289d2e58870ea2a |      1049769 |             0 |    1 |
| a915aab4e40fbfc7 |       300292 |             7 |    1 |
| 71c15b51f8f1b0d0 |      2623634 |             0 |    1 |
+------------------+--------------+---------------+------+
130 rows in set (0.06 sec)

mysql:research@analytics-store.eqiad.wmnet [(none)]> SELECT COUNT(*) FROM log.MobileWebSectionUsage_15038458 WHERE event_eventName = 'scrolled-into-view';
+----------+
| COUNT(*) |
+----------+
|    32044 |
+----------+
1 row in set (0.09 sec)

124 / 32 044 = 0.00386...
[2]
mysql:research@analytics-store.eqiad.wmnet [(none)]> SELECT event_section, COUNT(*) FROM  log.MobileWebSectionUsage_15038458 WHERE event_eventName = 'scrolled-into-view' GROUP BY event_section ORDER BY event_section;
+---------------+----------+
| event_section | COUNT(*) |
+---------------+----------+
|             0 |     8297 |
|             1 |     3885 |
|             2 |     3052 |
|             3 |     2292 |
|             4 |     1651 |
|             5 |     1115 |
|             6 |      719 |
|             7 |      511 |
|             8 |      377 |
|             9 |      263 |
|            10 |      227 |
|            11 |      174 |
|            12 |      132 |
|            13 |      115 |
|            14 |      112 |
|            15 |      108 |
|            16 |       88 |
|            17 |       69 |
|            18 |       62 |
|            19 |       52 |
|            20 |       40 |
|            21 |       37 |
|            22 |       32 |
|            23 |       29 |
|            24 |       28 |
|            25 |       27 |
|            26 |       24 |
|            27 |       19 |
|            28 |       17 |
|            29 |       11 |
|            30 |        9 |
|            31 |       10 |
|            32 |        9 |
|            33 |        7 |
|            34 |        8 |
|            35 |        8 |
|            36 |        8 |
|            37 |        9 |
|            38 |       10 |
|            39 |        9 |
|            40 |        9 |
|            41 |        8 |
|            42 |        7 |
|            43 |        7 |
|            44 |        5 |
|            45 |        6 |
|            46 |        6 |
|            47 |        6 |
|            48 |        5 |
|            49 |        6 |
|            50 |        6 |
|            51 |        5 |
|            52 |        4 |
|            53 |        4 |
|            54 |        4 |
|            55 |        3 |
|            56 |        3 |
|            57 |        4 |
|            58 |        3 |
|            59 |        4 |
|            60 |        3 |
|            61 |        3 |
|            62 |        3 |
|            63 |        4 |
|            64 |        4 |
|            65 |        3 |
|            66 |        3 |
|            67 |        2 |
|            68 |        2 |
|            69 |        3 |
|            70 |        3 |
|            71 |        3 |
|            72 |        2 |
|            73 |        2 |
|            74 |        3 |
|            75 |        3 |
|            76 |        3 |
|            77 |        2 |
|            78 |        2 |
|            79 |        3 |
|            80 |        3 |
|            81 |        3 |
|            82 |        2 |
|            83 |        2 |
|            84 |        3 |
|            85 |        3 |
|            86 |        2 |
|            87 |        2 |
|            88 |        2 |
|            89 |        2 |
|            90 |        2 |
|            91 |        2 |
|            92 |        2 |
|            93 |        2 |
|            94 |        1 |
|            95 |        1 |
|            96 |        1 |
|           109 |        1 |
|           123 |        1 |
|           124 |        1 |
|           125 |        1 |
|           126 |        1 |
|           127 |        1 |
|           128 |        1 |
|           129 |        1 |
|           130 |        1 |
|           131 |        1 |
|           132 |        1 |
|           133 |        1 |
|           134 |        1 |
|           135 |        1 |
|           136 |        1 |
|           137 |        1 |
|           138 |        1 |
|           139 |        1 |
|           140 |        1 |
|           141 |        1 |
|           142 |        1 |
|           143 |        1 |
+---------------+----------+
119 rows in set (0.38 sec)
[3]
mysql:research@analytics-store.eqiad.wmnet [(none)]> SELECT event_isTestA, event_eventName FROM log.MobileWebSectionUsage_15038458 WHERE event_eventName <> 'scrolled-into-view' AND event_isTestA <> NULL;
Empty set (0.04 sec)
Jdlrobson added a comment.Dec 17 2015, 4:45 PM

S

In T120292#1887422, @Tbayer wrote:

(The new schema is live now, and @Jdlrobson suggested to do some sanity checks before it is deployed to enwiki too and deployments are frozen. Three such checks are below.)

A. Check that a particular section is only "scrolled-into-view" once for each page and session:

In T120292#1886687, @Jdlrobson wrote:

@phuedx @Tbayer a quick skim through the result of select event_eventName, event_sessionId, event_pageId, event_section from MobileWebSectionUsage_15038458 where event_eventName = 'scrolled-into-view' order by event_sessionId, event_pageId, event_section

I cannot see any events where sectionId is the same for a given event_pageId and event_sessionId
Not sure if there is an sql query to reinforce this manual check.

We can simply group by sectionId, event_pageId, event_sessionId and then use COUNT(*). It turns out that unfortunately multiple events do happen, albeit rarely so - less than 0.4% of 'scrolled-into-view' events are duplicates .[1] Do we have an idea under what circumstances this could occur?

If a page is refreshed or a user navigates back to the same page after clicking a link this is possible.

B. Assume that if a section is scrolled into view, all previous sections have been scrolled into view too (probably disregarding some edge cases with some large scroll steps that skip over some sections, e.g. when the user uses "find in page"). Then the frequency of scroll events should be monotonously decreasing by section numbers. This turns out to be almost entirely true[2] - good enough I would say, although I would feel a bit more comfortable if someone familiar with the code can confirm that it's indeed possible to skip over sections (i.e. their "scrolled-into-view" events) when using "find in page" (or scrolling fast).

C. According to the schema description, the isTestA property "is only used with the 'scrolled-into-view' event." That's consistent with the data so far [3]. However, I'm not sure why it is not also attached to "entered" events. How else can we establish a baseline of users who open an article but do not either scroll or open sections? (for queries analogous to e.g. T118041#1847031 )

Yeh this is not ideal. It's worth noting though sessionId can be used to work out what their bucket was given there is at least one scrolled into view event since isTestA remains the same value throughout.

[1]
mysql:research@analytics-store.eqiad.wmnet [(none)]> SELECT event_sessionId, event_pageId, event_section, COUNT(*) AS freq FROM log.MobileWebSectionUsage_15038458 WHERE event_eventName = 'scrolled-into-view' GROUP BY event_sessionId, event_pageId, event_section ORDER BY freq DESC LIMIT 130;
+------------------+--------------+---------------+------+
| event_sessionId  | event_pageId | event_section | freq |
+------------------+--------------+---------------+------+
| e72205faae7c9957 |       151157 |             0 |    3 |
| 56eb450b928a3235 |      1500013 |             0 |    3 |
| 6262baf35c90f87f |         5705 |             0 |    2 |
| d26d287500a9f7a9 |         1920 |             0 |    2 |
| 2e98907e0a1c8522 |         6919 |            10 |    2 |
| 7173d772c558f2d2 |      1033972 |             4 |    2 |
| 1998baa0cb14da1e |        15311 |             9 |    2 |
| 50f97eaf59ce133f |     10392268 |             0 |    2 |
| c9b68662b8a5ee30 |       293160 |             6 |    2 |
| eba716b139389ae6 |      5212996 |             1 |    2 |
| 0ff4dcf49244041d |       423256 |             0 |    2 |
| 271cbb8cc93e5783 |        18163 |             0 |    2 |
| e6b1c97656543fe0 |         2853 |            40 |    2 |
| b9f8cdcd1f511223 |         7984 |             5 |    2 |
| 6262baf35c90f87f |         5705 |             2 |    2 |
| 1998baa0cb14da1e |        15311 |             6 |    2 |
| 54ca597d4b521233 |        39359 |             1 |    2 |
| 1998baa0cb14da1e |        15311 |             4 |    2 |
| 4ad25424a5eb8424 |      2049466 |             0 |    2 |
| e21c5cccf9c2d3c5 |         3694 |             2 |    2 |
| d189e284ea1800ab |         2147 |             1 |    2 |
| 1998baa0cb14da1e |        15311 |             2 |    2 |
| b1ffe5fa888e8f2b |        10302 |             2 |    2 |
| 8dee885064f71f57 |       147165 |             6 |    2 |
| 4ad25424a5eb8424 |      2049466 |             1 |    2 |
| 271cbb8cc93e5783 |        18163 |             4 |    2 |
| c9b68662b8a5ee30 |       293160 |             8 |    2 |
| b9f8cdcd1f511223 |         7984 |             2 |    2 |
| 8dee885064f71f57 |       147165 |             5 |    2 |
| 7173d772c558f2d2 |      1033972 |             2 |    2 |
| b9f8cdcd1f511223 |         7984 |             4 |    2 |
| b9f8cdcd1f511223 |         7984 |             3 |    2 |
| 4ad25424a5eb8424 |      2049466 |             2 |    2 |
| 1998baa0cb14da1e |        15311 |             0 |    2 |
| 814991359a0c8534 |          933 |             4 |    2 |
| c9b68662b8a5ee30 |       293160 |             7 |    2 |
| 77a6015114896d0e |        41453 |             1 |    2 |
| 7173d772c558f2d2 |      1033972 |             0 |    2 |
| 1998baa0cb14da1e |        15311 |            15 |    2 |
| bf3dd6cf97d03061 |       256977 |             0 |    2 |
| 1998baa0cb14da1e |        15311 |             1 |    2 |
| 814991359a0c8534 |          933 |             3 |    2 |
| b1ffe5fa888e8f2b |        10302 |             6 |    2 |
| c9b68662b8a5ee30 |       293160 |             1 |    2 |
| bdf93f98cc22af78 |        81148 |             1 |    2 |
| 8dee885064f71f57 |       147165 |             1 |    2 |
| 271cbb8cc93e5783 |        18163 |             2 |    2 |
| d189e284ea1800ab |         2147 |             0 |    2 |
| e21c5cccf9c2d3c5 |         3694 |             0 |    2 |
| 5fbd1057001c0631 |          164 |             4 |    2 |
| 8dee885064f71f57 |       147165 |             7 |    2 |
| b9f8cdcd1f511223 |         7984 |             0 |    2 |
| 4ad25424a5eb8424 |      2049466 |             5 |    2 |
| 54ca597d4b521233 |        39359 |             2 |    2 |
| 40d1461cdb87e1d8 |       248582 |             5 |    2 |
| 40d1461cdb87e1d8 |       248582 |            10 |    2 |
| 77a6015114896d0e |        41453 |             2 |    2 |
| b9f8cdcd1f511223 |         7984 |             1 |    2 |
| 1998baa0cb14da1e |        15311 |            14 |    2 |
| bdf93f98cc22af78 |        81148 |             0 |    2 |
| 0ff4dcf49244041d |       423256 |             1 |    2 |
| ecd1c4eec96258d3 |       315185 |             1 |    2 |
| 7ee741748ba35cbf |        15258 |             0 |    2 |
| 40d1461cdb87e1d8 |       248582 |             2 |    2 |
| 271cbb8cc93e5783 |        18163 |             3 |    2 |
| eba716b139389ae6 |      5212996 |             2 |    2 |
| b1ffe5fa888e8f2b |        10302 |             5 |    2 |
| 090be1476196019d |        38504 |             0 |    2 |
| c9b68662b8a5ee30 |       293160 |             3 |    2 |
| 814991359a0c8534 |          933 |             1 |    2 |
| 5fbd1057001c0631 |          164 |             3 |    2 |
| 1998baa0cb14da1e |        15311 |            13 |    2 |
| 814991359a0c8534 |          933 |             2 |    2 |
| b1ffe5fa888e8f2b |        10302 |             1 |    2 |
| 96f99f1b125a8fb8 |       699211 |             1 |    2 |
| b1ffe5fa888e8f2b |        10302 |             3 |    2 |
| 814991359a0c8534 |          933 |             5 |    2 |
| 1998baa0cb14da1e |        15311 |             7 |    2 |
| 96f99f1b125a8fb8 |       699211 |             0 |    2 |
| 3138f8cb75a6e8d4 |        46148 |             0 |    2 |
| 1edf1eb23d2a3adc |         9782 |             0 |    2 |
| 5fbd1057001c0631 |          164 |             2 |    2 |
| e677d56a4a837f58 |       188080 |             0 |    2 |
| 8dee885064f71f57 |       147165 |             0 |    2 |
| 40d1461cdb87e1d8 |       248582 |             1 |    2 |
| 1998baa0cb14da1e |        15311 |            11 |    2 |
| 7173d772c558f2d2 |      1033972 |             3 |    2 |
| 6262baf35c90f87f |         5705 |             3 |    2 |
| 814991359a0c8534 |          933 |             0 |    2 |
| 8709bfc58ec18a4f |       536841 |             0 |    2 |
| 7173d772c558f2d2 |      1033972 |             1 |    2 |
| 4b52021a9e3b8235 |       219094 |             0 |    2 |
| c9b68662b8a5ee30 |       293160 |             0 |    2 |
| 1998baa0cb14da1e |        15311 |             3 |    2 |
| 1998baa0cb14da1e |        15311 |             8 |    2 |
| b1ffe5fa888e8f2b |        10302 |             0 |    2 |
| 5fbd1057001c0631 |          164 |             0 |    2 |
| e677d56a4a837f58 |       188080 |             1 |    2 |
| 1998baa0cb14da1e |        15311 |            12 |    2 |
| d1a1d7e1954f2cd7 |        22550 |             0 |    2 |
| bdf93f98cc22af78 |        81148 |             2 |    2 |
| c9b68662b8a5ee30 |       293160 |             5 |    2 |
| 5fbd1057001c0631 |          164 |             5 |    2 |
| 271cbb8cc93e5783 |        18163 |             1 |    2 |
| 69195465728d3caf |         4519 |             0 |    2 |
| c9b68662b8a5ee30 |       293160 |             4 |    2 |
| acbbe484df37f389 |        68279 |             2 |    2 |
| c9b68662b8a5ee30 |       293160 |             2 |    2 |
| 4ad25424a5eb8424 |      2049466 |             3 |    2 |
| 5fbd1057001c0631 |          164 |             1 |    2 |
| 6db95735d51b1c40 |         4532 |             0 |    2 |
| 65e0c93bcd3027fc |         3639 |             0 |    2 |
| 96f99f1b125a8fb8 |       699211 |             2 |    2 |
| 6262baf35c90f87f |         5705 |             1 |    2 |
| 1998baa0cb14da1e |        15311 |            10 |    2 |
| eba716b139389ae6 |      5212996 |             0 |    2 |
| b1ffe5fa888e8f2b |        10302 |             4 |    2 |
| 54ca597d4b521233 |        39359 |             0 |    2 |
| 4ad25424a5eb8424 |      2049466 |             4 |    2 |
| e21c5cccf9c2d3c5 |         3694 |             1 |    2 |
| 77a6015114896d0e |        41453 |             0 |    2 |
| 1998baa0cb14da1e |        15311 |             5 |    2 |
| d3cb29421a842341 |       202067 |             5 |    1 |
| 95165247c6242d10 |       535420 |             0 |    1 |
| b16d56036272c6d9 |         4974 |             8 |    1 |
| 04353f07a25c87f1 |         2584 |             3 |    1 |
| de3b8e51f7107278 |          614 |             5 |    1 |
| c289d2e58870ea2a |      1049769 |             0 |    1 |
| a915aab4e40fbfc7 |       300292 |             7 |    1 |
| 71c15b51f8f1b0d0 |      2623634 |             0 |    1 |
+------------------+--------------+---------------+------+
130 rows in set (0.06 sec)

mysql:research@analytics-store.eqiad.wmnet [(none)]> SELECT COUNT(*) FROM log.MobileWebSectionUsage_15038458 WHERE event_eventName = 'scrolled-into-view';
+----------+
| COUNT(*) |
+----------+
|    32044 |
+----------+
1 row in set (0.09 sec)

124 / 32 044 = 0.00386...
[2]
mysql:research@analytics-store.eqiad.wmnet [(none)]> SELECT event_section, COUNT(*) FROM  log.MobileWebSectionUsage_15038458 WHERE event_eventName = 'scrolled-into-view' GROUP BY event_section ORDER BY event_section;
+---------------+----------+
| event_section | COUNT(*) |
+---------------+----------+
|             0 |     8297 |
|             1 |     3885 |
|             2 |     3052 |
|             3 |     2292 |
|             4 |     1651 |
|             5 |     1115 |
|             6 |      719 |
|             7 |      511 |
|             8 |      377 |
|             9 |      263 |
|            10 |      227 |
|            11 |      174 |
|            12 |      132 |
|            13 |      115 |
|            14 |      112 |
|            15 |      108 |
|            16 |       88 |
|            17 |       69 |
|            18 |       62 |
|            19 |       52 |
|            20 |       40 |
|            21 |       37 |
|            22 |       32 |
|            23 |       29 |
|            24 |       28 |
|            25 |       27 |
|            26 |       24 |
|            27 |       19 |
|            28 |       17 |
|            29 |       11 |
|            30 |        9 |
|            31 |       10 |
|            32 |        9 |
|            33 |        7 |
|            34 |        8 |
|            35 |        8 |
|            36 |        8 |
|            37 |        9 |
|            38 |       10 |
|            39 |        9 |
|            40 |        9 |
|            41 |        8 |
|            42 |        7 |
|            43 |        7 |
|            44 |        5 |
|            45 |        6 |
|            46 |        6 |
|            47 |        6 |
|            48 |        5 |
|            49 |        6 |
|            50 |        6 |
|            51 |        5 |
|            52 |        4 |
|            53 |        4 |
|            54 |        4 |
|            55 |        3 |
|            56 |        3 |
|            57 |        4 |
|            58 |        3 |
|            59 |        4 |
|            60 |        3 |
|            61 |        3 |
|            62 |        3 |
|            63 |        4 |
|            64 |        4 |
|            65 |        3 |
|            66 |        3 |
|            67 |        2 |
|            68 |        2 |
|            69 |        3 |
|            70 |        3 |
|            71 |        3 |
|            72 |        2 |
|            73 |        2 |
|            74 |        3 |
|            75 |        3 |
|            76 |        3 |
|            77 |        2 |
|            78 |        2 |
|            79 |        3 |
|            80 |        3 |
|            81 |        3 |
|            82 |        2 |
|            83 |        2 |
|            84 |        3 |
|            85 |        3 |
|            86 |        2 |
|            87 |        2 |
|            88 |        2 |
|            89 |        2 |
|            90 |        2 |
|            91 |        2 |
|            92 |        2 |
|            93 |        2 |
|            94 |        1 |
|            95 |        1 |
|            96 |        1 |
|           109 |        1 |
|           123 |        1 |
|           124 |        1 |
|           125 |        1 |
|           126 |        1 |
|           127 |        1 |
|           128 |        1 |
|           129 |        1 |
|           130 |        1 |
|           131 |        1 |
|           132 |        1 |
|           133 |        1 |
|           134 |        1 |
|           135 |        1 |
|           136 |        1 |
|           137 |        1 |
|           138 |        1 |
|           139 |        1 |
|           140 |        1 |
|           141 |        1 |
|           142 |        1 |
|           143 |        1 |
+---------------+----------+
119 rows in set (0.38 sec)
[3]
mysql:research@analytics-store.eqiad.wmnet [(none)]> SELECT event_isTestA, event_eventName FROM log.MobileWebSectionUsage_15038458 WHERE event_eventName <> 'scrolled-into-view' AND event_isTestA <> NULL;
Empty set (0.04 sec)
Jdlrobson added a comment.Dec 17 2015, 5:46 PM
In T120292#1887422, @Tbayer wrote:

(The new schema is live now, and @Jdlrobson suggested to do some sanity checks before it is deployed to enwiki too and deployments are frozen. Three such checks are below.)

A. Check that a particular section is only "scrolled-into-view" once for each page and session:

In T120292#1886687, @Jdlrobson wrote:

@phuedx @Tbayer a quick skim through the result of select event_eventName, event_sessionId, event_pageId, event_section from MobileWebSectionUsage_15038458 where event_eventName = 'scrolled-into-view' order by event_sessionId, event_pageId, event_section

I cannot see any events where sectionId is the same for a given event_pageId and event_sessionId
Not sure if there is an sql query to reinforce this manual check.

We can simply group by sectionId, event_pageId, event_sessionId and then use COUNT(*). It turns out that unfortunately multiple events do happen, albeit rarely so - less than 0.4% of 'scrolled-into-view' events are duplicates .[1] Do we have an idea under what circumstances this could occur?

B. Assume that if a section is scrolled into view, all previous sections have been scrolled into view too (probably disregarding some edge cases with some large scroll steps that skip over some sections, e.g. when the user uses "find in page"). Then the frequency of scroll events should be monotonously decreasing by section numbers. This turns out to be almost entirely true[2] - good enough I would say, although I would feel a bit more comfortable if someone familiar with the code can confirm that it's indeed possible to skip over sections (i.e. their "scrolled-into-view" events) when using "find in page" (or scrolling fast).

If you scroll super fast down the page the events will not fire as the scroll event in this implementation only fires at the end of scroll - so yes you can skip sections. It's only if you stop on the section that the event fires. Of course if someone uses find on page it might mean that they find what they were looking for better but don't ever trigger a scrolled-into-view event.

C. According to the schema description, the isTestA property "is only used with the 'scrolled-into-view' event." That's consistent with the data so far [3]. However, I'm not sure why it is not also attached to "entered" events. How else can we establish a baseline of users who open an article but do not either scroll or open sections? (for queries analogous to e.g. T118041#1847031 )

[1]
mysql:research@analytics-store.eqiad.wmnet [(none)]> SELECT event_sessionId, event_pageId, event_section, COUNT(*) AS freq FROM log.MobileWebSectionUsage_15038458 WHERE event_eventName = 'scrolled-into-view' GROUP BY event_sessionId, event_pageId, event_section ORDER BY freq DESC LIMIT 130;
+------------------+--------------+---------------+------+
| event_sessionId  | event_pageId | event_section | freq |
+------------------+--------------+---------------+------+
| e72205faae7c9957 |       151157 |             0 |    3 |
| 56eb450b928a3235 |      1500013 |             0 |    3 |
| 6262baf35c90f87f |         5705 |             0 |    2 |
| d26d287500a9f7a9 |         1920 |             0 |    2 |
| 2e98907e0a1c8522 |         6919 |            10 |    2 |
| 7173d772c558f2d2 |      1033972 |             4 |    2 |
| 1998baa0cb14da1e |        15311 |             9 |    2 |
| 50f97eaf59ce133f |     10392268 |             0 |    2 |
| c9b68662b8a5ee30 |       293160 |             6 |    2 |
| eba716b139389ae6 |      5212996 |             1 |    2 |
| 0ff4dcf49244041d |       423256 |             0 |    2 |
| 271cbb8cc93e5783 |        18163 |             0 |    2 |
| e6b1c97656543fe0 |         2853 |            40 |    2 |
| b9f8cdcd1f511223 |         7984 |             5 |    2 |
| 6262baf35c90f87f |         5705 |             2 |    2 |
| 1998baa0cb14da1e |        15311 |             6 |    2 |
| 54ca597d4b521233 |        39359 |             1 |    2 |
| 1998baa0cb14da1e |        15311 |             4 |    2 |
| 4ad25424a5eb8424 |      2049466 |             0 |    2 |
| e21c5cccf9c2d3c5 |         3694 |             2 |    2 |
| d189e284ea1800ab |         2147 |             1 |    2 |
| 1998baa0cb14da1e |        15311 |             2 |    2 |
| b1ffe5fa888e8f2b |        10302 |             2 |    2 |
| 8dee885064f71f57 |       147165 |             6 |    2 |
| 4ad25424a5eb8424 |      2049466 |             1 |    2 |
| 271cbb8cc93e5783 |        18163 |             4 |    2 |
| c9b68662b8a5ee30 |       293160 |             8 |    2 |
| b9f8cdcd1f511223 |         7984 |             2 |    2 |
| 8dee885064f71f57 |       147165 |             5 |    2 |
| 7173d772c558f2d2 |      1033972 |             2 |    2 |
| b9f8cdcd1f511223 |         7984 |             4 |    2 |
| b9f8cdcd1f511223 |         7984 |             3 |    2 |
| 4ad25424a5eb8424 |      2049466 |             2 |    2 |
| 1998baa0cb14da1e |        15311 |             0 |    2 |
| 814991359a0c8534 |          933 |             4 |    2 |
| c9b68662b8a5ee30 |       293160 |             7 |    2 |
| 77a6015114896d0e |        41453 |             1 |    2 |
| 7173d772c558f2d2 |      1033972 |             0 |    2 |
| 1998baa0cb14da1e |        15311 |            15 |    2 |
| bf3dd6cf97d03061 |       256977 |             0 |    2 |
| 1998baa0cb14da1e |        15311 |             1 |    2 |
| 814991359a0c8534 |          933 |             3 |    2 |
| b1ffe5fa888e8f2b |        10302 |             6 |    2 |
| c9b68662b8a5ee30 |       293160 |             1 |    2 |
| bdf93f98cc22af78 |        81148 |             1 |    2 |
| 8dee885064f71f57 |       147165 |             1 |    2 |
| 271cbb8cc93e5783 |        18163 |             2 |    2 |
| d189e284ea1800ab |         2147 |             0 |    2 |
| e21c5cccf9c2d3c5 |         3694 |             0 |    2 |
| 5fbd1057001c0631 |          164 |             4 |    2 |
| 8dee885064f71f57 |       147165 |             7 |    2 |
| b9f8cdcd1f511223 |         7984 |             0 |    2 |
| 4ad25424a5eb8424 |      2049466 |             5 |    2 |
| 54ca597d4b521233 |        39359 |             2 |    2 |
| 40d1461cdb87e1d8 |       248582 |             5 |    2 |
| 40d1461cdb87e1d8 |       248582 |            10 |    2 |
| 77a6015114896d0e |        41453 |             2 |    2 |
| b9f8cdcd1f511223 |         7984 |             1 |    2 |
| 1998baa0cb14da1e |        15311 |            14 |    2 |
| bdf93f98cc22af78 |        81148 |             0 |    2 |
| 0ff4dcf49244041d |       423256 |             1 |    2 |
| ecd1c4eec96258d3 |       315185 |             1 |    2 |
| 7ee741748ba35cbf |        15258 |             0 |    2 |
| 40d1461cdb87e1d8 |       248582 |             2 |    2 |
| 271cbb8cc93e5783 |        18163 |             3 |    2 |
| eba716b139389ae6 |      5212996 |             2 |    2 |
| b1ffe5fa888e8f2b |        10302 |             5 |    2 |
| 090be1476196019d |        38504 |             0 |    2 |
| c9b68662b8a5ee30 |       293160 |             3 |    2 |
| 814991359a0c8534 |          933 |             1 |    2 |
| 5fbd1057001c0631 |          164 |             3 |    2 |
| 1998baa0cb14da1e |        15311 |            13 |    2 |
| 814991359a0c8534 |          933 |             2 |    2 |
| b1ffe5fa888e8f2b |        10302 |             1 |    2 |
| 96f99f1b125a8fb8 |       699211 |             1 |    2 |
| b1ffe5fa888e8f2b |        10302 |             3 |    2 |
| 814991359a0c8534 |          933 |             5 |    2 |
| 1998baa0cb14da1e |        15311 |             7 |    2 |
| 96f99f1b125a8fb8 |       699211 |             0 |    2 |
| 3138f8cb75a6e8d4 |        46148 |             0 |    2 |
| 1edf1eb23d2a3adc |         9782 |             0 |    2 |
| 5fbd1057001c0631 |          164 |             2 |    2 |
| e677d56a4a837f58 |       188080 |             0 |    2 |
| 8dee885064f71f57 |       147165 |             0 |    2 |
| 40d1461cdb87e1d8 |       248582 |             1 |    2 |
| 1998baa0cb14da1e |        15311 |            11 |    2 |
| 7173d772c558f2d2 |      1033972 |             3 |    2 |
| 6262baf35c90f87f |         5705 |             3 |    2 |
| 814991359a0c8534 |          933 |             0 |    2 |
| 8709bfc58ec18a4f |       536841 |             0 |    2 |
| 7173d772c558f2d2 |      1033972 |             1 |    2 |
| 4b52021a9e3b8235 |       219094 |             0 |    2 |
| c9b68662b8a5ee30 |       293160 |             0 |    2 |
| 1998baa0cb14da1e |        15311 |             3 |    2 |
| 1998baa0cb14da1e |        15311 |             8 |    2 |
| b1ffe5fa888e8f2b |        10302 |             0 |    2 |
| 5fbd1057001c0631 |          164 |             0 |    2 |
| e677d56a4a837f58 |       188080 |             1 |    2 |
| 1998baa0cb14da1e |        15311 |            12 |    2 |
| d1a1d7e1954f2cd7 |        22550 |             0 |    2 |
| bdf93f98cc22af78 |        81148 |             2 |    2 |
| c9b68662b8a5ee30 |       293160 |             5 |    2 |
| 5fbd1057001c0631 |          164 |             5 |    2 |
| 271cbb8cc93e5783 |        18163 |             1 |    2 |
| 69195465728d3caf |         4519 |             0 |    2 |
| c9b68662b8a5ee30 |       293160 |             4 |    2 |
| acbbe484df37f389 |        68279 |             2 |    2 |
| c9b68662b8a5ee30 |       293160 |             2 |    2 |
| 4ad25424a5eb8424 |      2049466 |             3 |    2 |
| 5fbd1057001c0631 |          164 |             1 |    2 |
| 6db95735d51b1c40 |         4532 |             0 |    2 |
| 65e0c93bcd3027fc |         3639 |             0 |    2 |
| 96f99f1b125a8fb8 |       699211 |             2 |    2 |
| 6262baf35c90f87f |         5705 |             1 |    2 |
| 1998baa0cb14da1e |        15311 |            10 |    2 |
| eba716b139389ae6 |      5212996 |             0 |    2 |
| b1ffe5fa888e8f2b |        10302 |             4 |    2 |
| 54ca597d4b521233 |        39359 |             0 |    2 |
| 4ad25424a5eb8424 |      2049466 |             4 |    2 |
| e21c5cccf9c2d3c5 |         3694 |             1 |    2 |
| 77a6015114896d0e |        41453 |             0 |    2 |
| 1998baa0cb14da1e |        15311 |             5 |    2 |
| d3cb29421a842341 |       202067 |             5 |    1 |
| 95165247c6242d10 |       535420 |             0 |    1 |
| b16d56036272c6d9 |         4974 |             8 |    1 |
| 04353f07a25c87f1 |         2584 |             3 |    1 |
| de3b8e51f7107278 |          614 |             5 |    1 |
| c289d2e58870ea2a |      1049769 |             0 |    1 |
| a915aab4e40fbfc7 |       300292 |             7 |    1 |
| 71c15b51f8f1b0d0 |      2623634 |             0 |    1 |
+------------------+--------------+---------------+------+
130 rows in set (0.06 sec)

mysql:research@analytics-store.eqiad.wmnet [(none)]> SELECT COUNT(*) FROM log.MobileWebSectionUsage_15038458 WHERE event_eventName = 'scrolled-into-view';
+----------+
| COUNT(*) |
+----------+
|    32044 |
+----------+
1 row in set (0.09 sec)

124 / 32 044 = 0.00386...
[2]
mysql:research@analytics-store.eqiad.wmnet [(none)]> SELECT event_section, COUNT(*) FROM  log.MobileWebSectionUsage_15038458 WHERE event_eventName = 'scrolled-into-view' GROUP BY event_section ORDER BY event_section;
+---------------+----------+
| event_section | COUNT(*) |
+---------------+----------+
|             0 |     8297 |
|             1 |     3885 |
|             2 |     3052 |
|             3 |     2292 |
|             4 |     1651 |
|             5 |     1115 |
|             6 |      719 |
|             7 |      511 |
|             8 |      377 |
|             9 |      263 |
|            10 |      227 |
|            11 |      174 |
|            12 |      132 |
|            13 |      115 |
|            14 |      112 |
|            15 |      108 |
|            16 |       88 |
|            17 |       69 |
|            18 |       62 |
|            19 |       52 |
|            20 |       40 |
|            21 |       37 |
|            22 |       32 |
|            23 |       29 |
|            24 |       28 |
|            25 |       27 |
|            26 |       24 |
|            27 |       19 |
|            28 |       17 |
|            29 |       11 |
|            30 |        9 |
|            31 |       10 |
|            32 |        9 |
|            33 |        7 |
|            34 |        8 |
|            35 |        8 |
|            36 |        8 |
|            37 |        9 |
|            38 |       10 |
|            39 |        9 |
|            40 |        9 |
|            41 |        8 |
|            42 |        7 |
|            43 |        7 |
|            44 |        5 |
|            45 |        6 |
|            46 |        6 |
|            47 |        6 |
|            48 |        5 |
|            49 |        6 |
|            50 |        6 |
|            51 |        5 |
|            52 |        4 |
|            53 |        4 |
|            54 |        4 |
|            55 |        3 |
|            56 |        3 |
|            57 |        4 |
|            58 |        3 |
|            59 |        4 |
|            60 |        3 |
|            61 |        3 |
|            62 |        3 |
|            63 |        4 |
|            64 |        4 |
|            65 |        3 |
|            66 |        3 |
|            67 |        2 |
|            68 |        2 |
|            69 |        3 |
|            70 |        3 |
|            71 |        3 |
|            72 |        2 |
|            73 |        2 |
|            74 |        3 |
|            75 |        3 |
|            76 |        3 |
|            77 |        2 |
|            78 |        2 |
|            79 |        3 |
|            80 |        3 |
|            81 |        3 |
|            82 |        2 |
|            83 |        2 |
|            84 |        3 |
|            85 |        3 |
|            86 |        2 |
|            87 |        2 |
|            88 |        2 |
|            89 |        2 |
|            90 |        2 |
|            91 |        2 |
|            92 |        2 |
|            93 |        2 |
|            94 |        1 |
|            95 |        1 |
|            96 |        1 |
|           109 |        1 |
|           123 |        1 |
|           124 |        1 |
|           125 |        1 |
|           126 |        1 |
|           127 |        1 |
|           128 |        1 |
|           129 |        1 |
|           130 |        1 |
|           131 |        1 |
|           132 |        1 |
|           133 |        1 |
|           134 |        1 |
|           135 |        1 |
|           136 |        1 |
|           137 |        1 |
|           138 |        1 |
|           139 |        1 |
|           140 |        1 |
|           141 |        1 |
|           142 |        1 |
|           143 |        1 |
+---------------+----------+
119 rows in set (0.38 sec)
[3]
mysql:research@analytics-store.eqiad.wmnet [(none)]> SELECT event_isTestA, event_eventName FROM log.MobileWebSectionUsage_15038458 WHERE event_eventName <> 'scrolled-into-view' AND event_isTestA <> NULL;
Empty set (0.04 sec)
Jdlrobson closed subtask T121772: isTestA should always be logged with MobileWebSectionUsage schema as Resolved.Dec 18 2015, 12:26 AM
Jdlrobson moved this task from Ready for Signoff to Done on the Reading Web Sprint 62 - DJ-Jazzy-Jeff-and-the-Fresh-Sprints board.Dec 18 2015, 1:08 AM

select count(*) from MobileWebSectionUsage_15038458 where event_eventName= 'entered' and event_isTestA is not null

13691

isTestA is now logged for all events.

• Tbayer mentioned this in T108723: Make MobileWebUIClickTracking schema usable (too big).Dec 29 2015, 4:35 AM
Jdlrobson mentioned this in T124220: Close out A/B test for impact of section collapsing.Jan 20 2016, 8:40 PM
• Tbayer mentioned this in T128536: Compare usage of collapsed and uncollapsed section view on mobile web.Mar 1 2016, 11:33 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL