Page MenuHomePhabricator

Initiate Hovercards A/B Test 2: ru and it
Closed, ResolvedPublic1 Story Points

Description

  • Set Event Logging sampling rate on Russian to 1% (in contrast with the global sampling rate of 10%).
  • Using the config variable PopupsExperiment run an A/B test enabled on ru.wikipedia.org such that 10% of anons have Hovercards ON
  • Set Event Logging sampling rate on Italian to 2% (in contrast with the global sampling rate of 10%).
  • Using the config variable PopupsExperiment run an A/B test enabled on ru.wikipedia.org such that 20% of anons have Hovercards ON

Deploy on or after 9/19/2016

  • Once this task is scheduled, file follow on task for data analysis

Details

Related Gerrit Patches:
mediawiki/extensions/Popups : masterEventLogging sampling rate should default to 0
operations/mediawiki-config : masterInitiate Hovercards A/B test on ruwiki and itwiki

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
jhobs updated the task description. (Show Details)Aug 29 2016, 4:37 PM

does anyone know if the translations are in place for the popups extension for Russian?

Total credit to @Elitre for providing this link:

https://translatewiki.net/w/i.php?title=Special:Translate&group=ext-popups&language=ru&filter=%21translated&action=translate

There's the list of all Hovercard untranslated interface strings for Russian. A few missing at the moment.

I can reach out to some folks to ask for translations.

@Elitre - I don't see any in the untrnslated parts - thanks guys!

Elitre added a subscriber: Amire80.Sep 1 2016, 11:06 AM

@Amire80 is the hero here.

No problem, happy to help. I spoke to @ovasileva a few days ago and mentioned translatewiki—so now she got a little demo ;)

Usually I can consult about how translation works, and Russian only happens to be a language in which I'm actually able to write.

@Amire80 - do you know how I can perform the same check for Italian?

Direct link: https://translatewiki.net/w/i.php?title=Special:Translate&group=ext-popups&language=it&filter=%21translated&action=translate

In general, you can select the target language by clicking the language name after that "Translate to:" label at the top.

ovasileva updated the task description. (Show Details)Sep 6 2016, 1:27 PM
ovasileva renamed this task from Initiate Hovercards A/B Test 2 to Initiate Hovercards A/B Test 2: ru and it.Sep 6 2016, 5:01 PM
ovasileva updated the task description. (Show Details)
phuedx added a subscriber: phuedx.Sep 12 2016, 8:58 AM

@ovasileva: One of the outcomes of T141922 is logging all "abandoned" events when the user is in the ON condition. IIRC we don't log these events when the totalInteractionTime is < 250ms. See T141922#2627616 for context.

Can/should this test be blocked on removing that restriction?

Change 310483 had a related patch set uploaded (by Jhobs):
Initiate Hovercards A/B test on ruwiki and itwiki

https://gerrit.wikimedia.org/r/310483

I couldn't find any previously-set EL sampling rates for Popups, so I'd really appreciate eyes on the patch to make sure I set them properly.

Just a note - a/b test shouldn't start until T145379: Record all hover events resolved (even if that's after the 19th)

Using the config variable PopupsExperiment run an A/B test enabled on ru.wikipedia.org such that 10% of anons have Hovercards ON

@jhobs, I don't think our sampling code distinguishes between anons and logged in users. Your patch makes it available for all users. I think we may need to submit a patch to treat anons differently from logged in users. That said, I wonder why anons only can see hovercards? Why not all users, @ovasileva?

Using the config variable PopupsExperiment run an A/B test enabled on ru.wikipedia.org such that 10% of anons have Hovercards ON

@jhobs, I don't think our sampling code distinguishes between anons and logged in users. Your patch makes it available for all users. I think we may need to submit a patch to treat anons differently from logged in users. That said, I wonder why anons only can see hovercards? Why not all users, @ovasileva?

I don't think this was meant to imply that the sampling should only apply to anons, rather, I think it was trying to say that the experiment should not solely apply to logged-in users. @dr0ptp4kt may have to give some insight as the original author of the task.

@bmansurov, @jhobs - a note on this: logged-in users can turn the feature on/off as it is a beta feature. If it's off, we don't know if it's whether they chose so or because they bothered to look at beta features. If they've turned it off on purpose, it doesn't make sense to turn it on for those users again. Also, if they already have it on, would the only change be that we are now collecting data for those users?

Also, if they already have it on, would the only change be that we are now collecting data for those users?

Yes, I think so.

dr0ptp4kt added a comment.EditedSep 15 2016, 4:19 AM

It seems in the patch the $wgPopupsSchemaPopupsSamplingRate is saying there shouldn't be any sampling by default. Is that intentional? I don't know if anyone's querying anything other than for huwiki, so does there at least need to be a 0.1 entry for huwiki?

I can't tell if https://github.com/wikimedia/mediawiki-extensions-Popups/blob/master/Popups.hooks.php#L256 only honors and makes available the value from extension.json's config.SchemaPopupsSamplingRate or if when setting $wgPopupsSchemaPopupsSamplingRate in mediawiki-config PHP that will also in turn be automagically reflected in the JavaScript mw.config pertinent to the given wiki.

ovasileva updated the task description. (Show Details)Sep 15 2016, 12:58 PM

It seems in the patch the $wgPopupsSchemaPopupsSamplingRate is saying there shouldn't be any sampling by default. Is that intentional? I don't know if anyone's querying anything other than for huwiki, so does there at least need to be a 0.1 entry for huwiki?
I can't tell if https://github.com/wikimedia/mediawiki-extensions-Popups/blob/master/Popups.hooks.php#L256 only honors and makes available the value from extension.json's config.SchemaPopupsSamplingRate or if when setting $wgPopupsSchemaPopupsSamplingRate in mediawiki-config PHP that will also in turn be automagically reflected in the JavaScript mw.config pertinent to the given wiki.

This is precisely why I'd like another engineer to take a look at the patch, as I am confused as well. The -2 on the patch is just to make it clear that this task is blocked (although config changes are not merged without SWATs anyways AFAIK, so the patch shouldn't accidentally end up merged).
/cc @phuedx @Jdlrobson @Jhernandez @bmansurov

$wgPopupsSchemaPopupsSamplingRate dictates what % of users we log events for. It defaults to 0.1 in the extension so that's the sampling rate that's been applying to Hungarian Wikipedia and Adam's right in that we should maintain that.

Using PopupsExperimentConfig will graduate this from beta feature so it will load for all users. If a user has previously tried it as a beta feature, their previous preference will be respected. If they haven't they will need to disable it. We cannot run A/B tests where we only apply to anon users. If that's wanted that's a more complicated task.

I hope this clears up confusion.

See also, for reference, the documentation of the sampling method at https://meta.wikimedia.org/wiki/Schema_talk:Popups . (I have to admit that I forgot the rationale why we add more readers to the experiment than will be in the EventLogging sample, i.e. enable the feature for more sessions than we obtain data from.)
And yes, while the focus of the analysis has been on anonymous users in the huwiki test and will likely remain so for the other wikis, we have been looking at the EL data from logged-in users too, of course keeping in mind the slight complication that arises from the fact that they can also have enabled Hovercards via the beta feature channel.

@Jdlrobson - that's fine, I just wanted to double check their initial preferences are kept.

Elitre added a comment.EditedSep 16 2016, 12:18 PM

I'm confused: I thought this was the exact test carried on at hu.wp, and you said it would affect anons only?
Here's the thing: when announcing the test on it.wp, we specified anons only would see the hovercards (also because your goal is measuring interaction with fundraising banners, which logged in users do not get).
If your plan changes for any reason, this needs to be communicated in advance and you'll need to discuss with the community (that is, saying it now to start on the 19th is not enough).

ovasileva added a comment.EditedSep 16 2016, 12:38 PM

@Elitre - sorry for the confusion, I believe it was due to a misunderstanding on my side on the conditions of the hu test. The test is for both from the perspective that we are collecting data from both types of users. However, if a logged-in user has hovercards off (they have turned them off), hovercards will remain off. If the user has hovercards on, hovercards will remain on. If a user has never seen interacted with the beta feature, hovercards will be turned on - but they may turn them off upon initial viewing. Do you think we should issue a clarification with the community and if so, how long would you recommend for the notice to stay up? If necessary, we can postpone the launch of the A/B test by a 1-2 days and still have enough time between the launch and the fundraising campaign.

@Jdlrobson - from our side, I was also under the impression that we ran the hu test only on anons. Would this be significantly more difficult to implement?

Elitre added a comment.EditedSep 16 2016, 1:17 PM

If a user has never seen interacted with the beta feature, hovercards will be turned on - but they may turn them off upon initial viewing.

This is the thing, you'd be de facto enabling it to several logged-in people, and despite the fact that there would be multiple ways to turn it off/not get it at all, this is not what we communicated.
The problem is also that there's a weekend in between, and 5 days (more like four, because of timezones) are not so many when it comes to having to inform thousands people. You'll probably get considerably different feedback at this point, and you need to think about what you want to do in the worst case scenario in which the test is rejected.
However, you may want to start to write an explanation for the change (I believe that A/B tests only for anons were carried out in the past, so I don't know what the specific difficulty with Hovercards may be here), and list the several ways in which people can avoid getting hovercards in the first place or disable it later. I can discuss later today with Chris if he needs help with a new announcement. The key elements which would be needed at this point IMHO are:

  1. explanation that the Hungarian test was not anons only as well;
  2. clarification of why you'd need registered users as well, when they are not affected by fundraising banners;
  3. ways in which people could disable HC or prevent it from showing up - so far I can think of:
  • You won't see HC if you had disabled it in Beta Features (which should also mean, you won't be affected by the test if you enable and disable HC now)
  • You won't see HC if you have NavPopups enabled in any form
  • You can disable HC from the cog (includes anons).

@Elitre - replied in email and cc'd Chris. Also, cc @Ckoerner

Change 311148 had a related patch set uploaded (by Jhobs):
EventLogging sampling rate should default to 0

https://gerrit.wikimedia.org/r/311148

As per Baha's comment via email:

"Olga, to answer you question #1, the change seems trivial. Adding an additional condition to https://github.com/wikimedia/mediawiki-extensions-Popups/blob/master/resources/ext.popups.schemaPopups.utils.js#L48 seems to prevent A/B test from applying to logged in users. With this change, we maybe on course with our previous timeline."

Could we apply this change?

bmansurov added a comment.EditedSep 16 2016, 5:32 PM

Ooops, I mixed up schema sampling with A/B test. I'll update here soon. The correct solution seems trivial too. Another conditional is needed at https://github.com/wikimedia/mediawiki-extensions-Popups/blob/master/resources/ext.popups.experiment.js#L63. Note that there maybe some other changes. I'm just expressing my opinion after a cursory glance at the codebase.

Ok, so based on https://github.com/wikimedia/mediawiki-extensions-Popups/blob/master/resources/ext.popups.experiment.js#L63 we know whether user has feature enabled or disabled. Then we respect user settings when user has explicitly disabled (or enabled) feature. Can't we skip this for the user disabled group and instead disable for all users that have the feature disabled (without checking if they've explicitly done so first?)

Jhernandez added a comment.EditedSep 19 2016, 10:13 AM

We know whether user has explicitly enabled or disabled the feature via hovercard affordances (Enable previews at the bottom of the page, or settings cog in a hovercard).

Then we respect user settings when user has explicitly disabled (or enabled) feature.

Correct.

Can't we skip this for the user disabled group and instead disable for all users that have the feature disabled (without checking if they've explicitly done so first?)

I'm not following here. Users that haven't explicitly done anything (either via hovercards affordances or enabling via the beta features menu) are the ones we target for bucketing. Otherwise, if they have done something, we respect their decision.

If what we need now is to not target logged in users for bucketing in the experiment, then it should be pretty simple as Baha has mentioned:

resources/ext.popups.experiment.js#L81-L83
+ // Disable if experiment is disabled or user is logged in
- if ( !config ) {
+ if ( !config || !mw.user.isAnon() ) {
	return false;
  }

That would make the experiment:

  • Bucket between control and A:
    • Anonymous users
  • Have hovercards enabled:
    • Logged in users with Hovercards beta feature enabled
    • Anons in control that enable previews manually (via hovercard affordances)
    • Logged in users that enable previews manually (via hovercard affordances)
  • Have hovercards disabled:
    • Anonymous users that disabled previews manually (via hovercard affordances)
    • Beta feature enabled logged in users that disabled manually hovercards (via hovercard affordances)
    • All other logged in users that did nothing

If this is what we've promised and need then let's work on the subtask before enabling the test.

If it is still confusing, let's touch base this afternoon and clarify what is going on. Too many comments here and in email, this is getting too confusing.

Apparently my guess above was wrong and this is more involved than that.

See T146017

I'm under the impression that for logged in users, while the A/B test is in force on a wiki (e.g., like with huwiki) the existing Hovercards preference is honored. In other words, irrespective of what's happening in the underlying JavaScript, the UX for logged in users interacting with links is for all intents and purposes the same as it would be absent the A/B test. In other words, logged in users are not really part of the A/B test.

Regarding event logging percentages, those don't impact the UX of logged in users, either. The event logging percentage does need to be adjusted down like in the description for these larger wikis, in order to reduce the number of events that will ultimately need to be stored and later queried.

Regarding a user who is logged out and then later logs in, my understanding is the likelihood of having Hovercards ON for the user when she's not yet logged in would be the same as any other not-yet-logged-in user. But when she logs in, her pre-existing preferences as a logged in user for Hovercards are honored.

I see further discussion ensued via T146017: Hovercards beta feature preference OFF not honored for logged in users when the experiment is enabled. Perhaps it would be worth clearing up that task and the understanding on this task by yielding a few minutes during story prioritization today?

@dr0ptp4kt That's not really how it is working right now (huwiki).

  1. First, browser settings are honored: https://github.com/wikimedia/mediawiki-extensions-Popups/blob/master/resources/ext.popups.experiment.js#L66-L74
  2. Then if none, beta features preferences are considered (https://github.com/wikimedia/mediawiki-extensions-Popups/blob/master/resources/ext.popups.experiment.js#L76-L79)
  3. Lastly, if there's no browser or beta feature preference, the user is bucketed https://github.com/wikimedia/mediawiki-extensions-Popups/blob/master/resources/ext.popups.experiment.js#L87

AFAICT there's no anon vs logged in user logic. It seems like logged in users are bucketed as well as anons.

In fact we sent to event logging isAnon and the editCountBucket fields.

Regarding a user who is logged out and then later logs in, my understanding is the likelihood of having Hovercards ON for the user when she's not yet logged in would be the same as any other not-yet-logged-in user

Yes. Same chance of being bucketed. Which is what they don't want in this task, for logged in users to randomly get hovercards enabled.

But when she logs in, her pre-existing preferences as a logged in user for Hovercards are honored.

This is not true since as you saw first thing checked are the browser settings (initially set by the bucketing logic when anon before logging in), which prevent honoring the beta feature preferences.

Apparently it was understood that the code did some things that it doesn't really do in these use cases, probably mistakes in communicating what needs to be done and what it's actually being done.

T146017 is an attempt at skipping logged in users for the experiment and remembering their user preference from beta features exclusively as the way for logged in users to manage hovercards.

@Jhernandez okay, I think I understand what you're saying. Looking at those sections, I would draw a similar conclusion, barring some implicit logical condition. Here's what I understand based on your feedback and looking at that code:

In the case that the user was an anonymous users bucketed to have Hovecards ON, and then on a dwell of a link got a Hovercard and tapped the Settings cog, and then in the modal with the "Enabled" radio button selected (i.e., the default case) tapped Save (theoretically, seems like a no-op...but...), then the choice from that modal could override her Special:Preferences mapped checkbox value for Hovercards, in case it was unchecked in Special:Preferences. Likewise, if in the modal she chose "Disabled" and hit Save, yet her Special:Preferences had the checkbox checked for Hovercards, her choice from the modal could override Special:Preferences.

I suspect that when a user uses the modal and hits Save, she really does mean to keep that setting (that as I understand was the possibility for the logged in user all along, it's now the case that she could override the setting while logged out). Sounds like a verbal discussion would be helpful :)

I should say, I anticipate most users don't use the footer link to get the modal. Invocation of the cog from the Hovercard seemed rare in general as it was, and footer seems almost implausible for most.

dr0ptp4kt added a comment.EditedSep 19 2016, 4:30 PM

AFAICT there's no anon vs logged in user logic. It seems like logged in users are bucketed as well as anons.

From what I see, in the case that the user has not used the settings cog and tapped Save in the modal, the following does implicitly honor the logged in checkbox from Special:Preferences:

https://github.com/wikimedia/mediawiki-extensions-Popups/blob/master/resources/ext.popups.experiment.js#L76-L87

I believe that code is saying:

  1. Get me the checkbox value from Special:Preferences and if it was checked, Hovercards are on.
  2. In the case we're not doing an A/B test, Hovercards should be OFF. Implicitly, we know the Special:Preferences must be unchecked here, because otherwise we would have returned true in point 1.
  3. Otherwise, by implication the A/B test is in flight and the user is an anonymous user (who also hasn't ever saved the settings via the modal), and we should honor the bucketing.

Do I understand that correctly? It sounds like the edge case we're thinking about is the user who saved a setting in the modal, and that setting disagrees with the user's Special:Preferences.

Thanks for the comments, you're right, I'm going to have to re-evaluate that other patch and the subtask. More tomorrow.

ovasileva added a comment.EditedSep 19 2016, 6:05 PM

Ok, so based on the above, these should be our use cases. I'm not sure if I'm missing some that are important, but I'm assuming the rest are edge cases (like turning off via beta, then on via the link) I'm assuming we can only account for some of them:

Enabled

  1. User has hovercards on (logged in)
  2. A/B test starts
  3. Hovercards appear
  4. User disables from settings cog OR User disables via beta features
  5. Hovercards do not appear
    1. NOTE: current behavior - if user disables from settings cog, hovercards still appear "on" in beta (the checkbox is clicked). We should fix this, but it is an edge case.
  6. User logs out
  7. User is placed (or not) into bucket
  8. User logs back in
  9. Hovercards do not appear

Disabled by default

  1. User has hover cards off (by default)
  2. A/B test starts
  3. Hovercards do not appear
  4. User logs out
  5. User is placed (or not) into bucket
  6. Hovercards appear (or not)
  7. User logs back in
  8. Hovercards do not appear

Disabled via Settings

  1. User has hover cards off (explicitly via settings cog)
  2. A/B test starts
  3. Hovercards do not appear
  4. User logs out
  5. User is placed (or not) into bucket
  6. Hovercards appear (or not)
  7. User logs back in
  8. Hovercards do not appear

Disabled via Beta Settings

  1. User has hover cards off (explicitly via beta)
  2. A/B test starts
  3. Hovercards do not appear
  4. User logs out
  5. User is placed (or not) into bucket
  6. Hovercards appear (or not)
  7. User logs back in
  8. Hovercards do not appear

https://github.com/wikimedia/mediawiki-extensions-Popups/blob/master/resources/ext.popups.experiment.js#L76-L87
I believe that code is saying:

  1. Get me the checkbox value from Special:Preferences and if it was checked, Hovercards are on.
  2. In the case we're not doing an A/B test, Hovercards should be OFF. Implicitly, we know the Special:Preferences must be unchecked here, because otherwise we would have returned true in point 1.
  3. Otherwise, by implication the A/B test is in flight and the user is an anonymous user (who also hasn't ever saved the settings via the modal), and we should honor the bucketing.

@dr0ptp4kt - I think I'm still missing something. Here, explicitly enabled/disabled right above is referring to enabling/disabling from the settings cog and bottom link, not from the beta settings, right? And then we read the value of the preferences checkbox. Is it just that we need to say explicitly if checkbox is disabled then take them out? (or has this actually been working as expected all along...)

@dr0ptp4kt - I think I'm still missing something. Here, explicitly enabled/disabled right above is referring to enabling/disabling from the settings cog and bottom link, not from the beta settings, right? And then we read the value of the preferences checkbox. Is it just that we need to say explicitly if checkbox is disabled then take them out? (or has this actually been working as expected all along...)

I would say things are generally working as expected in practice, because it appears few people ever go to the settings cog. That said, it would probably be easier to discuss on a video call. Would it be possible to make the Hovercards meeting one hour later tomorrow? I'm happy to join if so.

@phuedx to add deployment plan

jhobs reassigned this task from jhobs to Jdlrobson.Sep 22 2016, 5:33 PM
jhobs added a subscriber: jhobs.

Over to @Jdlrobson for deployment. To be clear since the comment thread has gotten quite long:
This config change should be deployed first: https://gerrit.wikimedia.org/r/#/c/310483/
Then this MobileFrontend change can be merged, or deployed in the same SWAT if you feel like it: https://gerrit.wikimedia.org/r/#/c/311148/ (it's just hygiene)

@Jdlrobson - use cases here: https://phabricator.wikimedia.org/T146017#2651887 - ping me when we're good to go and I can help test

Change 310483 merged by Alex Monk:
Initiate Hovercards A/B test on ruwiki and itwiki

https://gerrit.wikimedia.org/r/310483

The A/B test is now live.
Verified that the code doesn't load on English Wikipedia (mw.loader.getState( 'ext.popups.core' ) === 'registered')
but does on Italian and Russian (mw.loader.getState( 'ext.popups.core' ) === 'ready')

  • Verified I could opt into the A/B test on both Italian and Russian
  • on Italian you can opt in via the following line of code:
localStorage.setItem( "PopupsExperimentID", "966facd00261c8e4" )

It's worth noting that on cached HTML popups will not show so we'll have to wait a few days before it can be considered a true A/B test. If you hit a page where you can't see them ?action=purge will need to be appended to the url.

  • On Russian you can opt in via the following line of code
localStorage.setItem( "PopupsExperimentID", "5" )

A quick peek at Grafana (change "Last 24 hours" to "last 7 days") shows the effect of the change going live: the maximum event rate has about doubled from 114 to 243/min. Which is still safely below the rates that caused database problems in the past.

Could someone update https://meta.wikimedia.org/wiki/Schema_talk:Popups with the current sampling rates (as a general practice going forward, per the recent email thread)? I would do it myself, but I'm not certain if I have understood the outcome of the above discussion in all details.

In particular, I am wondering why the event rate for the existing experiment on huwiki went up too, in the old version of the schema no less:

SELECT LEFT(timestamp,8) AS date, COUNT(*) AS events_15777589 FROM log.Popups_15777589 WHERE wiki ='huwiki' AND timestamp LIKE '201609%' GROUP BY date ORDER BY date;
+----------+-----------------+
| date     | events_15777589 |
+----------+-----------------+
| 20160901 |          104264 |
| 20160902 |          107854 |
| 20160903 |           97284 |
| 20160904 |          118570 |
| 20160905 |          141198 |
| 20160906 |          140092 |
| 20160907 |          126012 |
| 20160908 |          108734 |
| 20160909 |           41211 |
| 20160910 |           36878 |
| 20160911 |           49489 |
| 20160912 |           50219 |
| 20160913 |           53622 |
| 20160914 |           51756 |
| 20160915 |           49721 |
| 20160916 |           41077 |
| 20160917 |           44024 |
| 20160918 |           53896 |
| 20160919 |           55551 |
| 20160920 |           51952 |
| 20160921 |           75241 |
| 20160922 |          104782 |
| 20160923 |               2 |
+----------+-----------------+
23 rows in set (1.61 sec)

SELECT LEFT(timestamp,8) AS date, COUNT(*) AS events_15906495 FROM log.Popups_15906495 WHERE wiki ='huwiki' GROUP BY date ORDER BY date;
+----------+-----------------+
| date     | events_15906495 |
+----------+-----------------+
| 20160922 |           22373 |
| 20160923 |            8540 |
+----------+-----------------+
2 rows in set (0.05 sec)

(BTW there also appears to be a general problem currently whereby all EL tables are lagging by up to >7 hours or more. I pinged the DBAs about it on IRC, but haven't heard back yet, may file a bug later. Resolving this lag would only increase those recent huwiki numbers though.)

PS:

A quick peek at Grafana (change "Last 24 hours" to "last 7 days") shows the effect of the change going live: the maximum event rate has about doubled from 114 to 243/min. Which is still safely below the rates that caused database problems in the past.

It has now reached 746/min, but that should still be fine.

Could someone update https://meta.wikimedia.org/wiki/Schema_talk:Popups with the current sampling rates (as a general practice going forward, per the recent email thread)? I would do it myself, but I'm not certain if I have understood the outcome of the above discussion in all details.
In particular, I am wondering why the event rate for the existing experiment on huwiki went up too, in the old version of the schema no less:

SELECT LEFT(timestamp,8) AS date, COUNT(*) AS events_15777589 FROM log.Popups_15777589 WHERE wiki ='huwiki' AND timestamp LIKE '201609%' GROUP BY date ORDER BY date;
+----------+-----------------+
| date     | events_15777589 |
+----------+-----------------+
| 20160901 |          104264 |
| 20160902 |          107854 |
| 20160903 |           97284 |
| 20160904 |          118570 |
| 20160905 |          141198 |
| 20160906 |          140092 |
| 20160907 |          126012 |
| 20160908 |          108734 |
| 20160909 |           41211 |
| 20160910 |           36878 |
| 20160911 |           49489 |
| 20160912 |           50219 |
| 20160913 |           53622 |
| 20160914 |           51756 |
| 20160915 |           49721 |
| 20160916 |           41077 |
| 20160917 |           44024 |
| 20160918 |           53896 |
| 20160919 |           55551 |
| 20160920 |           51952 |
| 20160921 |           75241 |
| 20160922 |          104782 |
| 20160923 |               2 |
+----------+-----------------+
23 rows in set (1.61 sec)
SELECT LEFT(timestamp,8) AS date, COUNT(*) AS events_15906495 FROM log.Popups_15906495 WHERE wiki ='huwiki' GROUP BY date ORDER BY date;
+----------+-----------------+
| date     | events_15906495 |
+----------+-----------------+
| 20160922 |           22373 |
| 20160923 |            8540 |
+----------+-----------------+
2 rows in set (0.05 sec)

(BTW there also appears to be a general problem currently whereby all EL tables are lagging by up to >7 hours or more. I pinged the DBAs about it on IRC, but haven't heard back yet, may file a bug later. Resolving this lag would only increase those recent huwiki numbers though.)

They have since fixed this database replication issue. Regenerating the above queries appears to confirm the conclusion that the event rate on the Hungarian Wikipedia has increased considerably.
Perhaps this has instead to do with the fix for dropped events at T146035?

SELECT NOW();
+---------------------+
| NOW()               |
+---------------------+
| 2016-09-23 08:06:16 |
+---------------------+
1 row in set (0.00 sec)

SELECT LEFT(timestamp,8) AS date, COUNT(*) AS events_15777589 FROM log.Popups_15777589 WHERE wiki ='huwiki' AND timestamp LIKE '201609%' GROUP BY date ORDER BY date;
 
+----------+-----------------+
| date     | events_15777589 |
+----------+-----------------+
| 20160901 |          104264 |
| 20160902 |          107854 |
| 20160903 |           97284 |
| 20160904 |          118570 |
| 20160905 |          141198 |
| 20160906 |          140092 |
| 20160907 |          126012 |
| 20160908 |          108734 |
| 20160909 |           41211 |
| 20160910 |           36878 |
| 20160911 |           49489 |
| 20160912 |           50219 |
| 20160913 |           53622 |
| 20160914 |           51756 |
| 20160915 |           49721 |
| 20160916 |           41077 |
| 20160917 |           44024 |
| 20160918 |           53896 |
| 20160919 |           55551 |
| 20160920 |           51952 |
| 20160921 |           75241 |
| 20160922 |          104782 |
| 20160923 |              42 |
+----------+-----------------+
23 rows in set (1.57 sec)

SELECT LEFT(timestamp,8) AS date, COUNT(*) AS events_15906495 FROM log.Popups_15906495 WHERE wiki ='huwiki' GROUP BY date ORDER BY date;
+----------+-----------------+
| date     | events_15906495 |
+----------+-----------------+
| 20160922 |           22373 |
| 20160923 |           28194 |
+----------+-----------------+
2 rows in set (0.06 sec)

Could someone update https://meta.wikimedia.org/wiki/Schema_talk:Popups with the current sampling rates (as a general practice going forward, per the recent email thread)? I would do it myself, but I'm not certain if I have understood the outcome of the above discussion in all details.

  • I'll do it
ovasileva closed this task as Resolved.Sep 23 2016, 3:47 PM

@Tbayer with regards to the Hungarian wiki increase in traffic - yes T146035 is likely to increase numbers - that bug was likely to have been preventing Hovercards from showing for many users - which I noted in that ticket to be considered in any analysis.

The fix to T146035 was SWATTed Thursday morning (PST), whereas this particular SWAT happened Thursday PM so I'd probably you will see a spike in the morning if you want to look more closely at times.

Change 311148 merged by jenkins-bot:
EventLogging sampling rate should default to 0

https://gerrit.wikimedia.org/r/311148