Page MenuHomePhabricator

Enable e-mail notifications for watchlist (EnotifWatchlist) on all wikis
Closed, ResolvedPublic

Description

EnotifWatchlist can do wonders for users participation. It's obviously desirable to have this preference enabled (obviously opt-in) on all wikis.
It's been enabled since long time ago on quite big wikis as Commons and Meta for obvious reasons, but also on nlwiki.
I see that recently (bug 16410, bug 27003) it's been enabled also on some "random" small (content) wikis, so please consider to enable it everywhere if, as suggested by bug 5220 comment 32, there are no big load issues.

This could mean all wikis except those currently with EnotifuserTalk disabled, i.e.

'dewiki' => false,
'enwiki' => false,
'eswiki' => false,
'frwiki' => false,
'itwiki' => false,
'jawiki' => false,
'plwiki' => false,
'ptwiki' => false,
'ruwiki' => false,
'zhwiki' => false,

Version: unspecified
Severity: enhancement
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=36331

Details

Reference
bz28026

Event Timeline

bzimport raised the priority of this task from to Lowest.Nov 21 2014, 11:31 PM
bzimport set Reference to bz28026.

Mark, does this proposal (switch e-mail notifications for watchlist to blacklisting a few large wikis instead of whitelisting a bunch of small ones) sound reasonable to you in terms of e-mail sending capacity and such?

What is bad with sending a lot of emails? Can't we "just" add new SMTP out servers?

hello, I suggest "dry run":

i.e. activate but do not send the mails, but count for while (I mean, generate statistics, how many mails would be sent)

wgEnotifUserTalk @{

'wgEnotifUserTalk' => array(

'default' => true,
/*
'cswikinews' => true,
'dewiki' => false,
'enwiki' => false,
'enwikinews' => true, // https://bugzilla.wikimedia.org/show_bug.cgi?id=12975
'enwikiquote' => true,
'enwikisource' => true, // #13386
'eswiki' => false,
'frwiki' => false,
'itwiki' => false,
'jawiki' => false,
'nlwiki' => true,
'nowikinews' => true, // https://bugzilla.wikimedia.org/show_bug.cgi?id=13564
'plwiki' => false,
'ptwiki' => false,
'ruwiki' => false,
'sourceswiki' => true,
'yiwiktionary' => true, // http://bugzilla.wikimedia.org/show_bug.cgi?id=5323
'yiwiki' => true, // https://bugzilla.wikimedia.org/show_bug.cgi?id=12991
'zhwiki' => false,
 */

),

It's enabled everywhere now

This bug is about eNotifWatchlist? Aren't you talking about eNotifTalk, Reedy?

eNotifWatchlist (still defaulting to "false" AFAIK) includes a comment from Tim: "// No, this does not mean everyone can have it -- TS"

danny.leinad wrote:

In http://noc.wikimedia.org/conf/highlight.php?file=InitialiseSettings.php there is:

'wgEnotifWatchlist' => array(

'default' => false,

so reopened.

Restored keyword triage added by bugmeister.

Adding Tim as CC so we can see if this could be something we do like the notifications we did in Berlin.

Hello all developers and Toolserver admins (this needs access to Master db),
this bug's solution requires some statistical analysis.

Before switching on, it would be nice to have the hourly or daily number of enotifs (for the mentioned case) which _would_ be sent out.

This could be done through a modification of UserMailer in that way, that "enotifs for watched pages which are not user- or user_talk-pages" are _not_ sent, but only counted (you can call it "dry run", or "simulation" mode)

-shell as it's not in a position to actually be enabled at this time

(In reply to comment #10)

-shell as it's not in a position to actually be enabled at this time

Re-added the "shell" and "ops" keywords. You may have one or both. You may not have none.

(In reply to comment #9)

Before switching on, it would be nice to have the hourly or daily number of
enotifs (for the mentioned case) which _would_ be sent out.
This could be done through a modification of UserMailer in that way, that
"enotifs for watched pages which are not user- or user_talk-pages" are _not_
sent, but only counted (you can call it "dry run", or "simulation" mode)

Bugmaster Mark has coined this recently "null-mailer", a term I like. If someone supports the idea of a null-mailer, a separate bug with this as subject should be filed for the implementation.

(In reply to comment #12)

Bugmaster Mark has coined this recently "null-mailer", a term I like. If
someone supports the idea of a null-mailer, a separate bug with this as subject
should be filed for the implementation.

r93415 is my attempt to make the null mailer possible. I didn't coin the term, though: http://google.com/search?q=nullmailer

After the dry run, we should only do this for editors who positively opt-in, and probably only those who positively opt-in after we make these enotifs live. In other words start with all editor's enotifs disabled. Stack Overflow recently emailed users with dormant accounts and the results weren't pretty.

http://meta.stackoverflow.com/questions/102979/spam-from-super-user
(see also the discussion here: http://meta.stackoverflow.com/questions/102797/please-dont-send-me-email-that-i-didnt-ask-for)

I expect that our editors would react similarly if we were to start emailing them out of the blue.

(In reply to comment #14)

After the dry run, we should only do this for editors who positively opt-in,
and probably only those who positively opt-in after we make these enotifs live.

+1

In other words start with all editor's enotifs disabled.

+1

Mark, very good points.

After having read your comment, I got the following idea for an ever softer start:

This is an idea, which is not implemented in the code, and therefore not possible with current enotif options:

Idea

allow only those persons to opt-in, who have _edited_ (not an minor edit) an article which they are watching (and want to have e-notified now).

If someone likes that, a new bugzilla should be opened.

allow only those persons to opt-in, who have _edited_ (not an minor edit) an
article which they are watching (and want to have e-notified now).

I hereby withdraw my idea as in https://bugzilla.wikimedia.org/show_bug.cgi?id=28026#c15 .

Why ?

"opt-in" is globally - for the user - for all watched pages, a distinction between

  • "enotif opt-in for pages I edited and I am watching" and
  • "enotif opt-in for pages I did not edit but which I am watching"

would be necessary - which is too complicated.

A further code change may be required which nulls - or sets - also the enotif-timestamps for the notifications (not removing the page from wl) in the watching user's watchlist, when this feature is globally enabled - too much work, I guess.

Thanks. Another perspective: each enotif currently comes already with three single-click solutions for unwatching the notified page. These are defined in languages/MessageEn.php'enotif_body' = [[MediaWiki:enotif_body]] )

  • To change your email notification settings, visit ...
  • To change your watchlist settings, visit ...
  • To delete the page from your watchlist, visit ...

Proposal based on Mark's links re. email annoyances:

In the enotif_body, we could probably add a further direkt link as a single-click solution:

  • To disable your email notifications, visit ...

This will then be a single click solution, too, to fully reset enotif for the notified user, so that they can opt-out after having received their first single e-notif after this is enabled on that wiki.

(In reply to comment #14)

Stack Overflow
recently emailed users with dormant accounts and the results weren't pretty.

I don't think that sending newsletters and invites to get active again is the same thing as sending notifications for changes that the user has decided to follow. The previous experience with notifications for user talk message shows that this annoyed mainly users of the English Wikipedia who don't edit other wikis and hence didn't know that email notifications existed in MediaWiki and thought it was something suspect or some big change.
That said, I agree that the most sensible thing to start with is to enable the feature with the option disabled by the default for old users, but if possible enabled by default for new users (who won't experience problems).

(In reply to comment #18)

In the enotif_body, we could probably add a further direkt link as a
single-click solution:

  • To disable your email notifications, visit ...

I agree, but this is unrelated and applies to all email notifications, please file another bug for it.

I think we can deploy it to all wikis, not just small wikis. But $wgShowUpdatedMarker should be false by default to start with, so that we can separate the performance impact of enotif from the performance impact of update markers.

So the plan is to disable $wgShowUpdatedMarker by default and enable $wgEnotifUserTalk on all wikis. This allows better single-variable testing if a problem emerges. Now all that's needed is an okay from ops and a deployment window.

Mark H.: Can you find someone from ops to work with Reedy on scheduling a deployment time? I think a page on wikitech might be the place where this is coordinated currently.

When it is enabled, I would be interested in statistics

  • how many enotif mails are sent and
  • to how many distinct users

per time period i.e. per hour, or per day.

(In reply to comment #23)

RT: http://rt.wikimedia.org/Ticket/Display.html?id=1784

What's the status of that RT ticket?

(In reply to comment #24)

(In reply to comment #23)

RT: http://rt.wikimedia.org/Ticket/Display.html?id=1784

What's the status of that RT ticket?

Created on Oct 22, dormant since.

(begin RT report)
As MZ summarizes: "the plan is to disable $wgShowUpdatedMarker by default and enable $wgEnotifUserTalk on all wikis."

This ticket is to provide a heads-up to Ops so that you guys can work with Sam to set up a deployment window or
discuss other issues that relate to enabling a lot of emails.
(end RT report)

(In reply to comment #25)

(In reply to comment #24)

(In reply to comment #23)

RT: http://rt.wikimedia.org/Ticket/Display.html?id=1784

What's the status of that RT ticket?

Created on Oct 22, dormant since.

(begin RT report)
As MZ summarizes: "the plan is to disable $wgShowUpdatedMarker by default and
enable $wgEnotifUserTalk on all wikis."

This ticket is to provide a heads-up to Ops so that you guys can work with Sam
to set up a deployment window or
discuss other issues that relate to enabling a lot of emails.
(end RT report)

Cool, thanks for that.

Is there a way to get this in a queue to be deployed? Is there such a queue? If not, what are next steps here?

(In reply to comment #26)

Cool, thanks for that.

Is there a way to get this in a queue to be deployed? Is there such a queue? If
not, what are next steps here?

It's already on platform eng's radar, and they're the guys that typically deploy stuff. We just need to get ops to respond to the RT ticket now.

(In reply to comment #27)

(In reply to comment #26)

Cool, thanks for that.

Is there a way to get this in a queue to be deployed? Is there such a queue? If
not, what are next steps here?

It's already on platform eng's radar, and they're the guys that typically
deploy stuff. We just need to get ops to respond to the RT ticket now.

Is there a good way to do that?

(In reply to comment #27)

(In reply to comment #26)

Cool, thanks for that.

Is there a way to get this in a queue to be deployed? Is there such a queue? If
not, what are next steps here?

It's already on platform eng's radar, and they're the guys that typically
deploy stuff. We just need to get ops to respond to the RT ticket now.

(In reply to comment #28)

(In reply to comment #27)

(In reply to comment #26)

Cool, thanks for that.

Is there a way to get this in a queue to be deployed? Is there such a queue? If
not, what are next steps here?

It's already on platform eng's radar, and they're the guys that typically
deploy stuff. We just need to get ops to respond to the RT ticket now.

Is there a good way to do that?

Well, we don't really need Ops to do anything. It's a case of it just needs to be done when ops members are around incase it causes problems due to the increased volume of emails

(In reply to comment #28)

So the plan is to disable $wgShowUpdatedMarker by default and enable
$wgEnotifUserTalk on all wikis. This allows better single-variable testing if a
problem emerges. Now all that's needed is an okay from ops and a deployment
window.

http://noc.wikimedia.org/conf/highlight.php?file=InitialiseSettings.php
now shows:

'wgEnotifUserTalk' => array(

'default' => true,

),

$wgShowUpdatedMarker has not been changed and defaults to true.

Mark said he was willing to test wgEnotifUserTalk for an hour, but the impact was pretty minimal, so he left it on.

Now, do we still want EnotifWatchlist as this bug originally requested? If so, I'll create another RT ticket.

'wgEnotifUserTalk' is something different.

The bug's subject is for having
$wgEnotifWatchlist = true;

(In reply to comment #30)

(In reply to comment #28)

So the plan is to disable $wgShowUpdatedMarker by default and enable
$wgEnotifUserTalk on all wikis. This allows better single-variable testing if a
problem emerges. Now all that's needed is an okay from ops and a deployment
window.

http://noc.wikimedia.org/conf/highlight.php?file=InitialiseSettings.php
now shows:

'wgEnotifUserTalk' => array(

'default' => true,

),

$wgShowUpdatedMarker has not been changed and defaults to true.

Mark said he was willing to test wgEnotifUserTalk for an hour, but the impact
was pretty minimal, so he left it on.

Now, do we still want EnotifWatchlist as this bug originally requested? If so,
I'll create another RT ticket.

Okay, that one was my bad. I should have said "the plan is to disable $wgShowUpdatedMarker by default and enable $wgEnotifWatchlist on all wikis" (not $wgEnotifUserTalk). My apologies.

Can you file or un-file or re-file an RT ticket about EnotifWatchlist, please? Sorry for the confusion. :-(

(In reply to comment #32)

Okay, that one was my bad. I should have said "the plan is to disable
$wgShowUpdatedMarker by default and enable $wgEnotifWatchlist on all wikis"
(not $wgEnotifUserTalk). My apologies.

Nevertheless, it was probably a good first step. It increased email incrementally instead of exponentially. Filed http://rt.wikimedia.org/Ticket/Display.html?id=1927:

"Since #1784 went off without a hitch, we'd like to enable $wgEnotifWatchlist on all wikis. This ticket is to track discussion of any that relate to enabling a lot of emails."

(In reply to comment #30)

(In reply to comment #28)

So the plan is to disable $wgShowUpdatedMarker by default and enable
$wgEnotifUserTalk on all wikis. This allows better single-variable testing if a
problem emerges. Now all that's needed is an okay from ops and a deployment
window.

http://noc.wikimedia.org/conf/highlight.php?file=InitialiseSettings.php
now shows:

'wgEnotifUserTalk' => array(

'default' => true,

),

$wgShowUpdatedMarker has not been changed and defaults to true.

Mark said he was willing to test wgEnotifUserTalk for an hour, but the impact
was pretty minimal, so he left it on.

Now, do we still want EnotifWatchlist as this bug originally requested? If so,
I'll create another RT ticket.

Please carefully read what I did in CommonSettings.php line 1516 (search for "Trial by Roan, Mark B and Mark H") to understand what the config actually is. The values from InitialiseSettings are overridden there.

(In reply to comment #34)

Please carefully read what I did in CommonSettings.php line 1516 (search for
"Trial by Roan, Mark B and Mark H") to understand what the config actually is.
The values from InitialiseSettings are overridden there.

Essentially, this only changes $wgShowUpdatedMarker to false if $wgEnotifWatchlist is false and sets wgEnotifUserTalk true, regardless.

Except for the value of $wgShowUpdatedMarker, I'm not sure what is different from the statements you quoted.

(In reply to comment #35)

(In reply to comment #34)

Please carefully read what I did in CommonSettings.php line 1516 (search for
"Trial by Roan, Mark B and Mark H") to understand what the config actually is.
The values from InitialiseSettings are overridden there.

Essentially, this only changes $wgShowUpdatedMarker to false if
$wgEnotifWatchlist is false and sets wgEnotifUserTalk true, regardless.

Except for the value of $wgShowUpdatedMarker, I'm not sure what is different
from the statements you quoted.

The value of $wgShowUpdatedMarker was exactly what I meant. I'm sorry for not making that clearer.

(In reply to comment #37)

What's the status of this bug?

No action on the RT ticket. I'll talk to Mark tomorrow about RT#1927.

After talking to Mark, what I'll do is talk to Sam to get a test slot to get this deployed for an hour or so. That way we can have Ops monitoring it and have a chance to turn it off if something "bad" happens.

Mark just remembered that the Peter is puppet-izing the mail servers and would like to wait until he has finished that to act on this ticket.

again: I suggest you run it - but with a "null-mailer" (not sending mails to watching users but simply to see impact on servers, and to get some statistics).

(In reply to comment #40)

Mark just remembered that the Peter is puppet-izing the mail servers and would
like to wait until he has finished that to act on this ticket.

What's the ETA on this?

(In reply to comment #42)

(In reply to comment #40)

Mark just remembered that the Peter is puppet-izing the mail servers and would
like to wait until he has finished that to act on this ticket.

What's the ETA on this?

Mark said "i'm working on it as we speak" just now.

The mail servers are currently being migrated/re-installed, so still in progress

(In reply to comment #45)

The mail servers are currently being migrated/re-installed, so still in
progress

The migration is over since a while, any update?

(In reply to comment #46)

The migration is over since a while, any update?

I talked to CT Woo about this and he is on it.

So, what exactly needs changing configuration wise?

Changing $wgEnotifWatchlist to true, and leaving $wgShowUpdatedMarker as false for the moment. And then looking to set $wgShowUpdatedMarker to true at somepoint in the near future?

(In reply to comment #48)

So, what exactly needs changing configuration wise?

Changing $wgEnotifWatchlist to true, and leaving $wgShowUpdatedMarker as false
for the moment. And then looking to set $wgShowUpdatedMarker to true at
somepoint in the near future?

I think you can set both to true.

wgEnotifWatchlist is now set to true for all non big wikis

wgEnotifUserTalk is still true everywhere. wgShowUpdatedMarker is still false for the big wikis

For the record.

  1. This means that on those wikis: users can now set the preference to receive notifications on watchlist changes, which is disabled by default both for new and old users; history markers are enabled for everyone.
  2. There's been no noticeable load or network increase on mchenry (mail relay) nor DB servers in the last 30 min.

Therefore I suppose we can now safely enable EnotifWatchlist on all wikis, as suggested by Tim (comment 20: markers have an immediate performance impact, enotif have an impact only with time or if defaults are changed), and then think of changing the defaults for preferences (see comment 19, bug 5220 comment 38).

I AM VERY *HAPPY*

It works for me, I tested the en:Sandbox with an anonymous edit. There was no significant mail delay between the change and the arrival of the enotif.

MANY THANKS to all the people who helped to achieve this goal, started in 2004. See you @Hackathon in Berlin ?

Let me quickly close that super "bug".

This is not fixed yet, we're still waiting for the last two steps:

  • wgShowUpdatedMarker on all wikis,
  • changing default (or not) for new/old/subset of old users,

both after a while to measure performance impact of each change.

(In reply to comment #53)

This is not fixed yet, we're still waiting for the last two steps:

  • wgShowUpdatedMarker on all wikis,

ok, wasn't fully clear. Anyway, a big step!

I am still very interested in live statistics of how many mails are generated and sent per time interval and per own userpage- respectively per watchlist notifs, perhaps someone can code such a thing ?

(In reply to comment #54)

(In reply to comment #53)

This is not fixed yet, we're still waiting for the last two steps:

  • wgShowUpdatedMarker on all wikis,

ok, wasn't fully clear. Anyway, a big step!

I am still very interested in live statistics of how many mails are
generated and sent per time interval and per own userpage- respectively per
watchlist notifs, perhaps someone can code such a thing ?

For now http://ganglia.wikimedia.org/latest/?r=20min&cs=&ce=&m=&c=Miscellaneous+pmtpa&h=mchenry.wikimedia.org&tab=m&vn=&mc=2&z=medium&metric_group=ALLGROUPS and divide network by 3 KB? :) That would make about 10 emails per second.

(In reply to comment #53)

  • changing default (or not) for new/old/subset of old users,

both after a while to measure performance impact of each change.

By the way, changing the default only for new users requires just

$wgDefaultUserOptions['enotifwatchlistpages'] = 1;

to be put in the settings.

(In reply to comment #56)

(In reply to comment #53)

  • changing default (or not) for new/old/subset of old users,

both after a while to measure performance impact of each change.

By the way, changing the default only for new users requires just

$wgDefaultUserOptions['enotifwatchlistpages'] = 1;

to be put in the settings.

Yes.

Or run php userOptions.php with the appropriate settings, see https://www.mediawiki.org/wiki/Manual:UserOptions.php , to change that for _all_ existing users.

(In reply to comment #57)

(In reply to comment #56)

(In reply to comment #53)

  • changing default (or not) for new/old/subset of old users,

both after a while to measure performance impact of each change.

By the way, changing the default only for new users requires just

$wgDefaultUserOptions['enotifwatchlistpages'] = 1;

to be put in the settings.

Yes.

Or run php userOptions.php with the appropriate settings, see
https://www.mediawiki.org/wiki/Manual:UserOptions.php , to change that for
_all_ existing users.

No, there is no point on changing all existing users settings. If they want the feature, they can enable it.

Or run php userOptions.php with the appropriate settings, see
https://www.mediawiki.org/wiki/Manual:UserOptions.php , to change that for
_all_ existing users.

No, there is no point on changing all existing users settings. If they want the
feature, they can enable it.

oops, you misunderstood me, and I was not clear. I wanted just to inform about the possibility, not to request that.

Nemo_bis and I had an IRC conversation just now where we hashed out how this could be enabled by default for new users (but not existing ones):

  1. Add (or hack in) a MW config var that makes it so the enotifwatchlistpages preference is visible, but not functional (i.e. enabling it doesn't actually cause e-mails to be sent). This will stop watchlist e-mail notifs (similar to disabling $wgEnotifWatchlist) but preserve the preference
  2. Enable this config var
  3. Compile a list of users that have the preference enabled
  4. Set $wgDefaultUserOptions['enotifwatchlistpages'] = 1; . The preference is now enabled for all users, but because it's not functional, there is no deluge of e-mail notifications overwhelming our mail system
  5. Using a maintenance script, set the preference to 0 for each user individually, excluding users that already had it enabled (from the list made in step 3) and invalidating the user cache for each user that it touches. Running this will probably take a while (a few hours?) on a large wiki like enwiki
  6. Undo step 2 (disable the config var). The preference is now functional again and enabled for the same set of users, but it will now also be enabled by default for new users

(In reply to comment #60)

Nemo_bis and I had an IRC conversation just now where we hashed out how this
could be enabled by default for new users (but not existing ones):
[snip]

Is it not possible to repair the user_preferences table directly by converting old-style preferences to new ones? Or is the problem that old style preferences don't appear in the table at all, in which case could they not be created directly?

(In reply to comment #60)

Nemo_bis and I had an IRC conversation just now where we hashed out how this
could be enabled by default for new users (but not existing ones):

  1. Add (or hack in) a MW config var that makes it so the enotifwatchlistpages

preference is visible, but not functional (i.e. enabling it doesn't actually
cause e-mails to be sent). This will stop watchlist e-mail notifs (similar to
disabling $wgEnotifWatchlist) but preserve the preference

  1. Enable this config var
  2. Compile a list of users that have the preference enabled

I am guessing there is a time delay between 2 & 3 so that the community can rush and change their preferences.

This strategy sounds like a nightmare for the communication plan.

Also, the most important aspect of this implementation is enabling this feature for the existing users who haven't been active, but may be active again if they are sent emails.

If this is only enabled for users who explicitly enable it, and new users post implementation date, it is a missed opportunity to reengage existing low volume users who havent been active.

There are useful vars that could be added, easily, to control deployment.

A conf variable to specify the max number of watchlist-entries that users may have, after which enotif is disabled for them.

A conf variable to specify the max number of watchlist emails that will be sent per day to each account, after which they are sent an email saying their notification threshold has been exceeded.

These numbers could be increased incrementally on the server side when the devs are confident that the servers can handle additional load, up to levels that are optimal.

Or we could implement a digest mode (bug 8911) with a new user config var (Normal vs. Digest) and set everyone with lots of watchlist entries to Digest mode.

(In reply to comment #62)

(In reply to comment #60)

Nemo_bis and I had an IRC conversation just now where we hashed out how this
could be enabled by default for new users (but not existing ones):

  1. Add (or hack in) a MW config var that makes it so the enotifwatchlistpages

preference is visible, but not functional (i.e. enabling it doesn't actually
cause e-mails to be sent). This will stop watchlist e-mail notifs (similar to
disabling $wgEnotifWatchlist) but preserve the preference

  1. Enable this config var
  2. Compile a list of users that have the preference enabled

I am guessing there is a time delay between 2 & 3 so that the community can
rush and change their preferences.

No. EnotifWatchlist is already enabled, so users have set their preference before point 1.

This strategy sounds like a nightmare for the communication plan.

Also, the most important aspect of this implementation is enabling this feature
for the existing users who haven't been active, but may be active again if they
are sent emails.

The point is to change the default at least for new users to have a sane one for the future while overcoming the initial spam-flood risk for the very few super-users, and then think about who else add to the bunch.

Show updated markers is only now not enabled on enwiki and dewiki

In this comment https://bugzilla.wikimedia.org/show_bug.cgi?id=28026#c57 I already mentioned the possibility to run the maintenance script for setting the values. I think, you can set the option, where it has a certain value, to another temporary value, and apply your changes to the Settings. Later you reset to the value.

Sorry, I really lost track what problem you want to solve. Can we talk about this during the Berlin Hackthon ? I'll be there and (hopefully) can help to fix this finally, then.

This is all done now. If there is a want to change the defaults (a la bug 36316), open a new bug for that.