Page MenuHomePhabricator

Goal: Deploy Newsletter extension in Wikimedia
Closed, ResolvedPublic

Assigned To
Authored By
Qgil
Aug 25 2015, 9:18 AM
Referenced Files
F5165664: pasted_file
Dec 27 2016, 11:32 PM
F5165693: pasted_file
Dec 27 2016, 11:32 PM
F5165683: pasted_file
Dec 27 2016, 11:32 PM
F5165671: pasted_file
Dec 27 2016, 11:32 PM
Tokens
"Yellow Medal" token, awarded by TheDJ."Like" token, awarded by Aklapper."Yellow Medal" token, awarded by Qgil."Like" token, awarded by Elitre."Like" token, awarded by Keegan.

Description

Now that T76199 is complete, let's go for the deployment of the Newsletter extension in Wikimedia.

The process is described at https://www.mediawiki.org/wiki/Writing_an_extension_for_deployment

Let's create subtasks for the main steps of the process, and let's define other tasks like bugs or missing features that are blocking this goal.

Related Objects

StatusSubtypeAssignedTask
DuplicateQgil
ResolvedQgil
ResolvedQgil
DeclinedNone
ResolvedAddshore
ResolvedTinaj1234
Resolved01tonythomas
DuplicateTinaj1234
ResolvedTinaj1234
ResolvedTinaj1234
DuplicateTinaj1234
Resolved01tonythomas
ResolvedTinaj1234
ResolvedTinaj1234
ResolvedTinaj1234
ResolvedTinaj1234
ResolvedTinaj1234
ResolvedTinaj1234
ResolvedTinaj1234
Resolvedsiebrand
ResolvedTinaj1234
ResolvedTinaj1234
ResolvedTinaj1234
ResolvedTinaj1234
Resolved01tonythomas
ResolvedTinaj1234
Resolved01tonythomas
ResolvedTinaj1234
ResolvedQgil
ResolvedGlaisher
ResolvedQgil
ResolvedGlaisher
ResolvedTinaj1234
ResolvedGlaisher
ResolvedTinaj1234
ResolvedGlaisher
ResolvedGlaisher
ResolvedParent5446
ResolvedTinaj1234
ResolvedMooeypoo
DuplicateNone
ResolvedQgil
ResolvedQgil
ResolvedFlorian
InvalidGlaisher
ResolvedGlaisher
DeclinedGlaisher
ResolvedGlaisher
ResolvedGlaisher
DuplicateNone
ResolvedQgil
Declined01tonythomas
ResolvedTinaj1234
Duplicate01tonythomas
DuplicateTinaj1234
ResolvedNone
Resolvedori
ResolvedBawolff
ResolvedGlaisher
ResolvedTinaj1234
DeclinedNone
ResolvedGlaisher
Resolved01tonythomas
Resolved01tonythomas
Resolvednikitavbv
InvalidNone
ResolvedFilip
DuplicateNone
Resolved01tonythomas
ResolvedFilip
ResolvedMtDu
ResolvedMtDu
ResolvedGeorggi199
ResolvedFilip
ResolvedPppery
ResolvedFilip
ResolvedPppery
ResolvedPppery
ResolvedGlaisher
Resolved01tonythomas
ResolvedPppery
ResolvedDaimona
ResolvedDaimona
Resolved01tonythomas
ResolvedQgil
ResolvedFilip
ResolvedxSavitar
ResolvedNone
Resolved01tonythomas
Resolved01tonythomas
ResolvedNone
ResolvedFilip
ResolvedFilip
ResolvedMtDu
Resolvednikitavbv
ResolvedQgil
ResolvedAddshore
Resolved demon

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Howdy, was wondering if I could get a status update on this from you folks. I remember being involved in some of the original trials (as a test user) way back in mid-2015. I admit I'm surprised this is still in seemingly active development: I expected that the GSoC projection would either complete then and there, or that the feature would eventually disappear into the bugtracker. So my inquiry: realistically speaking, what are the chances that this feature will push by, say, mid-2017?

Hi @ResMar, the road has been bumpier than expected but thankfully @01tonythomas and others have stuck around stubbornly. :) The extension works, but we had to address a couple of "little details" coming from the security review. While fixing those problems, we have tested and bugfixed many minor problems.

Thanks to Google-Code-In-2016 we have got several contributors. I really hope that at the end of this program in January we will be able to run the final sprint and deploy the extension in Wikimedia. I'd say optimistically before April, but definitely before July. I hope @01tonythomas agrees!

Thank you @Qgil. From the current pace, I am confident that we would hit production on target.

@ResMar: The major change we had was the shift to ContentHandler, which kind of make the extension super strong I would say - like you can create a newsletter just like you create a wiki page, and delete it, undelete it from the logs again and do it as many just like you play with wiki pages (these are not normal test cases, but the extension should handle all this). Also, moving a newsletter to somewhere else - like renaming and everything works just like in wiki page, and we get deeper logging of everything! We had to rewrite part of the extension for this to happen, but its in a good shape as of now afaik. You can test these at http://newsletter-test.wmflabs.org/

The technicals look good. I only have one question on this front: will this extension allow cross-wiki notifications? This would be a very important feature for such a tool to have.

The other questions I have are with regards to the social aspect of this extension. This extension would for the first time allow easy subscribe and unsubscribe options for publications on Wikipedia that are integrated directly into the interface. This has the potential to be a boon for any publication of note in the wikispace. So the immediate question, for me, is threefold:

  1. How is the WMF planning on promoting this feature to users? How do people find out about this feature?
  2. Where is newsletter management going to be in the on-wiki interface? Is this a new tab in Preferences? A new Special page?
  3. What are the prospects for enabling subscriptions by default? What about promoting the feature when e.g. a user registers an account (as a way of learning about the community)?

Number 3 is particularly interesting at this juncture. Suppose that every user on the English Wikipedia above a certain edit level gets a notification asking them if they'd like a Signpost subscription. The same can be done with e.g. This Month in GLAM on the GLAMwiki, or of the Kurier on the German Wikipedia.

Doing something along those lines, or otherwise being good at promoting the feature's arrival in the space, would greatly enhance this feature's profile on-wiki. Arguably even more importantly, it would enhance the profile of the publications which benefit from it, which have historically struggled with retaining casual readership (most people do not want their talk page get constantly hit by subscription spam) and retaining writership (the health of these publications stems from their ability to attract readers and talent).

The Signpost in particular, the biggest and oldest of these publications, is having a discussion right now as to what is to be done for the future.

Wow. Nice to read. I would only reply to this one, and wait for @Qgil to give the answers for the rest.

  1. Where is newsletter management going to be in the on-wiki interface? Is this a new tab in Preferences? A new Special page?

I will be using examples from the labs instance, which would give us links and to the point thoughts. As of now, we have the following interfaces:

  1. Listing of all newsletters: http://newsletter-test.wmflabs.org/wiki/Special:Newsletters
  2. for a publisher (or a person with 'newsletter-manager' right) will see the following at http://newsletter-test.wmflabs.org/wiki/Newsletter:CustomOne:
    pasted_file (532×1 px, 70 KB)
    . The menu allows to (a) add or remove in publishers
    pasted_file (699×756 px, 40 KB)
    and lately add or remove in subscribers (this is to help migration of existing newsletters)
    pasted_file (424×752 px, 21 KB)
    .
  3. Announce a new newsletter issue at http://newsletter-test.wmflabs.org/wiki/Special:Newsletter/1/announce - where a publisher can announce in a new issue, and all the subscribers get an Echo notification with a link to the new issue page
    pasted_file (309×776 px, 20 KB)

In case you want to look around, I can add you as a publisher, or give you the necessary rights on the labs instance.

Yes, please do. There is still one thing I want to see, which is what the notification will look like when a new issue is announced.

The other question above was whether or not this extensions does interwiki notifications. If it does, great, if it doesn't, then at this stage I would suggest working towards deployment as is, and then making it a high priority feature enhancement for the post-release, because I really think the extension needs to have that ability to be feature complete. (the Signpost, for example, has an interwiki mailing target page, so a bot would still likely be involved in mailing notifications if they're not crosswiki).

@ResMar , thank you for these very reasonable questions and suggestions. @01tonythomas' reply made me think that we need to T154222: Create Newsletter Extension Help page to address these questions about use of the extension.

Interwiki notifications are enabled in Wikimedia wikis, and they work for any extension providing notifications (like the Newsletter extension does). If a user subscribes to a newsletter hosted in Meta but only checks xx.wiki regularly, if a new issue is announced, they will see the notification in xx.wiki or whatever Wikimedia wiki they log in.

How people find out about this feature will ultimately depend on the own newsletter promoters (publishers, readers) and the wiki projects linking to them. Ideas are welcome.

There are no "WMF plans" to promote this extension as of today, and (to be clear) this is not even a WMF project. Several teams in WMF Community Engagement are interested in using this extension for their existing newsletters or new initiatives. I believe it will play a prominent role enabling specialized "checkpoints" for communication between Product teams and communities in the context of the Technical Collaboration Guideline. If you have suggestions for the WMF about promoting this extension, please let me know.

Automatic subscriptions and invitations is a use case that I have been thinking for a while, and your comments helped me writing down my thoughts finally. Thank you! See T154228: Subscribing users to newsletters automatically when certain conditions are matched

Yes, please do. There is still one thing I want to see, which is what the notification will look like when a new issue is announced.

Added you as an 'Administrator' at http://newsletter-test.wmflabs.org/

  1. How is the WMF planning on promoting this feature to users? How do people find out about this feature?

This is not a full answer to your question – I'm not working on this extension – but it would definitely be included in Tech News, which goes out to ~80 community pages and ~580 individual subscribers (and some other venues – sometimes included in The Signpost, at least one mailing list etc) in around 20 languages every week.

Now that we are done with the security review, I guess the DB improvements at https://phabricator.wikimedia.org/T159083 looks like a blocker for deployment as per https://phabricator.wikimedia.org/T159081#3203545

@01tonythomas Security review passed! The database problems were solved according to T159083#3284734.

Can... we... request the deployment of this extension in mediawiki.org (as per T116271#2344835)?

Qgil raised the priority of this task from Low to High.Jun 26 2017, 7:50 PM

@Qgil @01tonythomas If we pick a day or a week then I can probably help with scheduling & deploying the extension, (please ping me so I don't miss it) :D

Woho. We got the final blocker https://gerrit.wikimedia.org/r/#/c/355226/ too merged! Yay! Time to get this on prod!

Adding @greg, because it seems the right thing to do.

@Addshore it seems that we have (finally!) remove all the blockers for a first deployment in mediawiki.org. If you can help, great!

Change 362388 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/tools/release@master] Add Newsletter to make-wmf-branch

https://gerrit.wikimedia.org/r/362388

Change 362393 had a related patch set uploaded (by Addshore; owner: Addshore):
[operations/mediawiki-config@master] Add Newsletter to extension-list

https://gerrit.wikimedia.org/r/362393

Change 362394 had a related patch set uploaded (by Addshore; owner: Addshore):
[operations/mediawiki-config@master] Enable Newsletter on mediawikiwiki

https://gerrit.wikimedia.org/r/362394

Don't forget to document before releasing. :)

A glossary is a good first step, and then detail features and user cases using simple words. Some ideas, feel free to adapt it:

  • what is the purpose of this feature
  • user documentation:
    • basic user actions
      • how to un/subscribe to a newsletter
      • how the newsletter is received and how to handle it
    • advanced user documentation
      • how to create a new newsletter (translations?)
      • how to announce a new newsletter
      • how to send a new newsletter

Change 362388 merged by jenkins-bot:
[mediawiki/tools/release@master] Add Newsletter to make-wmf-branch

https://gerrit.wikimedia.org/r/362388

Don't forget to document before releasing. :)

We have documentation:

https://www.mediawiki.org/wiki/Extension:Newsletter
https://www.mediawiki.org/wiki/Help:Extension:Newsletter

If there are any problems, please create the corresponding new tasks, which at this stage should not delay the deployment. Thank you!

Don't forget to document before releasing. :)

We have documentation:

https://www.mediawiki.org/wiki/Extension:Newsletter
https://www.mediawiki.org/wiki/Help:Extension:Newsletter

If there are any problems,

T169872 (not a big thing, just did what you suggested)

I was not aware of that existing documentation, sorry!

Looks like next step is to get it on Beta Cluster?

Looks like next step is to get it on Beta Cluster?

Looks like it is already there

Neat! (Yes I was being lazy...)

Who'll be the shepherd to get this out in production? What's the rollout plan (staggered release on some easy/friendly wikis first?)?

Neat! (Yes I was being lazy...)

Who'll be the shepherd to get this out in production?

Good question. I'm more than happy to help in some capacity. @Elitre is someone available from our team to work with me on this? The load will not be time consuming at all. I'd be more comfortable with a partner for this for at least thinking through the plan, and helping represent the community POV when we communicate this.

What's the rollout plan (staggered release on some easy/friendly wikis first?)?

Initial thought(s):

I think the ideal place or places would be a wiki that has a few active newsletters in place that can shift over to the new system. Whether or not these wikis are the same as the usual "go-to" wikis willing to test extensions in production would be more a matter of coincidence, in this case. Without looking into it yet, I'm not sure which wikis this would apply to. I do know that they exist :)

After that, assuming all goes well, we'd ask the other wikis if they would like the extension turned on for them.

Additionally, we need to make sure what the goal or goals of the first production releases are as well. Aside from bug testing, at the moment I assume we're also checking to make sure that the documentation (which appears to be great!) is accessible in practice, and that the overhead of moving over to a new system is low enough for people to bother doing so. I certainly believe it is, the smooth transition from EdwardsBot to MassMessage could indicate similar ease here, but we won't know until we try.

I'm sure there's some more to work out, but those details can come after we organize.

As per T116271, the plan is to start deploying in mediawiki.org only. In that task there is a link to the community check.

As per T116271, the plan is to start deploying in mediawiki.org only. In that task there is a link to the community check.

Thank you. Looks good to me.

I agree with the comments at T116271#2147927, supporting deployment to Outreach-wiki in the very near future as well as a production testing ground.

As per T116271, the plan is to start deploying in mediawiki.org only. In that task there is a link to the community check.

As mentioned somewhere above I can probably help in getting us to this point.

Change 364734 had a related patch set uploaded (by Addshore; owner: Addshore):
[operations/mediawiki-config@master] Enable Newsletter on mediawikiwiki

https://gerrit.wikimedia.org/r/364734

Change 362393 merged by jenkins-bot:
[operations/mediawiki-config@master] Add Newsletter to extension-list

https://gerrit.wikimedia.org/r/362393

I should be able to schedule some deploy slots this week and get the ball rolling for test and mediawikiwiki

Change 370848 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/WikimediaMaintenance@master] Add Newsletter extension

https://gerrit.wikimedia.org/r/370848

Change 370848 merged by jenkins-bot:
[mediawiki/extensions/WikimediaMaintenance@master] Add Newsletter extension

https://gerrit.wikimedia.org/r/370848

Change 371114 had a related patch set uploaded (by Reedy; owner: Addshore):
[mediawiki/extensions/WikimediaMaintenance@wmf/1.30.0-wmf.13] Add Newsletter extension

https://gerrit.wikimedia.org/r/371114

Change 371114 merged by jenkins-bot:
[mediawiki/extensions/WikimediaMaintenance@wmf/1.30.0-wmf.13] Add Newsletter extension

https://gerrit.wikimedia.org/r/371114

Change 362394 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable Newsletter on testwiki

https://gerrit.wikimedia.org/r/362394

If you (yes, you!) want to help testing the Newsletter extension in test.wikimedia.org (last step before deployment to MediaWiki.org), please go to https://test.wikipedia.org/wiki/Special:Newsletters and...

  • Subscribe, unsubscribe, and play with your notification preferences.
  • Check whether you receive notifications as expected when a new issue of a newsletter you have subscribed is announced.
  • If you want to test more actively, consider requesting to become a publisher of an existing newsletter or request the creation of a new newsletter.

Remember, this is test.wikipedia.org, none of the data or content created will be ever transferred to production. Don't try to create a serious newsletter there!

Feedback: T173111: Testing Newsletter extension in test.wikipedia.org

Thank you.

Do we need it on mediawiki.org, or is that simply for testing beyond testwiki? I'd rather not install it if we don't need it (mw.org is not a playground :))

(This is not trolling or trying to sidetrack, just genuinely asking)

Yes, we need it in MediaWiki.org. Technical Collaboration has real use cases for newsletters with a technical scope, and this is the main reason why I have been promoting this project since its inception. The Community Tech team also has an idea for a newsletter and they are waiting for this deployment.

Once the extension is available and real newsletters start to be created and get subscribers, I bet more people will find good uses for their own projects and interests.

For the choice of MediaWiki as first Wikimedia wiki to deploy this extension, see T116271: Decide which Wikimedia wiki gets Newsletter extension first, and seek support.

For the longer term aspiration to have this extension enabled in all Wikimedia wikis eventually (and also the current limitation not to enable it in a second Wikimedia wiki), see T110645: Interwiki support for Newsletter extension.

Yesterday:

16:14 addshore: addshore@terbium:~$ mwscript extensions/WikimediaMaintenance/createExtensionTables.php --wiki mediawikiwiki --extension newsletter

Change 364734 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable Newsletter on mediawikiwiki

https://gerrit.wikimedia.org/r/364734

Mentioned in SAL (#wikimedia-operations) [2017-09-06T08:43:02Z] <addshore@tin> Synchronized wmf-config/InitialiseSettings.php: T110170 [[gerrit:364734|Enable Newsletter on mediawikiwiki]] (duration: 00m 51s)

Addshore claimed this task.

Thank you very much @Addshore and CONGRATULATIONS TO EVERYBODY INVOLVED!

It has been a long journey and, well, it has just started. :)