Project

Possible Mentors

• Primary Mentor: @Parent5446
• Co-mentor: @01tonythomas

Synopsis

HTML emails have a lot of advantages over plain text emails; being responsive to screen size and supporting fonts other than monospace (difficult to read for dyslexics) are some of the many pointed out by Tim Starling. In the same email, another serious issue he points out is that plain-text does not work well with bidirectional languages, since plain-text just follows visual order. In addition, certain clients fail to correctly identify links in plaintext, leading to serious inconvenience for the users.

Given the fact that MediaWiki has a very large and diverse user base, these issues really ought to be taken care of - there is no reason to continue using only plain-text emails given the benefits of HTML emails.

As of now, MediaWiki has support only for plain text email. A few extensions (such as Echo) have their own mechanism for sending out HTML emails, but core as such does not, which means any mails that core sends (such as change notification emails) cannot be sent as HTML Emails. This project plans to implement this functionality in MediaWiki's core.

Prior Art

MediaWiki’s core already has a provision for sending HTML email, as long as one is being provided to it. Some extensions using it are:

• Extension:Echo - This extension already has HTML email support implemented. Once core has support for this, the extensions can use it instead of writing their own.
• Extension:EmailPage - This extension sends rendered HTML pages to email addresses using PHP Mailer. This is different from what we want to do, as we do not want to be email wiki pages, we wish to convert a given text email into an html email.
• Patches - Some patches have been submitted trying to implement support for HTML email. We should probably be working with these and extending them.

Phases​

Note: Testing and review will be done as and when the phases are being developed.

• Phase I : The MVP
  • Feed text coming from wfMessage as is, i.e., assume the text given has no HTML tags in it.
  • Implement very basic plain text to HTML conversions:
    • Use nl2br to convert newlines to <br/> tags
    • Include links in <a> tags. Account for links being in brackets (()), in quotes (‘’ or “”) and any other format that we find people tend to mention links in plain text.
  • In order to reduce redundancy and allowing extensions to reuse/alter their existing HTML email support, a new global $wgCoreHTMLEmail could be added to allow Wiki Administrators to send plain text emails if they want to (send plain text email if $wgCoreHTMLEmail is set to false, and send multipart email if it is true) , besides $wgAllowHTMLEmail, which decides whether the Wiki allows anybody (core or extension) to send an HTML Email. To explain the different cases more clearly,
    • if ($wgAllowHTMLEmail == false) only plain text emails go
    • if ($wgAllowHTMLEmail == true and $wgCoreHTMLEmail == false) Anybody *except core* can send an HTML email.
    • if ($wgAllowHTMLEmail == true and $wgCoreHTMLEmail == true) Everybody can and core *will* send an HTML email.
  • The server will send emails to the client depending upon user preferences:
    • The user will be given a “Receive plain text email only” option in Special:Preferences. When this is ticked, the server will send only plain text emails to that specific user.
    • When the above option is unticked, the server will send a multipart email (both HTML and plain text) when both $wgAllowHTMLEmail and $wgCoreHTMLEmail are set to true.

• Phase II : Preliminary styling
  • Integrate templates from Zurb Foundation, decide on which ones to keep, which ones to leave.
  • Include branding of any kind that might be required
    • Add the official MediaWiki logo at the beginning of the mail.
    • A default sign-off message, perhaps.
  • Ask community for design suggestions, something like a wish-list

• Phase III : Wish-list and translations
  
  Some alternative implementations might exist that achieve what I am trying to in a better way. I will take some time to look around for those. For now, my proposed solution for translations is as follows:
  • When generating a key to be passed to wfMessage(), it will be assumed that if the key does not end with -html, the text is wikitext/plaintext. If the caller of the function wishes to include html in the text, the key must be appended by -html.
    • When the key sent ends with -html, we can run wfMessage($key)->exists() on it, and if it does not exist, we can use the key without the ending -html, assuming that version exists. This will be done so that even if an html version of the text exists and the caller wants to use the plaintext version, they have an option to do so. This will require discussion with the i18n team. Allowing arbitrary HTML in the messages will have security concerns that will need to be looked into and taken care of.
  • I will try to implement the necessary features that come up on the wish-list, and any other features that may be very useful, if time permits.

• Phase IV : Wrap-up
  • Robust testing, to see if any possibility of a security breach has not been accounted for.
  • Create/Edit the required documentation pages for this functionality.
  • Announce the introduction of the functionality on wikitech-l mailing list.

Deliverables

• The MVP: Ability to be able to send plain text as an HTML email with minimal conversions of plaintext to HTML. (Phase I)
• Tests and Patches: They will be submitted on gerrit, and will be available for review and testing to the community.
• Documentation: Explaining in detail how the functionality is to be used, in terms of variables, functions and classes.

Timeline

Timeline Details

Week 0 (Community Bonding Period)

• Go through the existing art for implementing HTML support for Emails, and discuss whether these should be built upon or a different approach would be needed.
• Decide whether implementing only one function would do the job, or a class with internal functions would be needed.
• Research on templates other than those provided by the Zurb Foundation, and whether they can and should be used instead.
• Look for other extensions that have their own implementation for HTML Emails and investigate their workflow for the same, and discuss how they are to be adapted to, if at all.

Week 1

• Implement a function transformContentToHtml (probably as a static method of a class, so HtmlEmail::transformContentToHtml or EmailNotification::transformContentToHtml) that accepts a string with the plain-text version of the body of the email and returns an associative array with both html and text fields that can then be sent to UserMailer::send.
  • Basic operations would be performed on the text, such as nl2br and putting links into <a> tags
  • Example:

HtmlEmail::transformContentToHtml("Dear User,\n\nWelcome to https://www.mediawiki.org"`);

would return

Dear User,<br />
<br />
Welcome to <a href="https://www.mediawiki.org">https://www.mediawiki.org</a>

Week 2

• Change transformContentToHtml to accept an i18n key instead of a string directly.
• Implement the $wgCoreHTMLEmail and add checks for it in transformContentToHtml.
• Investigate changing Special:Preferences to add a user preference for receiving only plain-text versions of email.

Week 3

• Add the user preference to Special:Preferences
• Add some preliminary unit tests in order to complete an MVP:
  • Test that the transformContentToHtml gives the correct output for an obvious input
  • Test the auto-linking part of the function by attempting to feed in complex URLs
  • Check that different combinations of $wgCoreHTMLEmail and $wgAllowHTMLEmail give the expected output

Week 4

• Look into the Zurb Foundation templates, decide which ones are relevant and whether any changes are required to the existing templates
• Select the appropriate templates and make any required modifications
• Investigate how the HTML templates will be put into the repository and then used by the code (plain HTML files? includeing PHP files with variables? Some basic templating engine?)

Week 5

• Mid-Term Evaluation to-dos :
  • Check whether weekly reports are up to date
  • Make sure MVP is complete, functioning and deployed
  • Any known bugs in MVP are either solved or filed
• Implement the chosen templating mechanism into the transform method, including passing of the template name (or some other specification) in the call to the method
• Update tests to include the checking the templating system
• Look into the benefits of caching template processing to increase efficiency

Week 6

• Look into branding, start a wish-list sort of thing for the community (and the design team) to suggest/request inclusion of certain things: logos, some kind of default signature
• Look for other options for templates, consider whether implementing and maintaining a custom set of templates for Wikimedia is necessary/required/desired

Week 7

• Include elements required from branding and update tests as consensus is reached on them
• Finalise reviews on whether implementing and maintaining a custom set of templates for Wikimedia is necessary/required/desired
• Put basic architecture to implement the custom set of templates if necessary/required.

Week 8

• Start discussions on better i18n and a security review
• Discuss methods of including HTML in messages without breaking any existing systems (one of the possible disadvantages mentioned in the original task) with the i18n team
• Simultaneously discuss security issues with the current implementation as well as with the proposed i18n implementation. Simple htmlentities or possibly something full-fledged like HTMLPurifier

Week 9

• Implement agreed-upon i18n solutions
• Fix any identified security loopholes, add any advanced sanitization that was agreed upon.

Week 10

• Add more robust tests (since the feature-set will be near completion)
• Check if there are any incompatibilities with the way this has been implemented in core and the way extensions want it
• Refactoring code to make it more readable or to improve efficiency, if the need is felt (readability and efficiency will be ensured as the code is written, but it may so happen that this is not heeded as much attention at all times)

Week 11

• Start documenting the new functionality on mediawiki.org
• Attempt to modify extensions that are currently sending HTML email
• Implement caching of template processing if its importance is agreed upon

Week 12

• Buffer week. Complete any pending tasks or attempt to implement any extra features that were suggested during any of the reviews.
• File bugs found, attempt to fix most of them.

Week 13

• Pencils Down to-dos:
  • Make sure functionality is complete, functioning and deployed
  • Check that weekly reports are up to date
  • Make sure any known bugs are either solved or filed
  • Code submission to Google
• Complete documentation of the functionality.
• Announce new functionality on wikitech-l and mediawiki-l.

Profile

Name: Smriti Singh
​Email: smritis.31095@gmail.com
​IRC handle (freenode): galorefitz
Internet Presence : MediaWiki user page
​Location:
I'll be visiting 2 places this summer, India (+05:30 GMT), and Kuwait (+03:00 GMT)

​Typical Working Hours:

1. May-July: 6-8 hours a day, between 6:30 a.m - 8:30 p.m (GMT) (at least 40 hours a week)
2. August : 5-6 hours a day, between 3:30 a.m - 11:30 p.m (GMT) (at least 40 hours a week)

Communication

I will be submitting reports weekly on a phabricator task, tracking the progress of the project, on Phabricator. I will be available on #wikimedia-dev and #wikimedia-tech on IRC (freenode) during my working hours, so I can be reached there. I will also be responding on the relevant phabricator tasks. The source code will be pushed to Gerrit, and will be accessible, viewable and reviewable there.

Contributions

Microtasks: T31856 T48109 T62434
My contributions to the community can be viewed here and here. I haven't contributed to any other FOSS organisations yet, but plan to in the future.

About Me

Education: In progress. I am a Computer Science Student (currently in my third year), studying at the International Institute of Information Technology (IIIT), Hyderabad, India.

How did you hear about this program? From senior year students in college.

Why MediaWiki? I started contributing to MediaWiki in May, 2015. The people here were quick to respond, very encouraging, and amazingly helpful. Contributing, seeing my work make a difference to the community, felt great. Not many communities have all of the above, and this is what encouraged me to choose MediaWiki.

Why this project? HTML emails have a lot of advantages over plain text emails; being responsive to screen size, supporting fonts other than monospace (difficult to read for dyslexics), supports bidirectional languages, unlike plaintext, which only supports visual order. Certain clients fail to correctly identify links in plaintext, leading to serious inconvenience for the users. Given the fact that MediaWiki has a very large and diverse user base, these issues really ought to be taken care of - there is no reason to continue using only plain-text emails given the benefits of HTML emails.

Additional Information (as mandated by Gnome)

Do you meet the eligibility requirements outlined here ? - Yes.

Preferred pronoun - she

Prior Commitments -
May-July, 2016 : None.
11th August, 2016 - 23rd August, 2016 : College, will take approximately 25 hours a week

Course details
From 11th August, 2016 - 23rd August, 2016 : My college has a system of electives for every semester, and so I am not sure right now what courses I’ll be taking and how many credits they’ll be. The total number of credits, though is 20 (normal, full-time course load is of 20-24 credits). For more information, please refer to https://www.iiit.ac.in/academics/curriculum/undergraduate/BTech-CSE (Year IV, Semester I)