HomePhabricator
Improvements to Email QA
Especially useful in a mobile world

INTRO
When running QA on emails, the most important things to check are the dynamic content populating correctly for different user types, and ensuring there are no layout issues. Especially in an increasingly mobile world, it's important to review how an email will be displayed on a large variety of devices at once.

OLD PROCESS
In the beginning of our email QA days, the process was simple and very manual, and only allowed for checking how emails were displayed on a few devices. The email marketing platform we use is called Acoustic, which allows us to populate emails with dynamic content and create a storyline of emails to be sent (for example: users that donate on the first email, will not receive any further emails). Inside each email template, we would select send out test email, and select a given test user to send it to. This was a cumbersome process that left a lot of space open for human error, and only resulted in us checking content manually on our own clients.

NEW PROCESS
Recent improvements to email QA software provide testers with a quicker way of reviewing email layout and dynamic content on a wider variety of devices. This is particularly useful when performing QA for email on mobile devices, where screen sizes vary much more than on desktop devices. A new tool we added to our email testing suite, called Email on Acid, allows us to do this in just one click in Acoustic. The Email on Acid tool provides an email, which can then be input in Acoustic, and after just a few minutes, it will display how the email looks on Desktop, Mobile and even Web clients. This is a powerful tool that helps catch even the most edge case errors that may appear on less common devices, which is especially important here at Wikimedia given the our platform is used by so many users around the world.

image.png (799×1 px, 1 MB)

CONCLUSION
In an increasingly mobile world, it is becoming even more important to have an email QA system that allows for checking how emails are displayed on a wide variety of devices quickly. Email QA has always been a more niche area of QA with fewer specialized tools, but that is changing now and tools such as Email on Acid are a great step in that direction! I havent been able to find any open source alternatives yet but if anyone has any suggestions please add them to the comments!

Written by jbolorinos-ctr on May 7 2021, 4:50 PM.
QA Specialist
Projects
Subscribers
zeljkofilipin, jbolorinos-ctr
Tokens
"The World Burns" token, awarded by zeljkofilipin.

Event Timeline

A few more screenshots would be a good addition to the post. For example, how creating e-mails in Acoustic looks like, and a how you were checking e-mails before Email on Acid.