Page MenuHomePhabricator

Emails refer to out-of-date/inappropriate donation amounts
Closed, ResolvedPublic

Description

One of this year's fundraiser emails - the "I’ll keep it short" email - was sent to people who donated in the past and made specific reference to the amount that they previously donated.

However - in at least two cases that I can prove the amount named in the email is inappropriate. They're factually correct, but inappropriate. They indicate that the usage of the donor database to create mass-emails is not being done with sufficient care to thinking about the different kind of people who will receive it. I only know these two examples, but both are probably circumstances that have many thousand equivalent situations.

Example 1: "A year ago, you gave 0.00 € to keep Wikipedia online and ad-free."

IMG_1062.png (1×750 px, 206 KB)

This person has donated in the past, several years ago, but did not donate last year.
This text comes across as either passive-aggressive, or just plain silly. At the very least it certainly does not encourage the recipient to donate again. Presumably there are MANY people in the donor database in a similar circumstance - I wonder how many thousands of emails were sent thanking people for their donation of zero? People who have donated in the past, but stopped, are a particular sub-set of the donor database that needs to be written to with particular care (or perhaps not at all!) They should not be included in the same generic email that is sent to all current donors.

Example two: "Two years ago, you gave $5 to keep Wikipedia online for hundreds of millions of readers."

IMG_1063.png (1×750 px, 229 KB)

This person donated $5 two years ago, but donated more last year.
It is possible that the donor is associated with different email addresses/creditcard numbers in your database, but in that case, they should have received two emails with the same subject line - they only received this. As a result, it makes the donor feel that their larger donation in the last fundraiser was lost, or that you're trying to guilt them by reminding them of their previous less-generous donation.


In both of these cases the problem arrises from making incorrect assumptions about people's donation history over [at least] two years. Perhaps a matrix of different donor circumstances could be created, and different email texts crafted to suit each. Options could include:

  • people who donated in the past, but not last year (could be many reasons for this, can't prejudge why).
    • a subset of that is people who donated over several years (or had a recurring donation set up), but stopped.
  • people whose donation amount grew from year to year
  • the reverse, people who donated more in previous years, but less in more recent years.
  • ....?

Finally, if it's possible to identify all the people who received the 'zero' email, it might be worth sending an apology email. At the very least we should never be sending emails that say, in effect, "thanks for nothing".

[I should add that both recipients of these emails gave me permission to anonymise them and post them in a bug report]

Event Timeline

Wittylama raised the priority of this task from to Needs Triage.
Wittylama updated the task description. (Show Details)
Wittylama added a project: Fundraising-Backlog.
Wittylama subscribed.
Wittylama set Security to None.

@Wittylama thanks for the report. We definitely tailor our emails, though some of what you're suggesting may be beyond our current technical feasibility because of the state of our database. Also, if it's at all possible for you to privately share the email addresses, we can hunt these down more specifically (breadcrumbs are really helpful).

Example 1) That 0 amount email is definitely a problem - @CCogdill_WMF do you think this was resolved with T107045 or T117931? Can we dig into how many people this may have effected?

Example 2) that's happened since any changes we've made, so if there's a technical issue it's likely still out there. For this one particularly, the email address would be really helpful so we can inspect our database and the rules around emails to see what happened.

@MBeat33 seen any similar reports?

We definitely had a problem with the 'last donation date' and 'last donation amount' fields used in the mail merge, but I thought we fixed that on 11/23. The email pictured from 11/25 *might* be from the tail end of an affected mailing. @Wittylama, do you know of any bad emails since then?

@Wittylama, do you know of any bad emails since then?

No. Apart from the other email problem regarding the French translation of "let's end this" (which I believe the fundraising team already knows about), these are the only two *specific* examples I'm aware of with problems.

Thanks for the report, @Wittylama. @atgo hit it pretty spot on:

Example 1) That 0 amount email is definitely a problem - @CCogdill_WMF do you think this was resolved with T107045 or T117931? Can we dig into how many people this may have effected?

The email you're showing from 11/3 was the one of the first two emails we sent with our "verify past donation" copy. Following this email, a donor reported the $0 issue and we corrected the logic in the email personalization so that only donors with a latest donation > 0 would see that copy. In short, this has been resolved and this email was the last one that had the error you're seeing. We can dig into how many people were affected, although our data has changed since 11/3 with T107045.

For example 2, you are seeing the expected behavior. The database field we use is latest_native_amount, so it is intended to show the most recent donation. As Anne mentioned, our data is far from perfect. We have a relatively high proportion of duplicate records in our system which means we don't always have the donor's most recent record in our email database. To us, the most important thing is that we are displaying accurate donation data, which is why we don't say "this was your last donation" and instead simply refer to a past donation. I know our tech team is hoping to tackle the duplicate issue this FY (T77938).

We are keeping a close eye on donor comments regarding this copy - complaints have been very low - and saw a 22% improvement in donations by adding the donation verification in the email.

Thanks again for your suggestions, @Wittylama. Many of the things you suggest (see below) are high up on my wishlist, but until we have more faithful data, it doesn't make sense for us to segment our list in this way:

    • people who donated in the past, but not last year (could be many reasons for this, can't prejudge why).
  • a subset of that is people who donated over several years (or had a recurring donation set up), but stopped.
    • people whose donation amount grew from year to year
    • the reverse, people who donated more in previous years, but less in more recent years.
atgo claimed this task.

Closing this one and noting this task as a symptom in the dedupe task: T77938 Thanks again, @Wittylama