Page MenuHomePhabricator

Problem with email opt-outs reaching Civi as unsubscribes
Closed, ResolvedPublic2 Story Points

Description

The donor in Zendesk 617624 CID=35666251 chose No on the donation form about subscribing their email address on October 2nd:

but they are still not opted out in Civi from fundraising emails.

Two older examples (which we have since manually unsubscribed) in Civi are 597988 (CID 34737047) and 597983 (CID 1121306).

What is the process for the preferences of people who opt out of fundraising emails reaching Civi? Is there a lag or delay in them being unsubscribed automatically, and if so, how long is it?

If this was caused by a glitch or temporary error, would it be possible to batch unsubscribe everyone who was affected?

Errors from this process alienate donors and complicate our workflows. For legal reasons and general transparency, having the opt out process be efficient is important.

Details

Related Gerrit Patches:
wikimedia/fundraising/tools : masterSilverpop export: unsubscribe donors with opt_in=0

Event Timeline

MBeat33 created this task.Oct 8 2019, 1:41 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 8 2019, 1:41 PM
mepps added a subscriber: mepps.Oct 8 2019, 4:49 PM

@MBeat did you already manually update this contact? Just curious as I'm investigating.

Thanks @mepps CID 35666251 we did not update manually, I am pretty sure.

mepps added a comment.Oct 8 2019, 4:54 PM

@MBeat Thanks! I see opt-in listed as no, and my expectation is that that's what that form maps to, but do you think she would also get "On Hold" on her email address? I don't see "is_opt_out" available on her record as this documentation would suggest. https://www.mediawiki.org/wiki/Fundraising_tech/CiviCRM Which specific field are you looking for, just so I can start looking.

MBeat33 added a comment.EditedOct 8 2019, 4:59 PM

Thanks, @mepps that cid says in Communication Preferences NO BULK EMAILS (User Opt Out) as Unchecked, but I also do see what you see, Opt-in=No in the Communication block.

  • maybe it's a matter of reconciling those two fields?
  • can logs tell when the Opt-in=No processed in Civi? The donation date was October 2nd, 2019 7:14 PM and the TY email was sent October 2nd, 2019 8:45 PM. It would be great to know what the expected timeframe is.

Also, I think the On Hold designation may be unrelated to the opt out, I think that's a measure of whether they received the TY email.

mepps added a comment.Oct 8 2019, 5:47 PM

@MBeat it looks like the contact was created on 10/2 and the only change after that was the activity record.

mepps added a comment.Oct 8 2019, 6:11 PM

Basically the issue seems to be that not opting_in doesn't opt out of bulk mail, correct @MBeat? Did the donor get a bulk email? Also is this the behavior we want @CCogdill_WMF?

Definitely, the expected behavior is if someone opts-out of bulk email
while donating, they should be marked as opted-out in Civi and added to the
unsubscribes file that gets sent to IBM each night.

mepps added a comment.EditedOct 8 2019, 6:19 PM

Thanks @CCogdill_WMF! @Eileenmcnaughton was it intentional that saying "no" on opt-in doesn't lead to opt out or are they actually opted out?

Currently the field you see on that form is treated as 'opt in' - ie that person was a 'NO' to opt_in but not 'opted out' - as described.

The way that information looks in the silverpop export is it winds up transferred to Silverpop as 'latest_optin_response' & the expectation is that is respected at the Silverpop end as a filter.

I'm not clear why it was done that way - there is an implication that the 'latest' opt in response is less permanent than an opt out which is a one way street usually

Eileenmcnaughton added a comment.EditedOct 15 2019, 10:41 PM

Looking at the first example the code correctly marked that person as opt_in = NO - which I believe is enough to stop them getting emails - that information is exported to Silverpop. You can see the 'NO' on the left

Kristie then changed the to ALSO have is_opt_out / no bulk emails set to TRUE - as you can see on the right hand side of the screen shot.

I'm not clear if this is a bug, a change in specification or just a clarification as to what users can expect to see in the system

Previous related issues
https://phabricator.wikimedia.org/T225544
https://phabricator.wikimedia.org/T200941
https://phabricator.wikimedia.org/T200576

Huh, I remember us discussing that latest_optin_response field but I
thought we settled on opting these people out. I think that's what we need
to do--turn the opt out on the banner into a hard no.

Le mer. 16 oct. 2019 à 12:42 AM, Eileenmcnaughton <
no-reply@phabricator.wikimedia.org> a écrit :

Eileenmcnaughton added a comment. View Task
https://phabricator.wikimedia.org/T234925
Looking at the first example the code correctly marked that person as
opt_in = NO - which I believe is enough to stop them getting emails - that
information is exported to Silverpop.
Kristie then changed the to ALSO have is_opt_out / no bulk emails set to
TRUE - as you can see on the right hand side of the screen shot.
F30731588: Screen Shot 2019-10-16 at 11.39.51 AM.png
https://phabricator.wikimedia.org/F30731588
I'm not clear if this is a bug, a change in specification or just a
clarification as to what users can expect to see in the system
Previous related issues
https://phabricator.wikimedia.org/T225544
https://phabricator.wikimedia.org/T200941
https://phabricator.wikimedia.org/T200576
*TASK DETAIL*
https://phabricator.wikimedia.org/T234925
*EMAIL PREFERENCES*
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
*To: *Eileenmcnaughton
*Cc: *DStrine, Eileenmcnaughton, mepps, Aklapper, Pcoombe, krobinson,
CCogdill_WMF, MBeat33, EBjune, spatton, EWilfong_WMF

OK, I see two options:

  1. change the queue consumer to record the opt-in: NO response as both NO in the opt-in field and YES in the opt-out field. This will affect future donors but not those who responded NO to opt-in banners/payments-form radio buttons in the past.
  2. change the Silverpop export to put opt-in: NO donors in the unsubscribes file rather than in the databaseUpdate file. The latest_optin_response column would remain in the databaseUpdates file, but the file would only contain donors with 'YES' or a blank value for that column.

@CCogdill_WMF and @MNoorWMF, what's your preference?

I think my preference is option 2. I know we were concerned about people
who opted in NO and some point and changed their response to YES in the
future. It seems easier to me to set up an annual workflow where we remove
anyone from the unsubscribes list who has changed their response to YES.
Does that seem sensible?

Ejegg added a comment.Fri, Oct 25, 1:53 PM

Thanks for the guidance @CCogdill_WMF . Should be easy enough to implement!

Change 546186 had a related patch set uploaded (by Ejegg; owner: Ejegg):
[wikimedia/fundraising/tools@master] Silverpop export: unsubscribe donors with opt_in=0

https://gerrit.wikimedia.org/r/546186

Ejegg claimed this task.Fri, Oct 25, 3:49 PM
Ejegg triaged this task as Normal priority.
Ejegg set the point value for this task to 2.

Change 546186 merged by jenkins-bot:
[wikimedia/fundraising/tools@master] Silverpop export: unsubscribe donors with opt_in=0

https://gerrit.wikimedia.org/r/546186

@CCogdill_WMF and @KHaggard: I've just deployed this, so you'll probably see a jump in the Unsubscribe list tomorrow morning. There were 562,474 rows in the export view with latest_optin_response='NO', so expect around that number of new unsubscribes.

Thanks for the heads up @Ejegg ! It definitely did spike this morning :)

Ejegg closed this task as Resolved.Tue, Nov 12, 9:12 PM