All REPLY MAIL BLOCKS are transient and should not result in an opt-out. Once that is fixed, we need to figure out how to identify records Civi has opted out as a result of this issue. We might want to consider opting them back in.

This user was opted out by Civi, not Acoustic. Looking at the Opt Out Details field, the value is Opt Out Via ListImport. The timestamp is 11/22/2024 9:02. It appears Civi is opting out users with REPLY MAIL BLOCK, which is not something it should be doing. Transient bounces are separated into Soft Bounces and Mail Blocks, based on teh details of the message.

That is the domain for SFTP (transfer-campaign-us-4.goacoustic.com), which resolves to 3.209.96.38. I don't know what transfer.silverpop.com is or where that came from. Does that answer the question?

I bet the date is still there technically ... but the status is changed. You could check the master suppression list, yes.

On the second record, they opted out after snoozing, as the record in the Acoustic database is Opted Out as of this moment. What were the EVENT_TS on the other events? I'm guessing they were a few seconds after this? Also, you could opt-out and then later Snooze, but you would still be opted out. So could have happened in that direction as well.

This is the expected behavior, as the users signed up for RML emails. They have to be opted in order to be sent the RML email, which they signed up for. They must not have donated until after the RML emails, so that makes sense.

And I forgot... hard bounces also remove someone from the list. Yes, they are on the MSL. There is almost more data for those records on the opted-out record on the database. Opt-Out Details. Depends what level of detail you want on the opt-out. These are also in the raw data exports you get.

The ways a user can unsubscribe are (1) by clicking the linking and opting out on the page, (2) hitting Spam in Yahoo/Comcast (which are the only two mailbox providers with feedback loops) which is sent back to Acoustic via email, (3) Hitting Unsubscribe in the email client interface (Gmail, Yahoo, iOS, etc) which posts an opt-out call back to Acoustic, and (4) opted-out in Civi by hand.

I actually think you have done this before, on the unsubscribe page when you all used to host it many many years ago, we passed these values over and you would unencode them. I don’t where that was.

Correct.

It will come over encoded. That code is correct to convert it.

It’s been this way in email forever. I don’t know why.

When you search a record, you click Contact Insights on the right side. On the next page it lists Actions. This user filled out RML on 11/18 and 11/25.

Snooze is not a field you can query on.

I don't think there is an issue with the API calls either, the users are signing up to get emails after they Snooze. In order to get the emails, they are unsnoozed.

To document this, see https://help.goacoustic.com/hc/en-us/articles/360043235173-Create-a-web-form

I'm now recalling something.... we've had this conversation before. Maybe 6-7 years ago... yes, forms will un-snooze records as the user is opting back in. Which we have to do, or we can't send them the RML reminder they asked to signup for. I don't think there is an issue here.

Can someone provide a timestamp in GMT for a failed
API call?

Can you give me the timestamp of the latest Snooze that failed. We still need to get the API calls from logs so we can see why they fail.

I wonder if that was wrong, as UpdateRecipient is the call used, correct?

Thanks. Opening a ticket now. Did this API call get updated for a Flexible Database type?

Do you know the IPs this API calls are made from?

We need the API logs, the API call is failing.

The best way to figure this out is to run a Values report on the database. Open the database, click Values. That said, I did it already, results below. The value is "financial_reasons".

What do you logs say about the API call? The API call failed, that data will be on your end I suspect.

Don't Snooze that record, so we can figure out what's wrong with the API call. What logs do you have for that Snooze API call? (oh.... one theory, the record was Snoozed, you tired to re-Snooze it, the Snooze expired, and then they got the email). I don't know if a re-Snooze resets the Snooze period... that seems the less likely option, and the API just failed.

For CID 64598559, I see the Snooze on October 23rd by the user on the unsubscribe page.

So this is the error:

Can you provide the files names when you made the call? This was under user silverpop-upload@wikimedia.org, correct? And can you add the error response here as well. Thanks.

You can do that, but that will break the current recurring import. I would make this change when you switch to the Flexible Database type. After you turn off the recurring imports, you can rename the fields in the database. Then when you turn that back on (or move to the API-based import), the field names will match.

As for what calls I think need to be updated:

You would want to loop through all three statues to get all CAMPAIGNS (aka, program/automation emails).

I took you files and ran a job in Postman, and it was successful. The only thing I updated was the LIST_ID in the XML file.

Yes

Yahoo's interface confirmed and new feedback loop is setup.

Yahoo has started using this new process as of last weekend. As this DNS entry is not in place we can't setup the new feedback loop with Yahoo, so I've suggested to @Noah and @spatton they not send emails until this is in place.

Correct.

Ah... I'm only setting this up for the domain wikimedia.org with the DKIM selector of spop1014 (which is Acoustic).

There is no data access in Yahoo, and there is no way in Sender Hub to share that. The feedback loop emails are sent to Acoustic for processing. This is just setup, in which we provide proof of domain ownership, and then tell Yahoo where to email the feedback loop emails. This only applies to emails sent form Acoustic, if you ever wanted to setup FBLs for other senders on the domain, you can do that in a separate Sender Hub account.

https://senders.yahooinc.com/complaint-feedback-loop/ and here https://blog.postmaster.yahooinc.com/post/750658900050706432/its-here-meet-the-new-sender-hub-dashboard

This will be in Message Digital's Yahoo Sender Hub. There will be no verification email to postmaster@wikimedia.org, just this DNS entry.

Let's move that conversation to another ticket, once the texting contract is signed. Then get back to this ticket after that.

So you will be moving to an non-email keyed database as yo are adding SMS/MMS to Acoustic. That won't change much about this project, but I might wait until that is complete. The only real difference for this part is you will sync on your Civi CRM, not email address.

Confirmed access is working

So suspect I know what happened, it struck me you said the email is not in Acoustic anymore. My guess is Civi merged two records - and in the process deleted the record the email was sent to. (which is a bigger issue if you are removing emails from the list, but that's for another ticket) When the user clicked the link the personalization failed, as the record no longer exists. I suspect this is pretty edge case then. But, you could turn off link tracking on these emails, which would then populate the URL in the email itself at the time of send. That way even if Civi updates the record in Acoustic, the link in the email would be static. Is that change warranted for this kind of edge case I don't know, as you will lose link tracking data.

No humans ever see the text version, so it's not that. If if they seeing the text version they are using some kind of crazy mail client that could be alerting links.

We have access to Google Postmaster Tools, this is just with a new account that will allow us to send automated alerts based on a Spam Rate threshold, and alerts if Domain reputation changes or their are Delivery Errors.

In theory, yes you could do this, but it would require updating to a non-emailed keyed database. This would be a huge undertaking. Might be worth an investigation on your part, but would require a significant re-architecture of your integration and email program.

Those three fields are updated by Acoustic nightly (under Settings -> Organization Settings -> Automated Behavior Updates). You can export these fields as you can any other field.

More context. There are three types of suppressions (System, Org, Mailing). System suppressions are domains that are suppressed on all of Acoustic as they are not real, spam trap, generic garbage. Org level are the Master Suppression List, and Mailings level is when someone added a Suppression List at the time of send.

This was related to something @KHaggard sent me an email on, and that a large suppression list was added at the time of send.

Turns out it's not possible to export specifically the snoozed contacts. You have to ExportList and download the entire database, they are included and the "Opted Out" column will be populated with an S. It might make more sense to add Snooze to your unsubscribe page/preference center and go from it that direction.

In Acoustic under Show Additional Details within the database, the number of Snoozed records is 4,507. In the web interface, exporting SNOOZE only is an option, so suspect we just need to know the correct element name. I'm asking what that is.

The user you export data with is set to GMT. I checked data in some of our accounts and it's being exported as GMT. Can you provide some examples where you are seeing PT in the exported file? Is this something recentlly that changed or it's been like this for years?

@Eileenmcnaughton Not having an email in the upload should not have stopped the Contact List from being built. What API calls are you making and with what parameters?

No real person should have that as their email address, I would leave it and opt the user out.

Something struck me on how you can get this data. If you run this API call each day:

Both, you can see who is snoozed in the UI and via an API export - but you can't get the date.

You can export the current list of Snoozed records (which is currently 15,882) but not the date they will become un-snoozed.

SENT_CANCELLED does not do anything with GetSentMailingsForOrg. What is the mailing ID of the canceled mailing?

Your data retention policy is set to 450 days, you can see this under Organization Settings -> Archive Settings. I'll have to test that specific question. I would suggest you should cap all data at 90-120 days, anything after that is likely not real user activity, and has no chance of substantially changing the data. Frankly, I suspect if you had a cutoff of 60 days (or even shorter) that would capture 99.8% of all your data.

I'm hoping once you get rid of all the extra databases, this issue will no longer occur. As this seems to never happen with the main database. There is no reason to open Support tickets on these as they are aware of the issue.

Same issue as before. As this only happens with the non _All wiki DB, I'm hoping once these other DBs are removed this issue will be resolved.

I found an email from 2015 wher eI outlined this API call. I forgot there is an unpublsied elements called SENT_CANCELLED. I'll reforward the email.

Ah yes, remove <SENT/> and that will return all types, included Canceled.

Add List ID 9574332 and that should do it.

Ah... ok. So let's look at the specific API call. Need the full XML. Thanks.

(a) what is Superset? (b) what is the API call you are making, do you have the XML?

I have opened a case up on these two jobs.

No logs are stored more than four days (as that would not be GDPR complaint). I suspect you see the SUCCESS within 1-2 mins every time? Can you setup an alert that if don’t get that say within an hour, it sends us off an email?

Data job results are only retained for three days, so I can ask, but don't expect there is more infomraiton.

With an average of 68 GDPR data calls in the last three days (and the numbers are sequential) , that would make that ID around 50 days ago. Does that make sense?

That ID is a bit confusing. The current GDPR data job are IDs 33341 -> 33546, which all ran in the last three days and all are marked SUCCESS. Thought maybe this was a datajob ID, but that range is 173051574 -> 173776347. What was the API call make with this ID and when?

A 3% is normal for Verizon (aka, AOL/Yahoo). When I see blocks at Verizon it will be 100% soft boucning.

You want to build this report by domain group, not domain. Example, RoadRunner no longer exists, it is now part of Spectrum Communications. I will build a mapping for domains -> domain groups to use.

Each GDPR job has several subtasks. The job may get stuck after completing the subtask. This is a known issue. If a GDPR job does not complete within 60 minutes, just let me know and we can dig into it.

I'll have to ask if there are more details on the failures. When you get one that failed, are you able to re-run it? Did you mean the two from July 2019?

Also, I recommend you merge all your data into a single database to simplfy GDPR requests. There are seven databases right now, and ten suppression lists.

Reading back over the previous notes, wanted to mention "I realize RML contacts should have civi contacts" - there are several hundred thousands RML signups that are not in Civi. Some are historical, and some are recent. I'm unclear how you are getting RML contacts out of Acoustic into Civi, but happy to help debug that if you want.

Can we take one step back and sanity check that the Civi database has all the data from the Master Suppression List? I'm assuming (and this might be wrong) that there are emails on the Master Suppression List that are not marked as opt-outed in Civi. When you export the MSL, select the “Export Contacts, Opt-Outs, Snoozes and Undeliverables” option.

Rebuilding the Master Suppression List is very risky, and I would not recommend it. Anyone on that list should never be emailed. If Civi were missing any data, that could be catastrophic to your deliverability.

"there's a way to import the list of suppressed contacts to the main db with opt-out selected so that it just updates the contacts that are already in the main db" - Are you referring to Civi or the Acoustic database? For sure you can keep this all clean in Acosutic, but over time it will get out of sync again.

A few notes to add. You can do anything mentioned here with the API, it's just up to you how you want to structure it. The link above about CRM integration does not apply here (that is for Acoustic accounts that sync with SFMC or MS Dynamics).

IBM validated the DNS setting, and is using the new key:

The key is correct, and IBM will validate it before turing it live. I would recommend in four months, you remove the old key.