Thu, Jan 16
Ooh thanks for all the digging, Eileen. That copy is from an RML email for
Dec 17 2019
Hmm, that mailing is a remind me later mailing. Those are special cases
because they're sent as automated programs on timers based on user
behavior, rather than as bulk sends. We use different reports to get the
performance data for automated programs. Maybe that's why you can't see the
Dec 6 2019
Dec 5 2019
Thank you! That looks good to me.
Dec 2 2019
Ah, there must be duplicate rows on that import or something. We show
3,065,196 in IBM now. Higher than when I last checked :)
@Ejegg nope, that sounds really high. We have a total suppression list of
about 2M -- 426k would be a 25% increase in our total suppression list.
Nov 27 2019
Thanks so much for all the digging, Eileen! Looking forward to seeing how
things look tomorrow. I'll be happy to research the mailings that you think
may be deleted or do any other QA on the ESP side. Let me know how I can
Nov 26 2019
Wonderful, thank you!
Nov 25 2019
They should be... we aren't able to access the files after upload.
And the file we upload has no duplicate email addresses?
Nov 20 2019
Okay thanks for digging into this and explaining the current prcess! @MBeat33 please have your team chime in if they have other examples they wanted us to look at. It sounds like these were cases of email address typos.
Nov 19 2019
Some sample CIDs:
The new copy has been reviewed and approved by Sam, Michael, and Camille as well. This should be ready to go live :)
Nov 13 2019
Cool suggestion! I like it.
Nov 12 2019
Nov 9 2019
Here are the API details!
Nov 8 2019
Fwiw, I don't think the problem is the filtering scan, I think it's the timing of when this stripe donation got added to Civi. If I'm reading their contribution record right, their contribution was Enqueued At Timestamp Oct 24, 2019 16:10, which was 2 days after the email went out and 7 days after they donated.
Thanks for opening the ticket! If there's an API call to IBM to remove
people on the suppression list, that would be our best bet, though I don't
know if that exists. Should I ask our consultants?
Nov 6 2019
Ah, I see. Here's the current login page:
You don't have to actually reset the IBM id, IIRC, only the campaign
automation login. Let me know if you need us to check with our consultant
Thanks, Elliott! We didn't realize this was a requirement with the system.
We learned our lesson ;)
Nov 4 2019
Wonderful, thank you all for the help!
Okay, I saved the pw in a file called silverpop.txt as suggested.
Nov 1 2019
I just reset the password on the account so Katie could get in and try to re-run the file imports, but now y'all need to reset the pw again on your side. I'm going to update the task title to reflect that.
Oct 31 2019
Okay, then we need to find a way to use the silverpop API to remove people
from the suppression list when they opt back in.
That looks right to me! I want to note that from a silverpop perspective, scenario 2 would require an extra step of us removing a contact from the Suppression list. I think that has to be done manually, though maybe there's an API call for it.
Oct 24 2019
I think my preference is option 2. I know we were concerned about people
who opted in NO and some point and changed their response to YES in the
future. It seems easier to me to set up an annual workflow where we remove
anyone from the unsubscribes list who has changed their response to YES.
Does that seem sensible?
Oct 23 2019
So awesome! Thanks, everyone :)
Oct 16 2019
Huh, I remember us discussing that latest_optin_response field but I
thought we settled on opting these people out. I think that's what we need
to do--turn the opt out on the banner into a hard no.
Oct 8 2019
Definitely, the expected behavior is if someone opts-out of bulk email
while donating, they should be marked as opted-out in Civi and added to the
unsubscribes file that gets sent to IBM each night.
Oct 4 2019
Thank you all so much.
Oct 3 2019
Yeah, you'll need a CiviCRM cert to access the dash (
https://dash.frdev.wikimedia.org) at the least. Sorry, I take for granted
that we just set this up for fundraising people :)
Cool, we must've missed the cutoff time. Thanks, Katie!
Never manually add if it's data we're importing from Civi :) You should see
it when setting up the import. If you don't, just hit run on today's import
and we'll try again tomorrow.
I'm optimistic we caught it in time :) otherwise we'll try again tomorrow.
Hey Eileen, my mistake, I thought it had been deployed! We had already put
the import on pause last night so we could import the new data today. Is it
possible to squeeze it into today's import?
Oct 1 2019
Ooh awesome, sounds like hardly any lag at all, even in the event of a
Thanks so much for this! Just one last question--how often will this be
updated? Daily? Will there ever be a lag?
This is ultimately up to Michael, but I'm wondering if it'll be easier on
DS to send these in a few isolated batches so their team knows when to
expect them. Like 4 batchs of 10k, one per week?
We can use birth_date.
It would be great if she has access to all the same databases that I do,
but these are the ones on my mind:
Sep 24 2019
Okay, from meeting with Elliott, it sounds like the multi-currency
formatting is not a concern. I'm down with Michael's suggestion! Noting as
well that Michael confirmed it's good to remove endowment donations.
And @MBeat33 's suggestion above is fine for me, though I am skeptical as to how much sense it makes to support the annual totals in multiple currencies. Is that easy enough to do, or should we scrap the annual total?
Yes, omit endowment donations @Ejegg!
Sep 23 2019
Ah yessss this is so awesome, Eileen!! Would be good to rename that hash
field if possible to utm_source or mailing_id or something so it's easier
to find and join across tables, but I think this is exactly what I wanted
:D And just in time for our new analysts to use.
Sep 18 2019
Thanks, Eileen! We're going to set up a new recurring import tomorrow. I
think you can close this on your end.
Sep 11 2019
Thanks, Eileen! Let us know when that new data gets added. We didn't set up
those new fields to import on a recurring basis because we didn't want to
slow down import runtime, so we'll need to do another manual import to add
Sep 6 2019
Oh sorry! If we're talking exact matches, yes, I think they should be opted
out until the donor tells us otherwise. My bad 😊
Great, thanks @Eileenmcnaughton! I just told IBM to skip today's import so
I think the file should be around for Katie and me to import later. We'll
update this task with how it goes!
@Eileenmcnaughton let me know if I should pull in Trilogy for help!
Hm, I would disagree here. I might expect a donor would unsubscribe their
old email address knowing that they only check it for certain purposes and
only want to receive emails from us at a different address. I think we
should let donors specify communications preferences on a per-email basis.
Sep 4 2019
Sounds good! Thank you :)
It runs daily at 9am UTC.
Okay! Would be great to do this on the Friday morning UTC export since we
aren't sending emails on Friday, just in case something goes amiss with the
Aug 31 2019
Thanks, everyone, for jumping on a very unexpected task. I appreciate all
the follow-up and feedback.
Aug 29 2019
@MNoorWMF is checking into the opt-in language.
Aug 21 2019
Aug 20 2019
Thanks for scoping this out, @XenoRyet, seems like a pretty thorough
approach to me!
Aug 13 2019
Thanks so much for the quick turnaround, Eileen! I was able to disable the
automated import before the file was uploaded this morning and remapped the
fields. Everything seemed to import perfectly and I have successfully run
queries on the newly imported data. Seems like everything is working. I'm
happy to resolve this task! Thank you again.
Aug 12 2019
Hey all, I'm reopening this task as for some reason, we're only getting fields for 2018, 2019, and 2020. The whole reason we need these fields is to create donor segments based on *past* giving behavior, so we need fields showing donations going back at least the last five years (2014-2017 also need to be included).
Jul 30 2019
Spec doc for the preference center here: https://docs.google.com/document/d/1wKtGxJn06bs6OGAFV7DKUfVw2salKdU_NAUn5SMGks0/edit#
Jul 26 2019
Hi there! Just checking if there is an ETA on getting this data into silverpop? We are getting lots of pressure from higher up to provide our full segmentation strategy, and I had really been hoping to be able to work with the data in silverpop in order to create that schedule, so we don't have to duplicate work. If we don't see this getting done in the next week, please let me know and I will start querying in Civi to fully flesh out our segmentation strategy.
Jul 25 2019
Hmm... looks like I missed this two years ago :D
Jul 23 2019
IBM just likes to name its list types confusing (IMO) things. The two list
types we send to are Queries and Contact Lists. Not sure if it would help
if I walked someone through the UI to see what I mean, but happy to do that.
Jul 19 2019
Cool, thank you! That way we just disable the current import and set up a
new one mapping the new fields. Give me a 24 hours' heads up so we can
disable the active import (if it runs we lose the file on FTP and can't
It's easiest to add the fields once they appear in the export file. Are
they there already, just not populated?
Yep, we'd need to forget in each DB. And yes, the UI is spooky! But it
Here it is! https://www.mozilla.org/en-US/newsletter/existing/8813b665-cfa8-495b-ada5-9c0a9c011522/. Feel free to play with my preferences if you want--I'm opted into this list with multiple email addresses 🤓
I believe that a manual deletion in IBM and the forget me process are
different. The reason is that just in case this record exists anywhere else
in IBM, like on a different list, it will remain in IBM's logs. I think we
need to do the Forget Me if were going to do this properly.
Jul 18 2019
Thanks so much for the quick turnaround, Jeff! Katie is iiiiiin.
Thanks, Katie! Just a +1 here to prioritize this. Katie is our email stats
person (among other things!). We're running our Japan campaign which has
multiple tests per week, and need to be able to stay on top of stats. It
makes the campaigns way more labor intensive for Moska if she has to run
stats as well.
Jul 17 2019
Okay, looks like I missed something big here! @NNichols can I get some more context on why it's important to combine endowment gifts with annual fund gifts in these "latest donation" fields? These fields are some of the most important ones we use in our fundraising appeals. I'm worried that combining this data will get us in some legal trouble because we'll end up pulling them into our annual appeal emails and may suggest that an endowment gift actually went to WMF.
Oh huh I didn't realize that our totals were including endowment gifts. Can
we make it so that the non-endowment fields in the silverpop export are
just that--non endowment? We might get ourselves into some legal trouble if
our copy suggests that someone's last gift was to WMF when it was really to
the endowment... Why have these fields cross over when we can have two
Thanks for asking! It would be great to have some of those endowment
fields, too. These would be great to start with:
Okay, starting a thread with David+Eileen+Trilogy.
Got it, thank you!
Awesome! Are the stats from the one test we ran backfilled, or is that test a dud?
Hey all, apologies for not adding feedback sooner! Your suggestion was
interesting, Eileen, but felt like it's achieving something similar to the
calendar year totals we're adding. Maybe I'm not seeing what problem we're
trying to solve for here since I don't typically use the Civi UI? Let me
know if there's something for me to look at here.
@Eileenmcnaughton awesome! Do you have a projection on how long that will take? We've been waiting to be able to use this data to map out our Q2 schedule. Could we possibly have it run through Big English data before other stuff?
Hey @Eileenmcnaughton did this get deployed during the maintenance on
Hi @Eileenmcnaughton, thanks for investigating! I asked Brian and Eric at
Trilogy and they confirmed the API can't do this for us :( Their response,
Jul 9 2019
I can get the column headers for you super easily from IBM. If you actually
want to see the SQL that builds it, that's a different story. Should I get
you the column headers?