So I've been struggling all afternoon to get the script tests/test_update.py to run & still think I'm failing after lots of mucking around with what is or isn't installed.

But then I went back to the queries and I started to wonder if this is actually happening already

We build silverpop_export_staging with

``

WHERE
  e.email IS NOT NULL AND e.email != ''
  AND c.is_deleted = 0
  AND e.is_primary = 1;

``

ie no contribution filter

And later on we use no contribution filter on our final export tables

Hey @Ejegg, just checking--we are already recording opted-in (failed)
payment attempts in Civi? It was my assumption that we weren't creating
Civi records for those folks, so they wouldn't appear in the export.

@CCogdill_WMF you're correct, we are not yet creating Civi records for people with failed donations. I'm not sure where those 405,576 people are from, but a whole lot of them have the opt_in field empty. Maybe from the remind me later lists?

There are two code patches for creating records in civi for opt-in/failed donations, and each has their own Phab ticket. The payments-wiki patch (T216293) was just approved yesterday, and the civi patch (T217710) is still in process.

Ah okay. I'd be curious where those are coming from. We have more than 400k
on the RML lists so I would guess that's something else.

Ah @Ejegg <T217712+public+7ba0a0e50372df47@phabricator.wikimedia.org> can
we roll this out more slowly? I'm sorry, I didn't know it would be done so
quickly. I think it's essential these people get a double opt-in email and
I don't have that translated anywhere yet.