Jumping in since Katie is having a busy week. I want to add a response to
your question along with a few process notes:

1. We can stagger the runtime of the import jobs (we already treat the two separately). I'm interested in figuring out the best timing, and maybe if we should do the suppression list first? The main file import takes so long, and Acoustic can only perform one job at a time so it makes the other wait in the queue while it runs
2. It would be unusual for a donor to unsubscribe and get an email the very next day. Most people interact with our emails within 2 days of receiving it, and we almost never send emails with less than a week between them. The schedule allows for a little flexibility
3. Our official unsubscribe policy is it could take us up to 3-4 business days to process an unsub request--DS knows this and employs it in messaging. We strive for a shorter time frame, but legally, we have leeway
4. Katie monitors the imports daily and flags when they go down. If the suppression list import fails two days in a row, we pause all campaigns

@CCogdill_WMF thanks for this - that simplifies it - it sounds like the suppression list can run 'any time'

It looks like I cam close to getting the suppression list query time down to 'negligible'' ( I assume the file is already small.), but the suppression list is leveraging the data calculations from the main export so I need to do more work there before we could run it first and get much benefit from running it first.

Makes sense!

I'm a bit stuck on this - I can't understand why this isn't passing - https://gerrit.wikimedia.org/r/607156 & I'm having trouble running the tests locally & my attempt to fix https://gerrit.wikimedia.org/r/607166 is not working :-). I'm going to focus on another part of the sql for a bit (this this is kind of a discrete portion & I was going to 'knock it off & then look at the address part' ..... 4 hours ago....

If anyone wants to take over figuring this out feel free to grab this ticket. It wouldn't be a bad way to collaborate on the silverpop restructure since it is a task that can be separated from the borg.

@Eileenmcnaughton I added a patch that gets the tests working. I figure we'll squash it with yours but I wanted to separate it to make sure it's the best way (that it reflects the behavior we want). It looks like the error was causes by us not dropping and rebuilding the table, as well as adding the log_date check. What was the next thing you were going to focus on on this ticket?