Oops, thank you @jkim_wikimedia! @LeanneS Can you remind me why we don't want to send automatic thank you letters to donors entered by Engage?

Oops, can't deploy that till we figure out what to do with Engage donations. Turns out they end up with a blank in the no_thank_you field which till now has meant we don't send an automatic letter.

Yeah, that's an unavoidable consequence of our iframe flow. The part of the page that we host, including all the personal info, is totally unaware of events in the iframe hosted by the processor. If we're not going to change to a multi-page 'full redirect' flow, we could mark all the personal info fields as disabled / not editable after opening the iframe.

We figured this out - the list of recurring donations has been moved to a second sub-tab under the contributions. Under that sub-tab, the recurring donations in 'Completed' state for some reason show up under the inactive section, but they still have a functional Cancel button.

Got a patch that should allow '0' or '', but we might need to clean up T97684 first

Shoot, we're still using this in the query to get the batch of donations to send thank you letters for:

WHERE
 receive_date > :earliest AND
 thankyou_date IS NULL AND
 no_thank_you IS NULL

So '0' won't do the trick, unfortunately. We'll need to change that in the code.

No worries, I probably should have communicated this more clearly!

OK, numbers from the old integration on the 15th check out too. 1,315 of them were recurring, so that's 709 new one-time donations, a bit under the number of payments_fraud rows.

payments_fraud table numbers are a bit funny, though - the old integration only shows 757 rows from the 15th, while the new integration shows 9,044. Checking to see if the old integration donations were mostly recurring - we don't add fraud rows for the recurring installments.

Hi @krobinson! How are you looking for the donations in Civi? The Wednesday tests have been using our new integration, which codes the donations as gateway='ingenico' rather than gateway='globalcollect'. The trxn_ids for the new integration also start with 'INGENICO 0000%' rather than 'GLOBALCOLLECT %';

Hi @MBeat33, of the payments in the CSV, only one of them is from the new integration:
Order ID 4000012878 / Merchant Reference 58116445.1

Looks like https://ipstack.com/ has a free tier

Silverpop fetch jobs would be a great place to start. Often just pausing for a while is harmless and fixes it all.

@CCogdill_WMF Oooh, so we would have to remove the email from Civi, not just refrain from emailing them, if they don't opt in? That's a lot higher volume data deletion than we were picturing for Eileen's 'Forgetme' work.

Looks like we need one more thing here, to do something besides show the donor a broken form. The easiest seems to be to treat the canceled status as 'failed' and show an error page which includes a 'try again' link.

@MBeat33 want to put a csv file of order IDs on the server? I think I can come up with a batch query to sort out if any of them are related to the original group we tried to fix in this ticket.

CCogdill_WMF Hmm, the old record would stay opted in, and if we merge the records, the opt_in would apply to the merged record. I don't know if the newer email address would become primary for the merged record, but if it did we would consider the whole record opted in.

Well, if person X has primary email a@example.com, and person Y has primary email b@example.com, but a secondary email a@example.com, we would export X's record to IBM with a@example.com, and Y's record to IBM with b@example.com. Then if person X opts-in via a mail to a@example.com, we wouldn't want to opt in person Y and start sending emails to b@example.com. The issue is that opt_in is a flag on the contact record, not the email.

OK, let's keep it simple and just collect the email address on the form. When the message gets to Civi, we can mark all contacts with a matching primary email address as opted in.

Ugh. Looks like we actually have to do this from the variant editor. Need to go into the form variant, click on the language at the top right, click on the button to edit, erase the contents, and paste in the new text.
And you have to do that 6x for English (IE, CA, GB, US, NZ, AU), 4x for Spanish, etc. And then do it all over again for the sandbox. Boo.

@CCogdill_WMF do you want to let people opt-in with a link that just has their email address, or do we need to confirm with the contact_hash or contact_id?

@MBeat33 I looked at the last nightly charge jobs for 380844346, 400174014, 712250785, and 1026134918, and they all look totally normal. Our DO_PAYMENT call comes back with RESULT OK, our GET_ORDERSTATUS call comes back with STATUSID 600, then our SET_PAYMENT call comes back with RESULT OK.

So, there are at least 7 recurring donations from the initial test on 5/11 that we were able to charge successfully. Not a great rate, but it shows that it's theoretically possible to tokenize after the fact.

T172165 suggests that Debian will be providing security patches for 7.0 in stretch till June 2022. Does that sound right?

Thanks, @Nikerabbit ! It's great to see new translations coming in for DonationInterface again.

Oh hey, the from-address default to 'smashpig-failmail@' . gethostname() if we don't include it in settings. That's a quick fix to start with!

Hmm, I just ran the amazon download script again and it DID work. Let's just keep an eye on this and see if it stays working.

Aargh, I missed the error. Apparently the download is still borken, even though we can make API requests from the front-end.

Amazon found the problem on their side and resolved it. Still TODO: get Maggie rights to edit outages.js, CN privs.

Yes, please go ahead and kill this. It appears to be an obsolete (untouched by fr-tech since 2013) copy of wikimedia/fundraising/tools/DjangoBannerStats, which itself is on the verge of being deleted.

@jgleeson the code is deployed! Next steps: deploy the credentials, set the path, and set up a nightly job after exchange-rates-update. I guess we'd want to make a parent job to chain the two.

@cwdent our outbound IP address ranges haven't changed, have they? I gave them a range to allow to use the proxy.

@cwdent checked our firewall config and found that we're not blocking the TCP proxy.

Reassuring go-ahead for Wednesday's test: let's find at least one good recurring donation that we WERE able to charge, maybe not one of these.

Ooh, thanks! This is super helpful.

Looks like @Eileenmcnaughton's fix was the real deal! Just deployed it, and all of the cids mentioned in this ticket are loading snappily. Heck, even Anonymous Anonymous (cid=72), with more than 18,000 donations, is loading in under 5 sec.

OK, looks like this is working again

@Pcoombe OK, here's the final list of messages that will need to take the email as a parameter in the future:

duplicate of T197602 ?

Oh shoot, BHD is one of those currencies that takes three decimal points - I bet that has something to do with it. This is probably stopping us from getting anything at all in that currency!