I have a query that will generate the spreadsheet for PayPal batch refund and subscription cancellation.

Change 411386 merged by jenkins-bot:
[mediawiki/extensions/DonationInterface@master] Script for Paypal refunds

https://gerrit.wikimedia.org/r/411386

@XenoRyet in your csv, which is the subscription_id and which is the gateway_txn_id? I see columns with gateway_id and txn_id.

@XenoRyet, and is id order_id?

@XenoRyet, it looks like I might also need amount.

Change 415752 had a related patch set uploaded (by Mepps; owner: Mepps):
[mediawiki/extensions/DonationInterface@master] Fixes after testing with data

https://gerrit.wikimedia.org/r/415752

Change 415752 merged by jenkins-bot:
[mediawiki/extensions/DonationInterface@master] Fixes after testing with data

https://gerrit.wikimedia.org/r/415752

@mepps fields should be order_id, gateway_txn_id, subscr_id, then currency and now amount, but let me know if any of those don't look like what you were expecting or aren't working.

Change 416461 had a related patch set uploaded (by Mepps; owner: Mepps):
[mediawiki/extensions/DonationInterface@master] Push message to recurring

https://gerrit.wikimedia.org/r/416461

Change 417366 had a related patch set uploaded (by Mepps; owner: Mepps):
[mediawiki/extensions/DonationInterface@master] Add profile_id

https://gerrit.wikimedia.org/r/417366

Change 417366 merged by jenkins-bot:
[mediawiki/extensions/DonationInterface@master] Add profile_id

https://gerrit.wikimedia.org/r/417366

@XenoRyet for GlobalCollect we will need the following rows: // order_id, merchant_reference, effort_id, payment_submethod, currency, amount

Change 416461 abandoned by Mepps:
Push message to recurring

https://gerrit.wikimedia.org/r/416461

Sadly, the behavior we were trying to use for this script (using the PaypalEC to refund/cancel Paypal Legacy subscriptions) does not appear to work. @Ejegg is exploring whether there is a bulk upload feature. The other option is basically browser automation.

@XenoRyet Can you also pull a list of email addresses for the test list of paypal transactions you sent me? I also need additional sample transactions and hopefully we can ensure (with @MBeat33 or I can do it in the console) they're not transactions that have already been refunded.

@MBeat33 Do you have an easy list of user emails or transaction ids where you've already issued the refunds (from paypal) manually?

We will also discuss this more in depth as a group with @CCogdill_WMF and @Ppena tomorrow.

Change 419017 had a related patch set uploaded (by Mepps; owner: Mepps):
[mediawiki/extensions/DonationInterface@master] Remove PROFILEID from RefundTransaction

https://gerrit.wikimedia.org/r/419017

We have a script that can manually refund paypal payments and we can bulk upload emails to cancel subscriptions. Tomorrow we just need to make sure we know when we should run these. Similarly, with Ingenico/Globalcollect, we can end the subscriptions in our database and run a script for refunds.

Sure thing, @mepps. I shared the Master Refunds doc with you, see the PayPal tab, which will show IDs for the ones DS processed. There may also be some refunds initiated by donors directly at PayPal - so if you encounter any Civi refunds not in Master Refunds, that may be where they come from. Similarly for chargebacks and Ingenico.

And great news, thanks!

Change 419017 merged by jenkins-bot:
[mediawiki/extensions/DonationInterface@master] Remove PROFILEID from RefundTransaction

https://gerrit.wikimedia.org/r/419017

Change 419232 had a related patch set uploaded (by XenoRyet; owner: xenoryet):
[wikimedia/fundraising/crm@master] Cancel unintended Ingenico subscriptios

https://gerrit.wikimedia.org/r/419232

Change 419232 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Cancel unintended Ingenico subscriptios

https://gerrit.wikimedia.org/r/419232

GC subscriptions have been canceled. Still planning on issuing the refunds tomorrow as discussed.

@XenoRyet when you come online, can you post the test paypal emails on dev?

Looks like PayPal is being finicky, but we would love to send the emails to
the GC list if those cancellations and refunds will be done today (is that
confirmed?).

I don't have data on which emails on the main list of 7300 were GC or PP,
so it would be great if someone could provide me with a list of the GC
emails only. Thank you!

Change 419622 had a related patch set uploaded (by Ejegg; owner: Ejegg):
[mediawiki/extensions/DonationInterface@master] Stuff country with default for refunds

https://gerrit.wikimedia.org/r/419622

Change 419622 merged by jenkins-bot:
[mediawiki/extensions/DonationInterface@master] Stuff country with default for refunds

https://gerrit.wikimedia.org/r/419622

Change 419635 had a related patch set uploaded (by Ejegg; owner: Ejegg):
[mediawiki/extensions/DonationInterface@master] Add missing EFFORTID to DO_REFUND

https://gerrit.wikimedia.org/r/419635

Change 419635 merged by jenkins-bot:
[mediawiki/extensions/DonationInterface@master] Add missing EFFORTID to DO_REFUND

https://gerrit.wikimedia.org/r/419635

Donor response to yesterday's email informing Ingenico donors of their cancellation & refunds has been entirely positive. @krobinson noticed that for two of these donors, though the refunds are visible at Ingenico, their donations still show as active recurrings in Civi.

cid=24041832 & cid=4518643 are not lined out where we would normally cancel manually:

Can we confirm that the script really canceled all of these donations?

@MBeat33 I'll take a look at that today and let you know. I'm 95% sure everything is canceled, but I'll double check to be certain.

The refunds for individual contributions will only show up in Civi after we get a new audit file from GC and process it.

We did forget to update one column on the recurring rows - we set the cancel_date (which is enough to stop the charges from happening next month) but forgot to change the status. We'll update the status ASAP.

Cool, thanks for checking on that! Really glad to hear the donor response
has been positive.

I'll keep following the thread with PP to see when those emails should go
out.

Thank you for updating the Statuses, @Ejegg

The PayPal cancellations are underway, and the donors are first hearing from PayPal about it and then coming to donate@, so I think the sooner we send the PP email the better.

Change 419827 had a related patch set uploaded (by XenoRyet; owner: XenoRyet):
[wikimedia/fundraising/crm@master] Update status code on canceled unintended recurring donations.

https://gerrit.wikimedia.org/r/419827

Okay great, I'm prepping the PP email.

Change 419827 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Update status code on canceled unintended recurring donations.

https://gerrit.wikimedia.org/r/419827

@MBeat33 Statuses on the GC cancellations should be updated now.

Thank you @XenoRyet.

The Paypal refunds script has run, and we are waiting from an update from Paypal on how the mass subscription cancellation went but a review of the activity log looked good.

Just sent the email to PP donors at 2pm PST / 9pm GMT. All donors have been
notified!

Regarding the GC refunds - all were successful except for 3 associated with a single order id. That's order 4576470192, effort IDs 2, 3, and 4. The DO_REFUND calls for those came back with code 300450, ORDER WITHOUT REFUNDABLE PAYMENTS.

@XenoRyet This morning @MBeat33 noticed that not all the refunds have been processed yet, from him:

sure thing, paypal IDs like 5YX03610D73348834, 8PA66226P6924463B from one donor; 53T11576TA740044E & 3T249216BA688721P from another, don’t show as refunded at PayPal.

I looked in the csv you pulled and those weren't in there. Can you take a look at the query and see if we're pulling all the transactions we need to refund? I noticed that for at least the donor, none of the transactions have been refunded.

Mentioned in IRC but posting here for completeness: The transactions were missing from the refund list due to the subscriptions being canceled shortly before the query was run. I've fixed the query to account for that, and a new refund list is ready to be run.

We have run all the Paypal refunds as far as we know now and all the subscriptions should be canceled. Unfortunately, many paypal donors got notification from Paypal before our email or the emails bounced which caused some confusion so @MBeat33 is still getting new tickets from confused donors. Some GC donors may have also gotten emails about refunds even if they'd already canceled their subscription so we may still want to ensure all GC refunds went through. Putting this in review for now.

Thanks, @mepps - I'll add any GC transactions here for missing refunds.

For future reference emailing the PayPal donors before any refunds/cancels will be best practice - PayPal's messaging is not as clear as ours, and they send a separate email for each refund.

@mepps this one from PayPal looks like it hasn't been refunded (though it is canceled in Civi): cid=3454258, transactions are
3E524392EE162681K
5YX03610D73348834
8PA66226P6924463B

Could you check to see if that was in Friday's batch?

Also, plenty of the PayPal refunds that did process Friday are not in Civi yet, but I'm not sure if the audit should have captured them yet or not, so just fyi.

Same with the GC refunds like 9358349320 from the 15th, they are not yet in Civi as well.

@MBeat33 Looking into that--it appears that some refunds didn't process with an error message this error message: https://community.zuora.com/t5/Billing-Payments/Error-message-10009-You-do-not-have-a-verified-ACH/td-p/21401. It appears to mean we had insufficient funds. We are wondering whether because all these refunds processed, our account ran out of money but then more donations came in and covered later refunds. In the short term, I can re-run the refund since it won't affect already refunded transactions.

@MBeat33 All of the refunds for that user have now been run.

@MBeat33 and @XenoRyet , new list of banners that included the confusing toggle in the time range of October 11, 2017 to November 29, 2017. These are also in a new tab of the spreadsheet: https://docs.google.com/spreadsheets/d/1A3hJZEBROkbgWm5Xdaah3ehVJSJqJyrYmML9VtazEik/edit#gid=354218842.

('B1718_1011_en6C_ipd_p1_lg_frq_cnt', 'B1718_1011_en6C_ipd_p1_lg_frq_txttoggle', 'B1718_1011_en6C_mob_p1_lg_frq_cnt', 'B1718_1011_en6C_mob_p1_lg_frq_txttoggle', 'B1718_1018_en6C_ipd_p1_lg_frq_cnt', 'B1718_1018_en6C_ipd_p1_lg_frq_togunder', 'B1718_1018_en6C_ipd_p2_sm_frq_toggle', 'B1718_1018_en6C_mob_p1_lg_frq_cnt', 'B1718_1018_en6C_mob_p1_lg_frq_togunder', 'B1718_1018_en6C_mob_p2_sm_frq_toggle', 'B1718_1019_en6C_ipd_p1_lg_dsn_cnt', 'B1718_1019_en6C_ipd_p1_lg_dsn_greybknd', 'B1718_1019_en6C_ipd_p1_lg_dsn_legacycblinebreaks', 'B1718_1019_en6C_ipd_p1_lg_txt_jimmyIwaving', 'B1718_1019_en6C_mob_p1_lg_dsn_cnt', 'B1718_1019_en6C_mob_p1_lg_dsn_greybknd', 'B1718_1019_en6C_mob_p1_lg_dsn_legacycblinebreaks', 'B1718_1019_en6C_mob_p1_lg_txt_jimmyIwaving', 'B1718_1025_en6C_ipd_p1_lg_dsn_betterdonateux', 'B1718_1025_en6C_ipd_p1_lg_dsn_cnt', 'B1718_1025_en6C_ipd_p1_lg_dsn_stckrhello', 'B1718_1025_en6C_ipd_p1_lg_dsn_stckrhellocoffee', 'B1718_1025_en6C_ipd_p2_sm_txt_2016fus', 'B1718_1025_en6C_ipd_p2_sm_txt_cnt', 'B1718_1025_en6C_mob_p1_lg_dsn_betterdonateux', 'B1718_1025_en6C_mob_p1_lg_dsn_cnt', 'B1718_1025_en6C_mob_p1_lg_dsn_stckrhello', 'B1718_1025_en6C_mob_p1_lg_dsn_stckrhellocoffee', 'B1718_1025_en6C_mob_p2_sm_txt_2016fus', 'B1718_1025_en6C_mob_p2_sm_txt_cnt', 'B1718_1101_en6C_ipd_p1_lg_dsn_cnt', 'B1718_1101_en6C_ipd_p1_lg_dsn_coffee', 'B1718_1101_en6C_ipd_p2_sm_frq_belowamts', 'B1718_1101_en6C_ipd_p2_sm_frq_cnt', 'B1718_1101_en6C_mob_p1_lg_dsn_cnt', 'B1718_1101_en6C_mob_p1_lg_dsn_coffee', 'B1718_1101_en6C_mob_p2_sm_frq_belowamts', 'B1718_1101_en6C_mob_p2_sm_frq_cnt', 'B1718_1107_en6C_ipd_p1_lg_dsn_cnt', 'B1718_1107_en6C_ipd_p1_lg_dsn_refactor', 'B1718_1107_en6C_ipd_p2_sm_txt_cnt', 'B1718_1107_en6C_ipd_p2_sm_txt_usealot', 'B1718_1107_en6C_mob_p1_lg_dsn_cnt', 'B1718_1107_en6C_mob_p1_lg_dsn_refactor', 'B1718_1107_en6C_mob_p2_sm_txt_cnt', 'B1718_1107_en6C_mob_p2_sm_txt_usealot', 'B1718_1111_en6C_ipd_p1_lg_txt_awkwknd', 'B1718_1111_en6C_ipd_p1_lg_txt_awkwkndregret', 'B1718_1111_en6C_ipd_p1_lg_txt_cnt', 'B1718_1111_en6C_ipd_p1_lg_txt_happysat', 'B1718_1111_en6C_ipd_p2_sm_txt_cnt', 'B1718_1111_en6C_ipd_p2_sm_txt_thissat', 'B1718_1111_en6C_mob_p1_lg_txt_awkwknd', 'B1718_1111_en6C_mob_p1_lg_txt_awkwkndregret', 'B1718_1111_en6C_mob_p1_lg_txt_cnt', 'B1718_1111_en6C_mob_p1_lg_txt_happysat', 'B1718_1111_en6C_mob_p2_sm_txt_cnt', 'B1718_1111_en6C_mob_p2_sm_txt_thissat', 'B1718_1114_en6C_ipd_p1_lg_dsn_iosjim', 'B1718_1114_en6C_ipd_p1_lg_txt_awkv1long', 'B1718_1114_en6C_ipd_p1_lg_txt_awkv2short', 'B1718_1114_en6C_ipd_p1_lg_txt_cnt', 'B1718_1114_en6C_ipd_p2_sm_txt_cnt', 'B1718_1114_en6C_ipd_p2_sm_txt_tues', 'B1718_1114_en6C_mob_p1_lg_dsn_iosjim', 'B1718_1114_en6C_mob_p1_lg_txt_awkv1long', 'B1718_1114_en6C_mob_p1_lg_txt_awkv2short', 'B1718_1114_en6C_mob_p1_lg_txt_cnt', 'B1718_1114_en6C_mob_p2_sm_txt_cnt', 'B1718_1114_en6C_mob_p2_sm_txt_tues', 'B1718_1116_en6C_ipd_p1_lg_dsn_cnt', 'B1718_1116_en6C_ipd_p1_lg_dsn_plusminus', 'B1718_1116_en6C_ipd_p1_lg_dsn_redcta', 'B1718_1116_en6C_ipd_p1_lg_txt_thisweekdaycoffee', 'B1718_1116_en6C_ipd_p2_sm_dsn_cnt', 'B1718_1116_en6C_ipd_p2_sm_dsn_exp2ndpara', 'B1718_1116_en6C_mob_p1_lg_dsn_cnt', 'B1718_1116_en6C_mob_p1_lg_dsn_plusminus', 'B1718_1116_en6C_mob_p1_lg_dsn_redcta', 'B1718_1116_en6C_mob_p1_lg_txt_thisweekdaycoffee', 'B1718_1116_en6C_mob_p2_sm_dsn_cnt', 'B1718_1116_en6C_mob_p2_sm_dsn_exp2ndpara', 'B1718_1116_enGB_ipd_p1_lg_amt_250', 'B1718_1116_enGB_ipd_p1_lg_amt_cnt', 'B1718_1116_enGB_ipd_p2_sm_amt_250', 'B1718_1116_enGB_ipd_p2_sm_amt_cnt', 'B1718_1116_enGB_mob_p1_lg_amt_250', 'B1718_1116_enGB_mob_p1_lg_amt_cnt', 'B1718_1116_enGB_mob_p2_sm_amt_250', 'B1718_1116_enGB_mob_p2_sm_amt_cnt', 'B1718_1118_en6C_ipd_p1_lg_txt_awkwkdyregret', 'B1718_1118_en6C_ipd_p1_lg_txt_awkwkndregret', 'B1718_1118_en6C_ipd_p1_lg_txt_cnt', 'B1718_1118_en6C_ipd_p1_lg_txt_wkday', 'B1718_1118_en6C_ipd_p2_sm_dsn_cnt', 'B1718_1118_en6C_ipd_p2_sm_dsn_exp2pp', 'B1718_1118_en6C_mob_p1_lg_txt_awkwkdyregret', 'B1718_1118_en6C_mob_p1_lg_txt_awkwkndregret', 'B1718_1118_en6C_mob_p1_lg_txt_cnt', 'B1718_1118_en6C_mob_p1_lg_txt_wkday', 'B1718_1118_en6C_mob_p2_sm_dsn_cnt', 'B1718_1118_en6C_mob_p2_sm_dsn_exp2pp', 'B1718_1122_en6C_ipd_p1_lg_txt_cnt', 'B1718_1122_en6C_ipd_p1_lg_txt_dayawkbusyremind', 'B1718_1122_en6C_ipd_p1_lg_txt_dayawkbusyremindemb', 'B1718_1122_en6C_ipd_p1_lg_txt_daycoffee', 'B1718_1122_en6C_ipd_p2_sm_dsn_cnt', 'B1718_1122_en6C_ipd_p2_sm_dsn_expday', 'B1718_1122_en6C_mob_p1_lg_txt_cnt', 'B1718_1122_en6C_mob_p1_lg_txt_dayawkbusyremind', 'B1718_1122_en6C_mob_p1_lg_txt_dayawkbusyremindemb', 'B1718_1122_en6C_mob_p1_lg_txt_daycoffee', 'B1718_1122_en6C_mob_p2_sm_dsn_cnt', 'B1718_1122_en6C_mob_p2_sm_dsn_expday')

Thank you, @spatton. We checked the creation dates for 267 of the ~618 Ingenico's manually canceled since January 1st. 50 of them (18.7%) were created before 11/30, with Nov 11 & Nov 18 high concentrations. A significant portion of the pre Nov 30 recurrings may also be unintended.

@mepps, thanks for the refunds update. I pinged Tony Le about how much cash is generally on hand at Paypal to cover batch refunds; maybe we can temporarily increase that amount for batches. Also, thanks for suggesting to update any future donor-facing emails to include that the refunds may appear as new charges on donor bank statements.

per Tony Le, fyi:

The USD account is swept daily but to maintain a minimum balance of $25K. If this is not sufficient we will need to work with PayPal to reset the minimum balance.

@MBeat33 With the new banner list, looks like we get 2109 donors with active recurring subscriptions from that time range and those banners. 1074 of them are from Ingenico, the rest would be PayPal.

Thanks, @XenoRyet

As it seems like a lot of those 2109 may be unintended, I will add the possibility of a second round of cancel/refunds to the wider email thread, so all stakeholders see it.

Since this round seems complete, I'm going to go ahead and mark this task done. If we end up wanting to do another round, we can track that under a new ticket.

Apologies for not replying on this yesterday. As 8am is pretty early for
me, I would prefer to have this scheduled so I know to be up to hit send.
Can we schedule this to go out on Thursday morning?

@CCogdill_WMF Tomorrow works fine for Donor Services.

I think the current task is T190854: Refund the unintended recurring donations from the tests pre big english

My bad, thanks, Mbeat.

Similar to T190854: Refund the unintended recurring donations from the tests pre big english could we do a scan of this batch to make sure all the refunds have processed? Thank you

@XenoRyet the missing refund appears to be from GlobalCollect--can you look into it?

10-4 I'll see what I can find out.

@MBeat33 I did a random sample of the paypal refund transactions and so far they all went through.

Great news, thank you @mepps. If you and @XenoRyet are confident about the list, I'm fine with closing this task.

Everything I can find looks resolved. I'll go ahead and close.

@XenoRyet did you ever find out why that one that MBeat33 found didn't get refunded?

@mepps Actually that one weirdly doesn't turn up in the logs anywhere outside of the initial transaction. That said, in my searches I did come across a handful of other failures from the script, but they all eventually had successful refunds one way or another. There might be more phantoms out there, but there'd be no way to track them down that I can think of.

@XenoRyet what were the reasons for the other failures?

@mepps The logs weren't that specific that I could see, the refund function just reported a failure, then later in the logs there were successful attempts. I suspect timeouts or hiccups Ingenico side, but that's just a guess.

@mepps @MBeat33 Ok, it looks like cid=23929416 didn't make it into the refund batch because it doesn't appear to be a recurring type, but rather a one-time GC donation. I reran the queries I used to generate the email lists and that donor doesn't come up in those either, so I'm not sure how he snuck into the email list.

I did check the actual file with the email list for the second round of refunds, and the donor isn't on that. I don't have a copy of the file from the first round available anymore, but if someone does we could check that to see when this donor crept into the process.

Thanks, @XenoRyet cid=23929416 looks recurring as the successive donations have the order id tacked on at the end, like 5962376988-4, so (from what I can see) it was good to have it in the initial list. I think T190098: Ingenico audit wobble? March 13 refunds not in Civi covers why these refunds show at Ingenico but not yet in Civi.