Page MenuHomePhabricator

Adapt the PayPal notification controller to handle the new types of transaction IDs
Open, Needs TriagePublic

Description

The transaction notifications sent by PayPal will slightly change. We need to adapt our application to be able to process them. Please refer to the investigation documentation for more details.

Acceptance Criteria

  • Payment notifications for recurring payments (both from legacy and API) can be processed by our application.
  • Dependeing on the payment, the notification controller dispatches to the correct notification use case (donation, membership). Since memberships are not implemented, the controller should log an error instead.

Implementation Notes

  • To get a data for "real" IPN (needed for the test cases), you can activate the API on the test server and make a PayPal donation with our test account. Paypal will send an IPN, that will be rejected (because of its unknown transaction type) and be logged in logs/application/paypal.log on the server.
  • The transaction types (txn_type) for recurrring payments will be:
    • recurring_payment_profile_created (you can ignore this, it might get relevant later when we handle memberships)
    • recurring_payment
  • The transaction type (txn_type) for one-time payments will be cart (see T344839: Integrate PPL 5 - Adapt the Paypal notification controller to handle one-time payments)

The Fundraising Application needs to be changed to handle the new transaction types:

  • Create a a "lenient" authorizer implementation that always returns true for systemCanModifyDonation. Background Info: With API payments we no longer have an updateToken. But that's not a problem since we have other means to make the end point secure.
  • Construct the BookDonationUseCase (for PayPal) with the new lenient authorizer
  • Create a new service class (e.g. ConfirmationUseCaseDispatcher) for looking up donation/membership ID and type using IPN data: It should look at the transaction type to determine the name of the ID field (recurring_payment_id for recurring_payment, txn_id for cart ), use the database table payment_paypal_identifier to get a payment id, payment_id field in donation and membership tables to get to the type (donation/membership) and the entity id. Dispatch (with similar code as you can find in the HandlePayPalPaymentNotificationController) as follows:
    • if the transaction type is neither recurring_payment nor cart, dispatch to BookDonationUseCase (using the existing logic in the controller)
    • if payment is for membership, log a "not implemented" error
    • Do the current lookup of the donation (by Id) for all other txn_type values, calling BookDonationUseCase
  • Make the ConfirmationUseCaseDispatcher resilient: the transaction ID for a one-time donation is not found in payment_paypal_identifier, that might be a race condition, occuring when PayPal sends the notification after the use case from T354963: Add PayPal Payment Capture Use Case to Payment bounded context has issued the capture API call but before it stored the modified Order with transaction ID to the database. In this case (transaction type is cart but txn_id was not found in payment_paypal_identifier), try to read the database again after an increasing amount of seconds (flushing query caches in between) before failing. The retry mechanism should be an interface, so in production we can have an exponential backoff (e.g. 1, 2, 4, 8, 16 and 32 seconds) but when unit-testing the route we can update the database after the first or second try
  • Use ConfirmationUseCaseDispatcher in HandlePayPalPaymentNotificationController.
  • Minimum suggested additional test cases in HandlePayPalPaymentNotificationRouteTest that check the new payment data:
    • recurring_payment & matching recurring_payment_id for donations should be handled
    • recurring_payment & non-matching recurring_payment_id should be dropped (and logged)
    • cart & matching txn_id for donations should be handled for donations
    • cart & non-matching txn_id should retry until either the donation is found or the maximum limit of retries is reach (it should log then). 2 test cases
    • cart & non-matching txn_id for donations should be dropped (and logged)
    • recurring_payment & matching recurring_payment_id for memberships should log an error

Event Timeline

kai.nissen set the point value for this task to 8.
gabriel-wmde renamed this task from Adapt the PayPal notification controller to handle the new types of transaction IDs to Integrate PPL 4 - Adapt the PayPal notification controller to handle the new types of transaction IDs.Jul 11 2023, 1:48 PM

Putting this in "Blocked Externally" for now, because we're waiting for the IPNs on the test system.
@kai.nissen Can you check if the test account ( paypal-test-merchant@wikimedia.de ) is configured to send IPNs to test-spenden-2.wikimedia.de (*not* test-spenden.wikimedia.de)? If everything is configured correctly and we don't receive IPNs we might need to contact support

We discovered that the IPNs for payment created by API requests don't contain donation IDs and tokens any more, which we need for dispatching to the use case. We'll have to refactor the payment bounded context first (see T343846: Adapt CreatePayment use case to allow storing external payment ID from payment provider and T343847: Create database migration to store url and external ID for paypal payments) before coming back to this ticket (which also needs to be reformulated)

Draft PR (with failing test case that has "real" data from an IPN): https://github.com/wmde/fundraising-application/pull/2826

gabriel-wmde renamed this task from Integrate PPL 4 - Adapt the PayPal notification controller to handle the new types of transaction IDs to Adapt the PayPal notification controller to handle the new types of transaction IDs.Jan 12 2024, 5:32 PM
gabriel-wmde updated the task description. (Show Details)
gabriel-wmde removed the point value for this task.