Tracking issue for the WMDE Fundraising frontend "rewrite".
This "rewrite" will happen by starting from scratch architecture and design wise. As top down implementation is done, the corresponding parts of the old codebase can be used as they are, used in a refactored form, or thrown away and replaced entirely as we see fit.
**Proposed approach**
First we identify a use case that we predict is easy to implement and minimally depends on behaviour we'd need to implement for another one. We can then implement this use case in top down manner, creating the needed services, entities and other stuff as we see fit. This is where we decide for each part of the implementation if we simply use some old code, use a refactored version of some old code, or start from scratch. Once the use case is completed, we repeat the process, until we have all use cases (and thus completed both the red and yellow circles in this diagram). The next step then is to decide how we want to proceed with the presentation mechanism(s).
(https://www.youtube.com/watch?v=Nsjsiz2A9mg, https://blog.8thlight.com/assets/posts/2012-08-13-the-clean-architecture/CleanArchitecture-81565aba46f035911a5018e77a0f2d4e.jpg)
**Use case identification**
Visualization of conceptual components and their dependencies in the current codebase:
{F2624061}
Identified use cases:
## Display specific page from the CMS
Which page is specified in url, CMS may return content not found
GET request
URL: https://test.wikimedia.de/spenden/PAGENAME
new URL will probably something like https://test.wikimedia.de/page/PAGENAME
## Validate Email
Validate email structure & do MX check
GET request
URL: https://test.wikimedia.de/ajax.php?module=action&action=checkEmail&eaddr=gb@blah.com
Success response: `{"status":"OK"}`
Error response: `{"status":"ERR"}`
## Validate Bank Account
Validate bank data and return full set of bank data (converted from input)
GET request
URL for IBAN: https://test.wikimedia.de/ajax.php?module=action&action=checkIban&iban=DE12500105170648489890
URL for acct. number: https://test.wikimedia.de/ajax.php?module=action&action=generateIban&bankCode=50010517&accNum=0648489890
Success response `{"bic":"INGDDEFFXXX","iban":"DE12500105170648489890","account":"0648489890","bankCode":"50010517","bankName":"ING-DiBa","status":"OK"}`
Error response: `{"status":"ERR"}`
## Add Donation
Add entry to donation table and send confirmation email. Redirect to appropriate payment processor if needed.
Verify the request's integrity and check for irregularities (i. e. bad words, exceeding donation amount limit). Refuse insertion of data if integrity check fails, set status to pending on irregularities. Must handle locale/country-dependent data (eg. different number format with comma as decimal separator, different ZIP code lengths depending on country, etc).
POST request, application/x-www-form-urlencoded
URL https://test.wikimedia.de/spenden/spende.php
Request Body: betrag_auswahl=5%2C00&betrag=&intervalType=0&zahlweise=BEZ&debit-type=non-sepa&iban=DE12500105170648489890
&bic=INGDDEFFXXX&konto=0648489890&blz=50010517&adresstyp=person&anrede=Herr&titel=Kein+Titel&vorname=jhebk&nachname=veebwk&firma=&strasse=mevbek3b2hn&plz=11234&ort=Berlin&country=DE&email=gb%40blah.com
&go_prepare--pay%3Aeinzug=&periode=0&bankname=ING-DiBa&iframe=&form=10h16_Confirm&appeal=&from_form=10h16_Form2_Ticker_Intrvar
&tracking=&source=web&referrer=https%3A%2F%2Ftest.wikimedia.de%2Fspenden%2F&wikilogin=&layout=form%3D10h16_Form1
&color=&impCount=0&bImpCount=0&skin=10h16&from_pay=&utoken=1ad43457353b285d23b109df379424b80f000d46&stamp=2016-01-04+15%3A52%3A45&PHPSESSID=nsh4hui3055nl76v9873invn33&appealExpanded=&back_form=10h16_Form1
Request model parameters
- betrag
- zahlweise
- periode
- iban
- bic
- konto
- blz
- bankname
- adresstyp
- anrede
- titel
- firma
- vorname
- nachname
- firma
- strasse
- plz
- ort
- country
- email
The required field set depends on the address type:
- anonymous: all address fields can be left out
- firma: vorname and nachname can be left out
- person: firma can be left out
Additional parameters for the endpoint (presentation-related):
- tracking (set of piwik campaign and keyword (e. g. campaignName/bannerName)
- source (host name of referring site)
- impCount (total impressions before donating)
- bImpCount (current banner's impressions before donating)
- form
- from_form: current form
- layout
- color
- skin
- back_form: Form before the current form
## Add Comment
Append a comment to the donation data.
- All payment types except UEB (bank transfer), because we want to have a validated payment.
- After a donation (but not membership), the user is being asked for a comment.
- Can only be submitted
URL: https://test.wikimedia.de/ajax.php?module=action&action=addComment
Request body: kommentar=fjhwbjev+ejbjvbffjfhjf2hf2&eintrag=2&public=1&tracking=&source=web&referrer=https%3A%2F%2Ftest
.wikimedia.de%2Fspenden%2F&wikilogin=&layout=form%3D10h16_Form1&color=&impCount=0&bImpCount=0&skin=10h16
&token=1276888%2459b42194b31d0265df452735f6438a234bae2af8&sid=2343&utoken=b5b249c8beefb986faf8d186a3f16e86ef509ab3
&PHPSESSID=nsh4hui3055nl76v9873invn33
Error Response: `{"status":"ERR","message":"Der Datensatz kann nicht mehr ver\u00e4ndert werden. Bitte wenden Sie sich an unsere Hotline."}`
Success response: `{"status":"OK","message":"Something"}`
Relevant fields: kommentar, public, eintrag, token, utoken, sid
## List public donations and comments
List comments in three formats: HTML page, RSS feed, JSON
Make name of anonymous donor translatable
GET request
Current URLs:
- https://spenden.wikimedia.de/spenden/list.php
- https://spenden.wikimedia.de/spenden/rss.php
- https://spenden.wikimedia.de/spenden/json.php
URLs apparently not used anymore:
- https://spenden.wikimedia.de/spenden/html.php
- https://spenden.wikimedia.de/spenden/latest.php
json.php, html.php, and latest.php accept one request parameter:
- n (number of entries to return)
json.php accepts additional parameters:
- f (callback method wrapper for jsonp requests)
- anon (include anonymous entries)
json.php Response:
- betrag
- spender
- kommentar
- datum
- id
Parameters:
- Paging (number of entries per page, page number or start date)
- show anonymous comments yes/no
## Add Membership Request
Generate entry in "request" table and send confirmation email
Verify the request's integrity and check for irregularities (i. e. bad words, exceeding membership fee limit). Refuse insertion of data if integrity check fails, set status to pending on irregularities.
URL: https://test.wikimedia.de/spenden/contact.php
POST request
Request Body:
membership_type=sustaining&adresstyp=person&anrede=Herr&titel=Kein+Titel&vorname=blah&nachname=blub&firma=&strasse=cnkevhwnw&plz=11207&ort=Berlin&country=DE&email=gb%40blah.com&dob=30.02.9999&phone=1234555&membership_fee_interval=12&membership_fee=25%2C00&membership_fee_custom=&debit-type=sepa&iban=DE12500105170648489890
&bic=INGDDEFFXXX&account_number=0648489890&bank_code=50010517&go_contact=&purpose=membership-full&back_form=Mitgliedschaft&member_agree=1&account_holder=blank&bank_name=ING-DiBa&wikilogin=
Request model parameters: All from request body except back_form and wikilogin
## Add Subscription Request
Save personal data from a contact form in the "request" table (marked unconfirmed) and send a confirmation email with a link containing a unique ID.
Feature is partly implemented on wikipedia de and fundraising page, but not finished.
POST request
URL: https://test.wikimedia.de/ajax.php?module=action&action=subscribe
## Confirm Subscription Request
Look up a user from the "request" table by unique ID and mark the entry as confirmed
GET request
URL: https://test.wikimedia.de/spenden/?action=confirmSubscription&id=UNIQUE_ID
## Cancel Donation
mark donation as canceled and send email confirming the cancellation.
Can only be called if utoken is not expired.
URL: https://test.wikimedia.de/spenden/storno.php
Request model parameters:
- token
- sid
- utoken
## Handle Callback (PayPal Handler)
Handle all payment notification requests from PayPal.
POST request (must be validated, HTTP 406 error if other method was used)
URL: https://test.wikimedia.de/spenden/paypal_handler.php
Request model parameters:
- payment_status, see https://developer.paypal.com/webapps/developer/docs/classic/ipn/integration-guide/IPNandPDTVariables/#id091EB04C0HS Only some are handled (see below)
- txn_type: Transaction type, see https://developer.paypal.com/webapps/developer/docs/classic/ipn/integration-guide/IPNandPDTVariables/#id08CTB0S055Z Of the `txn_type` beginning with `subscr_`, only `subscr_payment` is handled, the others are silently dropped.
- item_number, spenden_id, sid. Donation ID (from GET/POST params), pick first non-null value from this list
- custom: JSON-encoded custom data:
- sid, id. Donation ID. Pick first non-null value from this list if donation id was not in in GET/POST from previous data
- token
- utoken
Values that are being stored for documentation purpose only:
- paypal_payer_id => payer_id
- paypal_subscr_id => subscr_id
- paypal_payer_status => payer_status
- paypal_address_status => address_status
- paypal_mc_gross => mc_gross
- paypal_mc_currency => mc_currency
- paypal_mc_fee => mc_fee
- paypal_settle_amount => settle_amount
Values that are being stored for later export:
- txn_id => ext_payment_id
- payment_type => ext_payment_type
- payment_status => ext_payment_status
- payer_id => ext_payment_account
- ext_payment_timestamp (generated at runtime)
- subscr_id => ext_subscr_id
As the first step, the handler must post back the raw POST data (with the additional field `cmd=_notify-validate`) to the PayPal URL to signal that the request was received (See https://developer.paypal.com/webapps/developer/docs/classic/ipn/integration-guide/IPNImplementation/#specs). If posting back was not successful (got no `200 OK` back or response content is not `VERIFIED`), the script has to quit.
Now the script does different things, depending on `payment_status` (see below).
### `Pending`
If we're using the PayPal sandbox, set to `Processed`, otherwise ignore. Check if the sandbox still does not send `Processed`, otherwise just ignore.
### `Completed` and `Processed`
Check if a related donation exists. If not, create a new record from the PayPal data and process it.
If the donation id is not empty and handle according to status. Do nothing except logging for states that don't make sense. Create a new donation record (with data from the old record) when status is "B" (bezahlt) and `txn_type` is `subscr_payment`. Process (new) donation.
The payment processor does **not** check the utoken, only the token, because payments can come in up to 5 days after creating the donation.
### Other status strings
Log and do nothing. In the future we should check with the fundraising team, which states make sense in our context and which are already handled by defaults and Backend Application actions.
See https://github.com/wmde/fundraising/issues/970#issuecomment-164474583
## Handle Callback (Credit Card Handler)
Called by the payment provider to confirm a payment. Validates the submitted data updates the status of the donation and triggers the confirmation mail. Callbacks concerning failed transactions are currectly being ignored, but a log entry should probably be added to the data set.
The submitted data is validated against the data set using the primary key, utoken, and donation amount. The status is updated only if the three values match, otherwise the handler responds with an error message. An additional requirement is to determine the credit card's expiration date by using the payment provider's API.
GET request
URL: https://test.wikimedia.de/spenden/mcp-handler.php
Request model parameters:
- function (billing for successful transactions; error for failed transactions)
- amount (in cents)
- auth (16B; hex representation)
- customerId (20B; hex representation; only set on function=billing)
- sessionId (20B; hex representation; only set on function=billing)
- transactionId (e. g. spenden.wikimedia.de-IDv5mgzw10kp)
- utoken (20B; hex representation; created when adding the donation data set and passed to the payment provider)
- donation_id (primary key of the donation data set)
- errorcode (as defined in techdoc.micropayment.de; only on function=error)
- errormessage (as defined in techdoc.micropayment.de; only on function=error)
- title (transaction description as printed on the invoice)
- country (ISO 3166-1 alpha-2)
- currency (ISO 4217)
- token
- testmode
The serialized data array in the data set is being updated with the passed values. The values are mapped to different keys, though:
Values that are being stored for documentation purpose only:
- currency => mcp_currency
- amount => mcp_amount
- country => mcp_country
- auth => mcp_auth
- title => mcp_title
- sessionId => mcp_sessionid
Values that are being stored for later export:
- transactionId => ext_payment_id
- function => ext_payment_type
- status => ext_payment_status
- customerId => ext_payment_account
- mcp_cc_expiry_date (needs to be determined by requesting the payment provider's API)
- ext_payment_timestamp (generated at runtime)
## Remaining use cases to discuss
* Confirm donation (direct debit only at the moment, should probably be handled by "Add Donation" use case instead)
* List Recent Donations (for "Donation Liveticker")
- list includes N recent donations
- each entry consists of donation date and time, and donation amount
- this list covers all donations, not only those that have a donor added a comment
**The new code**
So far the idea is to put the rewrite in a new git repository at https://github.com/wmde/FundraisingFrontend. This way we get rid of any issues present in the old git repository history and can easily see what has been "rewritten" and what still needs to be done. (The code already there is for exploration and demonstration purposes and is not intended to be used or seen as a replacement of the original code.)
**Next steps**
(In no particular order)
* Implement "list comments" use case to get a better idea of the amount of work involved and the types of issues we'll run into (to be able to better estimate total project cost).
* Describe use cases in more detail, figure out which ones actually describe multiple use cases and which ones we missed.