Background

As part of the donor identification project we will need a script that can parse a CSV file exported by CiviCRM and translate it into user preferences. This work will build on the preference added in T416946: Add a preference for receiving donor information from CiviCRM and the export provided by T416948: CiviCRM should export a CSV that can be used in a MediaWiki maintenance script to update the user preference table based on the contents of the CSV file.

User story

As a user who has consent donor identification I want MediaWiki have awareness of how recently I last donated so that it can tailor my experience accordingly.

Requirements

• The maintenance script lives in WikimediaCustomizations
• A user preference is added (initially hidden) to support donor identification. It should be consistent with the preference proposed in T416946: Add a preference for receiving donor information from CiviCRM and associated patch.

The script will process every file in the provided CSV as follows:

• If the e-mail address does not correspond with any user in the user table, the row will be discarded.
• If the e-mail address corresponds with a user who has not consented to donor identification the row will be discarded.
• If the e-mail address corresponds with a user that has not confirmed their email address the row will be discarded.
• If the email address corresponds with a user that has confirmed their email address and given consent, a donor preference will be derived from the donor segment provided by the CSV. For now derive this by function (a) => a but see assumptions section that this will change in future. This will override any existing value.
• After the CSV has been fully processed, the CSV will be discarded (or will notify an external service that this has been done)
• The script takes a dry-run parameter to allow us to update
• The script should be able to handle large CSVs e.g. 1 million rows. If you run into issues with this please document those (it is okay to handle them in a follow up task)
• Never output email addresses to the logs

Donor segment updating

• For the initial pass we'll set the donor preference to the value in the CSV but the solution should allow for change to this logic in future. For example we may decide to set the user preference to 2 for all donors before going live for privacy purposes. Make sure the script is flexible to support mapping the values to other values in future.

Assumptions

• You can assume the job will be run automatically.
• You can assume that will not be running the job until further review (feel free to merge something incomplete or defer merging the patch to a later task)
• you can assume the CSV does not contain duplicate rows (e.g. the email address is unique for each row)
• You can assume an SRE review will occur prior to use in a production environment
• You can assume T416946: Add a preference for receiving donor information from CiviCRM is or will be soon resolved.
• You can assume that the CSV will contain rows in the form x@x.com,300 (but please verify that against the work in T416948: CiviCRM should export a CSV that can be used in a MediaWiki maintenance script)
• Assume that we will need to map the donor segment numbers in the CSV to another number by a function that it yet to be determined. For the first version it is fine to use as is as we just want to demonstrate the solution works end to end.
• You can assume the preference has already been added e.g. the gerrit patch here has been merged.

BDD

• For QA engineer to fill out

Test Steps

• For QA engineer to fill out

Design

• Add mockups and design requirements

Acceptance criteria

• We have an unmerged patch that is capable of parsing the file exported by CiviCRM
• We have an understanding of how this script would fare processing thousands/millions of rows (how will it batch).

Communication criteria - does this need an announcement or discussion?

• Add communication criteria

Sign off

This task was created by Version 1.2.0 of the Web team task template using phabulous