Page MenuHomePhabricator
Create Task
Maniphest T297159

Switch eoy email over fully to the new method
Closed, ResolvedPublic
Actions

Description

Task for fr-test sending side of it

@Dwisehaupt @Jgreen do you have any thoughts as to whether we could safely test a mailin job on staging without any mail actually going out - ie a smtp server that doesn't go anywhere? My guess is it's too late for this round but I thought I'd ask the question

Details

Related Changes in Gerrit:
SubjectAuthorRepoBranchLines +/-
Fix text only version of eoy_template to not break some tokensEileenwikimedia/fundraising/crmmaster+11 -1
Add unit test for single quote in nameEileenwikimedia/fundraising/crmmaster+50 -16
Handle single quote in nameEileenwikimedia/fundraising/crm/civicrmmaster+1 -1
Handle quotes in tokensEileenwikimedia/fundraising/crmmaster+7 -7
Clean up MakeJobEileenwikimedia/fundraising/crmmaster+14 -50
Check status when checking if job is emptyEileenwikimedia/fundraising/crmmaster+1 -1
Improve UI when contact does not have a valid emailEileenwikimedia/fundraising/crmmaster+41 -29
Fix bug where no language was falling back to first translationEileenwikimedia/fundraising/crmmaster+16 -11
Cleanup on sendLettersEileenwikimedia/fundraising/crmmaster+18 -5
Alter query to ensure email is uniqueEileenwikimedia/fundraising/crmmaster+20 -18
Finally fully remove the drupal moduleEileenwikimedia/fundraising/crmmaster+0 -67
Do not create a job when there is contact idEileenwikimedia/fundraising/crmmaster+14 -12
Update tests to test sent mail, not db tableEileenwikimedia/fundraising/crmmaster+174 -48
Add year column to wmf_eoy_receipt_donorEileenwikimedia/fundraising/crmmaster+32 -27
Fix error - use custom fields not core fields for original currencyEileenwikimedia/fundraising/crm/civicrmmaster+7 -4
Add limit to EoyEMail::sendEileenwikimedia/fundraising/crmmaster+1 -1
Remove reference to legacy propertyEileenwikimedia/fundraising/crmmaster+0 -7
Test readability improvementEileenwikimedia/fundraising/crmmaster+15 -26
Set original currency details in testEileenwikimedia/fundraising/crmmaster+4 -0
Add Send api, move tests over.Eileenwikimedia/fundraising/crmmaster+96 -16
Goodbye old templatesEileenwikimedia/fundraising/crmmaster+0 -971
Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

Eileenmcnaughton created this task.Dec 6 2021, 9:24 PM
Restricted Application removed a project: Patch-For-Review. · View Herald TranscriptDec 6 2021, 9:24 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
• DStrine moved this task from Triage to Current Sprint on the Fundraising-Backlog board.Dec 6 2021, 11:14 PM
gerritbot added a comment.Dec 7 2021, 1:04 AM

Change 744108 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Goodbye old templates

https://gerrit.wikimedia.org/r/744108

gerritbot added a project: Patch-For-Review.Dec 7 2021, 1:04 AM

Change 744109 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Add Send api, move tests over.

https://gerrit.wikimedia.org/r/744109

Eileenmcnaughton updated the task description. (Show Details)Dec 7 2021, 1:12 AM
Eileenmcnaughton added subscribers: Dwisehaupt, Jgreen.
Jgreen added a comment.Dec 7 2021, 4:16 PM

@Eileenmcnaughton testing on frdev1001 is definitely feasible but we need to make some configuration changes to insure messages can't leave the server. I'll look at this today.

gerritbot added a comment.Dec 7 2021, 6:14 PM

Change 744108 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Goodbye old templates

https://gerrit.wikimedia.org/r/744108

gerritbot added a comment.Dec 7 2021, 6:27 PM

Change 744109 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Add Send api, move tests over.

https://gerrit.wikimedia.org/r/744109

gerritbot added a comment.Dec 7 2021, 7:41 PM

Change 744855 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Remove reference to legacy property

https://gerrit.wikimedia.org/r/744855

Eileenmcnaughton claimed this task.Dec 7 2021, 8:18 PM
• DStrine added a project: Fundraising Sprint Xenomorph Petting Zoo.Dec 7 2021, 9:32 PM
• DStrine triaged this task as High priority.Dec 8 2021, 5:01 PM
Eileenmcnaughton moved this task from Backlog to Doing on the fundraising sprint Wireless Zipline board.Dec 8 2021, 9:30 PM
gerritbot added a comment.Dec 8 2021, 9:31 PM

Change 745330 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm/civicrm@master] Fix error - use custom fields not core fields for original currency

https://gerrit.wikimedia.org/r/745330

gerritbot added a comment.Dec 8 2021, 11:04 PM

Change 745336 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Set original currency details in test

https://gerrit.wikimedia.org/r/745336

gerritbot added a comment.Dec 8 2021, 11:05 PM

Change 745337 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Test readability improvement

https://gerrit.wikimedia.org/r/745337

gerritbot added a comment.Dec 9 2021, 12:02 AM

Change 745336 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Set original currency details in test

https://gerrit.wikimedia.org/r/745336

gerritbot added a comment.Dec 9 2021, 12:26 AM

Change 745337 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Test readability improvement

https://gerrit.wikimedia.org/r/745337

gerritbot added a comment.Dec 9 2021, 4:40 AM

Change 745362 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Add limit to EoyEMail::send

https://gerrit.wikimedia.org/r/745362

Jgreen closed subtask T297304: set up catch-all mail handling on fundraising staging server as Resolved.Dec 9 2021, 10:17 PM
gerritbot added a comment.Dec 9 2021, 11:35 PM

Change 744855 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Remove reference to legacy property

https://gerrit.wikimedia.org/r/744855

gerritbot added a comment.Dec 9 2021, 11:35 PM

Change 745362 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Add limit to EoyEMail::send

https://gerrit.wikimedia.org/r/745362

gerritbot added a comment.Dec 10 2021, 12:01 AM

Change 745330 merged by jenkins-bot:

[wikimedia/fundraising/crm/civicrm@master] Fix error - use custom fields not core fields for original currency

https://gerrit.wikimedia.org/r/745330

Eileenmcnaughton moved this task from Backlog to Doing on the Fundraising Sprint Xenomorph Petting Zoo board.Dec 10 2021, 1:59 AM
gerritbot added a comment.Dec 10 2021, 2:35 AM

Change 745645 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Add year column to wmf_eoy_receipt_donor

https://gerrit.wikimedia.org/r/745645

gerritbot added a comment.Dec 12 2021, 9:04 PM

Change 746013 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Update tests to test sent mail, not db table

https://gerrit.wikimedia.org/r/746013

gerritbot added a comment.Dec 12 2021, 9:57 PM

Change 746015 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Do not create a job when there is contact id

https://gerrit.wikimedia.org/r/746015

gerritbot added a comment.Dec 12 2021, 11:04 PM

Change 746016 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Finally fully remove the drupal module

https://gerrit.wikimedia.org/r/746016

gerritbot added a comment.Dec 13 2021, 12:58 AM

Change 746020 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Alter query to ensure email is unique

https://gerrit.wikimedia.org/r/746020

gerritbot added a comment.Dec 13 2021, 1:43 AM

Change 745645 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Add year column to wmf_eoy_receipt_donor

https://gerrit.wikimedia.org/r/745645

Eileenmcnaughton added a comment.Dec 13 2021, 5:58 AM

Note I've done some documenting at https://wikitech.wikimedia.org/wiki/Fundraising/Internal_Endpoints/End_of_year_emails

gerritbot added a comment.Dec 14 2021, 5:18 AM

Change 746992 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Fix bug where no language was falling back to first translation

https://gerrit.wikimedia.org/r/746992

gerritbot added a comment.Dec 14 2021, 7:37 PM

Change 746013 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Update tests to test sent mail, not db table

https://gerrit.wikimedia.org/r/746013

gerritbot added a comment.Dec 15 2021, 9:08 PM

Change 747612 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Cleanup on sendLetters

https://gerrit.wikimedia.org/r/747612

gerritbot added a comment.Dec 15 2021, 10:48 PM

Change 747625 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Improve UI when contact does not have a valid email

https://gerrit.wikimedia.org/r/747625

gerritbot added a comment.Dec 16 2021, 12:00 AM

Change 747638 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Handle quotes in tokens

https://gerrit.wikimedia.org/r/747638

gerritbot added a comment.Dec 16 2021, 3:27 AM

Change 746015 merged by Eileen:

[wikimedia/fundraising/crm@master] Do not create a job when there is contact id

https://gerrit.wikimedia.org/r/746015

gerritbot added a comment.Dec 16 2021, 4:02 AM

Change 747673 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Clean up MakeJob

https://gerrit.wikimedia.org/r/747673

gerritbot added a comment.Dec 16 2021, 4:22 AM

Change 746016 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Finally fully remove the drupal module

https://gerrit.wikimedia.org/r/746016

gerritbot added a comment.Dec 16 2021, 4:48 AM

Change 746020 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Alter query to ensure email is unique

https://gerrit.wikimedia.org/r/746020

gerritbot added a comment.Dec 16 2021, 5:02 AM

Change 747679 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Check status when checking if job is empty

https://gerrit.wikimedia.org/r/747679

gerritbot added a comment.Dec 16 2021, 8:58 PM

Change 747612 merged by Eileen:

[wikimedia/fundraising/crm@master] Cleanup on sendLetters

https://gerrit.wikimedia.org/r/747612

gerritbot added a comment.Dec 16 2021, 10:36 PM

Change 746992 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Fix bug where no language was falling back to first translation

https://gerrit.wikimedia.org/r/746992

Ejegg moved this task from Doing to Review on the Fundraising Sprint Xenomorph Petting Zoo board.Dec 16 2021, 10:39 PM
gerritbot added a comment.Dec 16 2021, 10:48 PM

Change 747625 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Improve UI when contact does not have a valid email

https://gerrit.wikimedia.org/r/747625

gerritbot added a comment.Dec 17 2021, 2:22 AM

Change 747679 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Check status when checking if job is empty

https://gerrit.wikimedia.org/r/747679

gerritbot added a comment.Dec 17 2021, 2:52 AM

Change 747673 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Clean up MakeJob

https://gerrit.wikimedia.org/r/747673

gerritbot added a comment.Dec 20 2021, 9:31 PM

Change 747638 abandoned by Eileen:

[wikimedia/fundraising/crm@master] Handle quotes in tokens

Reason:

I have a new plan

https://gerrit.wikimedia.org/r/747638

gerritbot added a comment.Dec 20 2021, 10:39 PM

Change 748834 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm/civicrm@master] Handle single quote in name

https://gerrit.wikimedia.org/r/748834

gerritbot added a comment.Dec 20 2021, 10:40 PM

Change 748833 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Add unit test for single quote in name

https://gerrit.wikimedia.org/r/748833

Eileenmcnaughton added a comment.Dec 21 2021, 1:41 AM

Moving this to review as I have

  1. updated docs https://wikitech.wikimedia.org/wiki/Fundraising/Internal_Endpoints/End_of_year_emails
  2. updated jobs on process control
  3. 1 remaining fix https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/civicrm/+/748834 along with a test for it https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/748833 and a very superficial fix https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/747988
gerritbot added a comment.Dec 21 2021, 5:05 AM

Change 748834 merged by jenkins-bot:

[wikimedia/fundraising/crm/civicrm@master] Handle single quote in name

https://gerrit.wikimedia.org/r/748834

Ejegg moved this task from Review to Pending Deployment on the Fundraising Sprint Xenomorph Petting Zoo board.Dec 21 2021, 8:17 PM
Ejegg moved this task from Pending Deployment to Review on the Fundraising Sprint Xenomorph Petting Zoo board.
• DStrine added a project: fundraising sprint Yeet-coaster.Dec 21 2021, 9:11 PM
gerritbot added a comment.Dec 21 2021, 10:15 PM

Change 748833 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Add unit test for single quote in name

https://gerrit.wikimedia.org/r/748833

Eileenmcnaughton moved this task from Backlog to Review on the fundraising sprint Yeet-coaster board.Dec 21 2021, 10:24 PM
Eileenmcnaughton moved this task from Review to Done on the fundraising sprint Yeet-coaster board.Dec 21 2021, 11:08 PM
gerritbot added a comment.Jan 6 2022, 4:51 AM

Change 751857 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Fix text only version of eoy_template to not break some tokens

https://gerrit.wikimedia.org/r/751857

Eileenmcnaughton added a comment.EditedJan 6 2022, 4:59 AM

I started looking at T298664 and applying later civi code triggered a test fail. On digging it is a real issue with eoy emails - but an obscure one.

We send out an html version AND a text version of the emails. The text version is ONLY seen in email clients configured not to show htm - which is very rare these days. The code that inserted the text version into the DB mangled the tokens within the html bold tags - converting

Your {$year} total was {$total.currency} {$total.amount}.

to

YOUR {$YEAR} TOTAL WAS {$TOTAL.CURRENCY} {$TOTAL.AMOUNT}.

Since those are not valid we wind up with

YOUR TOTAL WAS .

Not great but not that bad in context as the next line is

TOTAL DONATIONS TO WIKIMEDIA FOUNDATION:

so it only seems a bit odd if you only gave in one currency (and more odd for multiple currencies)

My basic feeling is the is very edge & not that bad but a little more bad if there is a compounding edge case. I put up a patch that fixes but only on dev.

I could manually fix on live - but my gut says fixing at this point has some risk and I land on do nothing being the best option

gerritbot added a comment.Jan 6 2022, 9:55 PM

Change 751857 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Fix text only version of eoy_template to not break some tokens

https://gerrit.wikimedia.org/r/751857

• DStrine closed this task as Resolved.Jan 18 2022, 8:16 PM
• DStrine set Final Story Points to 4.
• DStrine closed subtask T299011: Deal with EOY emails that did NOT go out as Resolved.Feb 1 2022, 8:10 PM
Cstone mentioned this in T325596: Check that end of year email receipt templates are ready for January 2023 send.Dec 19 2022, 9:44 PM
Cstone mentioned this in T353206: Check that end of year email receipt templates are ready for January 2024 send.Dec 11 2023, 11:01 PM
Cstone mentioned this in T380330: Check that end of year (EOY) email receipt templates are ready for January 2025 and send them.Nov 19 2024, 10:58 PM
Cstone mentioned this in T413373: After January 12 - Check that end of year (EOY) email receipt templates are ready for January 2026 and send them.Dec 22 2025, 4:47 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits