@Ejegg Adding Elliott to investigate as he developed the tool

For donor 397205, it does look like a bad link. That screencap seems to be the text-only version of the email. Has that been QA'ed? Can we check that the checksum token is there in the text-only version?

51838355: The link looks good. Can see them clicking on the upgrade form Mar 22 00:01:55 and all the resources loading with successful status (including the back-end request to Civi), but no submission. Perhaps there was an error in the Javascript that prevented the submission? We should add logging for client-side errors. Logs show browser as Chrome 122 on Chrome OS

54172937: Similar, initial form load is in logs but no record of attempted submission. Firefox 123 on Windows 10

12658493: Submit appears in logs for this one, but the status is '200' and not '302'. '200' means a successful page load, but we would expect 302 here to mean a redirect to the thank you page. I'd like to review which code paths could lead to that outcome.

8467494: Link looks fine, submit is in the logs, Safari on MacOS. Again, I'd like to add more logging of client-side errors, possibly including amount validation that would prevent form submission.

For donor 397205, I was able to retrieve their email address from Civi and try testing it within Acoustic. Their address couldn't be found in Acoustic's database though. Having checked the template itself, both the HTML and Text versions included the "checksum" URL parameter. All links had the following format:

https://donorpreferences.wikimedia.org/index.php?title=Special:RecurUpgrade&variant=v01&contact_id=%%ContactID%%&checksum=%%checksum%%&wmf_medium=email&wmf_campaign=C2324_UpgradeRecurring&link_id=[x]

For context, When building an email template, Acoustic allows you to plug-in an email address to populate any personalised data associated with that address and render a preview (including personalised hyperlinks).

As a final check, I "plugged-in" an email address that I know has a recurring donation and was able to render the text version of the email. Confirming that the text version hyperlinks worked as expected.

Change #1015076 had a related patch set uploaded (by Ejegg; author: Ejegg):

[mediawiki/extensions/DonationInterface@master] Extract base client-side error logging API

https://gerrit.wikimedia.org/r/1015076

Thanks for checking @ppenloglou! So I wonder how that full real URL gets translated to the long gibberish links.email.wikimedia.org URL that we see in the screenshot, and how the long URL gets translated back to the one with the full params. If it's something dynamic I guess it's possible that Acoustic could break a link after send. Could it have to do with the fact that their email address isn't in Acoustic?

Sure thing @Ejegg! Indeed you're right, Acoustic generates these links.email.wikimedia.org hyperlinks during send time. But I am unaware of how exactly they redirect back to the original URL format that's included in the template. Perhaps @bsisolak could shed more light on the matter.

No humans ever see the text version, so it's not that. If if they seeing the text version they are using some kind of crazy mail client that could be alerting links.

The links.email.wikimedia.org is encoded so that Acoustic knows who clicks the link in the email, etc. It then forwards the user to the URL @ppenloglou puts into the email. The data used to personalize the link is pulled from the Acoustic database when the user clicks the link.

One thing that can happen, if an email is sent on 3/1, and the recipient clicks the link on 3/15. I wonder if Civi has updated the data used in the database at that point, maybe that could cause an issue on your end?

I kind of recall talking about checksum, but not sure how that works.

Change #1015161 had a related patch set uploaded (by Ejegg; author: Ejegg):

[mediawiki/extensions/DonationInterface@master] Add new API action to log recurring upgrade errors

https://gerrit.wikimedia.org/r/1015161

I'm not sure if this will be helpful to you all so please disregard if not! :) @Ejegg @PPenloglou-WMF

To clarify re: CID 397205 when tickets come into Zendesk it strips the messages of all HTML, etc. So we always see this raw version of all messages such as shown above and in the image below.

However, if we select the "View Original Email" feature in Zendesk we can also see what the original email looked like to the donor and test the links that way as well. Below is a screenshot of part of the "original email" CID 397205 actually saw...

In the "original email" view if I click on either link the green button "Yes, I'll give a little more each month" or the blue link that starts "We'd like to ask:.." for this particular donor I get an error page.

Should it be helpful and you wish to view this ticket from within Zendesk this ticket is #1461060. :)

Thanks @AMJohnson, that is good information.

Hi @AMJohnson I can't seem to sign in to Zendesk either via Okta or with my old password, and the reset password link doesn't send me anything. Are you able to reactivate agent accounts?

Thanks @AMJohnson for the extra info! @Ejegg , I went to Zendesk myself and had a look at the original email. Clicked on the hyperlink which resolved to:

https://donorpreferences.wikimedia.org/index.php?title=Special:RecurUpgrade&variant=v01&contact_id=397205&checksum=&wmf_medium=email&wmf_campaign=C2324_UpgradeRecurring&link_id=b1&1=1

So suspiciously, the checksum parameter never loaded any data, that's why the error page is loaded. Some questions that come to mind (but are currently outside of my technical expertise):

• How is the checksum column populated for contacts? Could something have gone wrong there?
• That contact's email address wasn't in Acoustic's database when I started debugging but presumably it was there during send time. Was the checksum column null then?

@Ejegg Ignore my previous comment if you saw it. I re-activated your Zendesk access and resent a password request to your email. Hopefully you receive. :)

So suspect I know what happened, it struck me you said the email is not in Acoustic anymore. My guess is Civi merged two records - and in the process deleted the record the email was sent to. (which is a bigger issue if you are removing emails from the list, but that's for another ticket) When the user clicked the link the personalization failed, as the record no longer exists. I suspect this is pretty edge case then. But, you could turn off link tracking on these emails, which would then populate the URL in the email itself at the time of send. That way even if Civi updates the record in Acoustic, the link in the email would be static. Is that change warranted for this kind of edge case I don't know, as you will lose link tracking data.

@bsisolak Very interesting insight into how Acoustic handles URLs. So you're saying that without click tracking, the URL that gets generated during send time will be created once and remain static? As opposed to with click tracking, each future click requests data anew from Acoustic?

I don't imagine we'll be turning off click tracking but this was indeed interesting...

• How is the checksum column populated for contacts? Could something have gone wrong there?

Every night we send a few files up to Acoustic. One has all the columns for donors in the _all_Wikimedia group who have some change in the last week. Another has just the emails to be suppressed (opt-outs and old emails from people whose email has changed). And the third is the checksum file. This has just the email and the checksum, which should be for every donor in the database.

Change #1015076 merged by jenkins-bot:

[mediawiki/extensions/DonationInterface@master] Extract base client-side error logging API

https://gerrit.wikimedia.org/r/1015076

Change #1015161 merged by jenkins-bot:

[mediawiki/extensions/DonationInterface@master] Add new API action to log recurring upgrade errors

https://gerrit.wikimedia.org/r/1015161