Page MenuHomePhabricator

Log full bounce email too before unsubscribing
Closed, ResolvedPublic

Description

Currently, on un-subscribing, we save the log as:

"Un-subscribed global user $originalEmail for exceeding Bounce Limit $this->bounceRecordLimit"

Since there has been T99444 where a user was unsubscribed due to the ISP bouncing back emails ( can be soft or hard according to the configuration of its mail server ), we thought about keeping full log of the bounce email too so that we can later analyse which bounce and headers caused for this permanent action.

Considering ~10Kib/boune email, according to @hoo, we should be looking at:
11:27 PM <hoo> We received 204340 bounces during the period we have logs in the table for (90 days?)
11:29 PM <hoo> That would be ~2GiB for all bounces we have right now

Event Timeline

01tonythomas raised the priority of this task from to Needs Triage.
01tonythomas updated the task description. (Show Details)
01tonythomas added subscribers: 01tonythomas, Jgreen, hoo and 2 others.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 20 2015, 7:21 AM

Change 212907 had a related patch set uploaded (by Nemo bis):
Match a delivery status code separated by space

https://gerrit.wikimedia.org/r/212907

Logging the entire email is impossible for privacy reasons and parsing the entire message is hard, but I think extracting the six digits from any line and logging them should be fine. Something very silly like expanding the regex as I just did might suffice.

Nemo_bis triaged this task as Normal priority.May 23 2015, 1:02 PM

Change 212907 merged by jenkins-bot:
Simplify and match a delivery status code separated by space

https://gerrit.wikimedia.org/r/212907

Reedy updated the task description. (Show Details)Nov 14 2015, 7:45 PM
Reedy added a subscriber: Reedy.

The count isn't quite so high now we've unsubscribed a lot of mails :P

today - 1 month = 4570 at the time of querying

Log rotated and gzipped, it's probably not that much data

Change 253081 had a related patch set uploaded (by Reedy):
Give email headers when bounce causes unsubscribe.

https://gerrit.wikimedia.org/r/253081

Change 253082 had a related patch set uploaded (by Reedy):
Give email headers when bounce causes unsubscribe.

https://gerrit.wikimedia.org/r/253082

Change 253082 merged by jenkins-bot:
Give email headers when bounce causes unsubscribe.

https://gerrit.wikimedia.org/r/253082

Change 253081 merged by jenkins-bot:
Give email headers when bounce causes unsubscribe.

https://gerrit.wikimedia.org/r/253081

Change 253172 had a related patch set uploaded (by 01tonythomas):
Log the entire bounce email too before invalidating email

https://gerrit.wikimedia.org/r/253172

Change 253172 merged by jenkins-bot:
Log the entire bounce email too before invalidating email

https://gerrit.wikimedia.org/r/253172

Reedy closed this task as Resolved.Nov 15 2015, 8:30 PM
Reedy claimed this task.