Page MenuHomePhabricator
Create Task
Maniphest T351407

Create a service that sends an email when a file matches
Closed, ResolvedPublic2 Estimated Story Points
Actions

Description

A service should be created that allows emailing a provided email address when a file is determined to be a match by PhotoDNA. This service should meet the following requirements:

  • The email is sent containing the filename, and if relevant, the specific version of the file that was determined to be a match.
  • If the email fails to send or for any other error, this should be logged so that the email can be re-attempted using a maintenance script created in T351406: Create a maintenance script to re-send emails for files determined to be a match through PhotoDNA
  • The email should handle a SHA-1 that has multiple filenames and/or revisions under a given filename without overloading the email with too much text.
Acceptance criteria
  • A service that meets the requirements above should be created
  • This service should be fully tested

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Follow-up changes for MediaModerationEmailer servicemediawiki/extensions/MediaModerationwmf/1.42.0-wmf.15+9 -53
Follow-up changes for MediaModerationEmailer servicemediawiki/extensions/MediaModerationmaster+9 -53
Add MediaModerationEmailer servicemediawiki/extensions/MediaModerationmaster+652 -7
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 Resolved• Dreamy_JazzT351200 [EPIC] Start automated scanning
 OpenNoneT351398 Scan the uploadstash table on WMF wikis automatically and hourly
 OpenNoneT351396 Create a maintenance script to check the uploadstash table
 DuplicateNoneT351400 Run the maintenance script scanning images in mediamoderation_scan on WMF wikis
 ResolvedkostajhT351544 [Sub EPIC] Start test wiki scan
 ResolvedNoneT351546 [Sub EPIC] Set up rows
ResolvedTchandersT350865 Import all existing images to the mediamoderation_scan table on WMF wikis
 Resolved• Dreamy_JazzT350863 Create maintenance script to import all existing images to mediamoderation_scan table
 Resolved• Dreamy_JazzT350323 Write an empty row to scan table on file upload
 Resolved• Dreamy_JazzT350864 [M] Create a service to manage rows in the mediamoderation_scan table
 ResolvedBUG REPORT• Dreamy_JazzT352234 MediaModerationFileProcessor::canScanFile incorrectly registers audio and video files as scannable when TimedMediaHandler extension is installed
 Resolved• Dreamy_JazzT351477 [S] Explore how many rows in the file tables have duplicated timestamp values on WMF wikis
 ResolvedBUG REPORT• Dreamy_JazzT351719 When virtual domains mapping config uses 'db' as false, update.php does not create the table
 DeclinedPRODUCTION ERRORNoneT353391 Deferred update '....UploadCompleteHandler->onUploadComplete' failed to run.
 ResolvedTchandersT351628 Run the maintenance script scanning images in mediamoderation_scan on a test wiki
 Resolved• Dreamy_JazzT353442 Update MediaModerationPhotoDNAServiceProvider::getThumbnailForFile to choose a thumbnail that meets minimum height requirements for PhotoDNA API
 ResolvedBUG REPORT• Dreamy_JazzT353758 MediaModerationPhotoDNAServiceProvider provides false to FileBackend::getFileContents causing a PHP Notice
 Resolved• Dreamy_JazzT351399 Create a maintenance script to automatically scan files listed in mediamoderation_scan
 ResolvedNoneT351420 [Sub EPIC] Create Grafana charts, and Logstash filters
 Resolved• Dreamy_JazzT351419 Create a Grafana chart to plot the number of PhotoDNA requests per day per wiki
 ResolvedkostajhT353441 Collect statsd metrics of requests per day to PhotoDNA and response times from the PhotoDNA
 ResolvedkostajhT351401 Create service(s) to send an image to PhotoDNA for a scan
 ResolvedSTranT351421 Create a Grafana chart to plot the number of checked and unchecked images per wiki
 ResolvedkostajhT353703 Implement daily runs of updateMetrics on WMF wikis
 Resolved• Dreamy_JazzT353074 Create a maintenance script to publish to statsd metrics about the mediamoderation_scan table
 Resolved• Dreamy_JazzT351422 Create a logstash filter for MediaModeration related logs
 Resolved• Dreamy_JazzT351406 Create a maintenance script to re-send emails for files determined to be a match through PhotoDNA
 Resolved• Dreamy_JazzT351407 Create a service that sends an email when a file matches
 Resolved• Dreamy_JazzT351417 Create a service that creates LocalFile and ArchivedFile objects for files that have a given SHA-1
 Resolved• Dreamy_JazzT351416 [S] Explore how many rows in the file tables have duplicated SHA-1 values on WMF wikis
 Resolved• Dreamy_JazzT351409 Expand the database lookup service to get rows from the mediamoderation_scan table
 Resolved• Dreamy_JazzT351112 [S] Create an initial service to lookup scan rows from the database
 Resolved• Dreamy_JazzT350321 [M] Create database table to store status of scans

Event Timeline

• Dreamy_Jazz created this task.Nov 16 2023, 1:54 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 16 2023, 1:54 PM
• Dreamy_Jazz added a parent task: T351406: Create a maintenance script to re-send emails for files determined to be a match through PhotoDNA.Nov 16 2023, 1:54 PM
• Dreamy_Jazz mentioned this in T351399: Create a maintenance script to automatically scan files listed in mediamoderation_scan.
• Dreamy_Jazz added a parent task: T351399: Create a maintenance script to automatically scan files listed in mediamoderation_scan.
• Dreamy_Jazz mentioned this in T351396: Create a maintenance script to check the uploadstash table.Nov 16 2023, 1:58 PM
• Dreamy_Jazz added a parent task: T351396: Create a maintenance script to check the uploadstash table.
• Dreamy_Jazz added a project: Trust and Safety Product Sprint (Sprint Shamisen (20th Nov - 1st Dec. 2023)).Nov 20 2023, 4:33 PM
• Dreamy_Jazz set the point value for this task to 2.Nov 20 2023, 4:41 PM
• Dreamy_Jazz claimed this task.Dec 9 2023, 12:25 AM
• Dreamy_Jazz moved this task from Priority Backlog to In Progress on the Trust and Safety Product Sprint (Sprint Shamisen (20th Nov - 1st Dec. 2023)) board.
• Dreamy_Jazz added a subtask: T351417: Create a service that creates LocalFile and ArchivedFile objects for files that have a given SHA-1.
• Dreamy_Jazz edited projects, added Trust and Safety Product Sprint (Sprint Lute (11th Dec. - 22nd Dec. 2023)); removed Trust and Safety Product Sprint (Sprint Shamisen (20th Nov - 1st Dec. 2023)).Dec 11 2023, 4:43 PM
• Dreamy_Jazz closed subtask T351417: Create a service that creates LocalFile and ArchivedFile objects for files that have a given SHA-1 as Resolved.Dec 13 2023, 4:06 PM
• Dreamy_Jazz moved this task from Priority Backlog to In Progress on the Trust and Safety Product Sprint (Sprint Lute (11th Dec. - 22nd Dec. 2023)) board.Dec 15 2023, 4:47 PM
gerritbot added a comment.Dec 22 2023, 4:20 PM

Change 985172 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/MediaModeration@master] [Very WIP] Add MediaModerationEmailer service

https://gerrit.wikimedia.org/r/985172

gerritbot added a project: Patch-For-Review.Dec 22 2023, 4:20 PM
• Dreamy_Jazz moved this task from Sprint Lute (11th Dec. - 22nd Dec. 2023) to Sprint Northumbrian smallpipes (8th Jan.‘24 - 19th Jan.'24) on the Trust and Safety Product Sprint board.Jan 8 2024, 4:13 PM
• Dreamy_Jazz edited projects, added Trust and Safety Product Sprint (Sprint Northumbrian smallpipes (8th Jan.‘24 - 19th Jan.'24)); removed Trust and Safety Product Sprint (Sprint Lute (11th Dec. - 22nd Dec. 2023)).
• Dreamy_Jazz moved this task from Priority Backlog to Ready on the Trust and Safety Product Sprint (Sprint Northumbrian smallpipes (8th Jan.‘24 - 19th Jan.'24)) board.
• Dreamy_Jazz moved this task from Ready to In Progress on the Trust and Safety Product Sprint (Sprint Northumbrian smallpipes (8th Jan.‘24 - 19th Jan.'24)) board.Jan 12 2024, 10:30 PM
• Dreamy_Jazz moved this task from In Progress to Needs review on the Trust and Safety Product Sprint (Sprint Northumbrian smallpipes (8th Jan.‘24 - 19th Jan.'24)) board.Jan 18 2024, 6:00 PM
gerritbot added a comment.Jan 22 2024, 8:20 PM

Change 985172 merged by jenkins-bot:

[mediawiki/extensions/MediaModeration@master] Add MediaModerationEmailer service

https://gerrit.wikimedia.org/r/985172

Maintenance_bot removed a project: Patch-For-Review.Jan 22 2024, 8:31 PM
ReleaseTaggerBot added a project: MW-1.42-notes (1.42.0-wmf.15; 2024-01-23).Jan 22 2024, 10:01 PM
gerritbot added a comment.Jan 24 2024, 1:05 PM

Change 992683 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/MediaModeration@master] Follow-up changes for MediaModerationEmailer service

https://gerrit.wikimedia.org/r/992683

gerritbot added a project: Patch-For-Review.Jan 24 2024, 1:05 PM
• Dreamy_Jazz mentioned this in T351406: Create a maintenance script to re-send emails for files determined to be a match through PhotoDNA.Jan 24 2024, 1:44 PM
gerritbot added a comment.Jan 29 2024, 10:27 AM

Change 992683 merged by jenkins-bot:

[mediawiki/extensions/MediaModeration@master] Follow-up changes for MediaModerationEmailer service

https://gerrit.wikimedia.org/r/992683

Maintenance_bot removed a project: Patch-For-Review.Jan 29 2024, 10:30 AM
ReleaseTaggerBot edited projects, added MW-1.42-notes (1.42.0-wmf.16; 2024-01-30); removed MW-1.42-notes (1.42.0-wmf.15; 2024-01-23).Jan 29 2024, 11:00 AM
• Dreamy_Jazz edited projects, added Trust and Safety Product Sprint (Sprint Kazoo (Jan 29 - Feb 9 2024)); removed Trust and Safety Product Sprint (Sprint Northumbrian smallpipes (8th Jan.‘24 - 19th Jan.'24)).Jan 29 2024, 1:08 PM
kostajh moved this task from Priority Backlog to Needs QA on the Trust and Safety Product Sprint (Sprint Kazoo (Jan 29 - Feb 9 2024)) board.Jan 29 2024, 1:12 PM
gerritbot added a comment.Jan 29 2024, 2:49 PM

Change 993502 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/MediaModeration@wmf/1.42.0-wmf.15] Follow-up changes for MediaModerationEmailer service

https://gerrit.wikimedia.org/r/993502

gerritbot added a project: Patch-For-Review.Jan 29 2024, 2:49 PM
gerritbot added a comment.Jan 29 2024, 3:10 PM

Change 993502 abandoned by Dreamy Jazz:

[mediawiki/extensions/MediaModeration@wmf/1.42.0-wmf.15] Follow-up changes for MediaModerationEmailer service

Reason:

Not going to be done in the upcoming backport window and isn't necessarily to be backported.

https://gerrit.wikimedia.org/r/993502

Maintenance_bot removed a project: Patch-For-Review.Jan 29 2024, 3:31 PM
Djackson-ctr subscribed.Jan 30 2024, 1:56 AM

I have verified that the new code has been implemented and is functioning and displaying as expected...
Below you will see 2 screenshots that show the contents of the email for an image file...
Thank you @Dreamy_Jazz

A screenshot in Plain Text format:

image.png (719×1 px, 126 KB)

A screenshot in Source format:

image.png (980×1 px, 160 KB)

Djackson-ctr moved this task from Needs QA to Done on the Trust and Safety Product Sprint (Sprint Kazoo (Jan 29 - Feb 9 2024)) board.Jan 30 2024, 1:58 AM
kostajh closed this task as Resolved.Jan 30 2024, 5:28 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits