Attempt to send original file to PhotoDNA for the scan if thumbnail fails
Closed, ResolvedPublic3 Estimated Story Points
Actions

Description

The process of generating a thumbnail to be sent to PhotoDNA is complicated and may fail. It also does not currently work for files which have been deleted (ArchivedFile objects). If a thumbnail cannot be generated, then the source file could be sent to PhotoDNA for scanning if it is in a supported format.

The list of supported formats is relatively large, such that most image files could be sent directly to PhotoDNA. However, just sending the source file may cause problems if the original file is very large.

This also allows reducing the number of images affected by T353853, as images that are at least 160px by 160px but no more than 320px wide will now be scannable.

Details

Related Changes in Gerrit:

Subject	Repo	Branch	Lines +/-
Attempt to send original file to PhotoDNA if no thumbnail	mediawiki/extensions/MediaModeration	wmf/1.42.0-wmf.10	+902 -306
Attempt to send original file to PhotoDNA if no thumbnail	mediawiki/extensions/MediaModeration	wmf/1.42.0-wmf.12	+902 -306
Attempt to send original file to PhotoDNA if no thumbnail	mediawiki/extensions/MediaModeration	master	+902 -306

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		• Dreamy_Jazz	T351200 [EPIC] Start automated scanning
		Resolved		• Dreamy_Jazz	T353854 Attempt to send original file to PhotoDNA for the scan if thumbnail fails

Event Timeline

• Dreamy_Jazz created this task.Dec 21 2023, 1:08 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 21 2023, 1:08 AM

• Dreamy_Jazz renamed this task from Attempt to send original file to PhotoDNA for the scan over a thumbnail to Attempt to send original file to PhotoDNA for the scan if thumbnail fails.Dec 21 2023, 1:08 AM

• Dreamy_Jazz added a project: Trust and Safety Product Team.

• Dreamy_Jazz added a parent task: T351200: [EPIC] Start automated scanning.

• Dreamy_Jazz updated the task description. (Show Details)Dec 21 2023, 1:11 AM

• Dreamy_Jazz added a project: Trust and Safety Product Sprint (Sprint Lute (11th Dec. - 22nd Dec. 2023)).Jan 3 2024, 9:43 AM

Change 987410 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/MediaModeration@master] [WIP] Attempt to send original file to PhotoDNA if no thumbnail

https://gerrit.wikimedia.org/r/987410

gerritbot added a project: Patch-For-Review.Jan 3 2024, 10:34 AM

• Dreamy_Jazz claimed this task.Jan 3 2024, 10:35 AM

• Dreamy_Jazz moved this task from Priority Backlog to In Progress on the Trust and Safety Product Sprint (Sprint Lute (11th Dec. - 22nd Dec. 2023)) board.

• Dreamy_Jazz set the point value for this task to 3.

• Dreamy_Jazz updated the task description. (Show Details)Jan 3 2024, 10:47 AM

• Dreamy_Jazz moved this task from In Progress to Needs review on the Trust and Safety Product Sprint (Sprint Lute (11th Dec. - 22nd Dec. 2023)) board.Jan 3 2024, 9:41 PM

Change 987410 merged by jenkins-bot:

[mediawiki/extensions/MediaModeration@master] Attempt to send original file to PhotoDNA if no thumbnail

https://gerrit.wikimedia.org/r/987410

Change 987485 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/MediaModeration@wmf/1.42.0-wmf.12] Attempt to send original file to PhotoDNA if no thumbnail

https://gerrit.wikimedia.org/r/987485

Change 987726 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/MediaModeration@wmf/1.42.0-wmf.10] Attempt to send original file to PhotoDNA if no thumbnail

https://gerrit.wikimedia.org/r/987726

Change 987485 merged by jenkins-bot:

[mediawiki/extensions/MediaModeration@wmf/1.42.0-wmf.12] Attempt to send original file to PhotoDNA if no thumbnail

https://gerrit.wikimedia.org/r/987485

Mentioned in SAL (#wikimedia-operations) [2024-01-04T14:16:27Z] <tchanders@deploy2002> Started scap: Backport for [[gerrit:987485|Attempt to send original file to PhotoDNA if no thumbnail (T353854)]]

Mentioned in SAL (#wikimedia-operations) [2024-01-04T14:20:04Z] <tchanders@deploy2002> dreamyjazz and tchanders: Backport for [[gerrit:987485|Attempt to send original file to PhotoDNA if no thumbnail (T353854)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-01-04T14:25:51Z] <tchanders@deploy2002> Finished scap: Backport for [[gerrit:987485|Attempt to send original file to PhotoDNA if no thumbnail (T353854)]] (duration: 09m 24s)

Change 987726 merged by jenkins-bot:

[mediawiki/extensions/MediaModeration@wmf/1.42.0-wmf.10] Attempt to send original file to PhotoDNA if no thumbnail

https://gerrit.wikimedia.org/r/987726

Maintenance_bot removed a project: Patch-For-Review.Jan 4 2024, 2:30 PM

Mentioned in SAL (#wikimedia-operations) [2024-01-04T14:30:49Z] <tchanders@deploy2002> Started scap: Backport for [[gerrit:987726|Attempt to send original file to PhotoDNA if no thumbnail (T353854)]]

Mentioned in SAL (#wikimedia-operations) [2024-01-04T14:34:27Z] <tchanders@deploy2002> tchanders and dreamyjazz: Backport for [[gerrit:987726|Attempt to send original file to PhotoDNA if no thumbnail (T353854)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-01-04T14:40:14Z] <tchanders@deploy2002> Finished scap: Backport for [[gerrit:987726|Attempt to send original file to PhotoDNA if no thumbnail (T353854)]] (duration: 09m 25s)

• Dreamy_Jazz moved this task from Needs review to Needs QA on the Trust and Safety Product Sprint (Sprint Lute (11th Dec. - 22nd Dec. 2023)) board.Jan 8 2024, 2:16 PM

This task will be difficult to QA without using a production instance as it modified the code that is only used when using the PhotoDNA API key. As such I propose that QA is skipped, but will leave this up to the QA engineers to make a decision either way. If QA is desired, I can see if I can find a way to allow QA.

• Dreamy_Jazz moved this task from Sprint Lute (11th Dec. - 22nd Dec. 2023) to Sprint Northumbrian smallpipes (8th Jan.‘24 - 19th Jan.'24) on the Trust and Safety Product Sprint board.Jan 8 2024, 4:13 PM

• Dreamy_Jazz edited projects, added Trust and Safety Product Sprint (Sprint Northumbrian smallpipes (8th Jan.‘24 - 19th Jan.'24)); removed Trust and Safety Product Sprint (Sprint Lute (11th Dec. - 22nd Dec. 2023)).

• Dreamy_Jazz moved this task from Priority Backlog to Needs QA on the Trust and Safety Product Sprint (Sprint Northumbrian smallpipes (8th Jan.‘24 - 19th Jan.'24)) board.