[S] Suggestions should only be images
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	mfossati
	Apr 7 2023, 4:33 PM

Description

In order to prevent downstream bugs such as T332757: Invalid file error for Add image - OFFICE is not valid mime type , we should restrict the type of suggested files to images only.
The ideal solution would be to look up relevant MIME types, although we currently don't have such information available in our data pipeline outputs.

An alternative are file extensions: instead of filtering SVG files to avoid logos/icons as in T331456: [S] Filter out all .svg files from section-level image suggestions, we should keep image files.
Pasting below the set of all file extensions found in a previous Section-Level-Image-Suggestions run, 2023-01-30 snapshot:
SVG, JPEG, JPg, Svg, Jpg, PNG, Jpeg, Gif, JpG, pdf, djvu, JPeG, GIF, xcf, png, Tif, gif, jPeG, jpeg, JPG, Png, svg, tif, webp, BMP, stl, bmp, tiff, jPg, WebP, jpg, TIF, webm, JpEg, jPG, TIFF

Tasks

revert T331456
select a set of relevant file extensions. A starting point is jpg, jpeg, tiff, bmp
keep those files through where conditions in both Image-Suggestions and Section-Level-Image-Suggestions

Details

	Title	Reference	Author	Source Branch	Dest Branch
	Use suffix allowlist instead of denylist	repos/structured-data/image-suggestions!29	mlitn	T334296	main

Customize query in GitLab

Related Objects

Mentioned Here: T331456: [S] Filter out all .svg files from section-level image suggestions
T332757: Invalid file error for Add image - OFFICE is not valid mime type

Event Timeline

mfossati created this task.Apr 7 2023, 4:33 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 7 2023, 4:33 PM

CBogen edited projects, added Structured-Data-Backlog (Current Work); removed Structured-Data-Backlog.Apr 17 2023, 7:26 PM

CBogen moved this task from Incoming to Ready for Estimation on the Structured-Data-Backlog (Current Work) board.

n.b. in estimation meeting we surmised it's probably only a matter of changing [[ https://gitlab.wikimedia.org/repos/structured-data/image-suggestions/-/blob/T330773/image_suggestions/unillustratable.py#L243

this ]] line of code to use the new types and inverting the logic. But there was a note of caution that it may be necessary to double-check the file extension list against what's supported/present.

matthiasmullie claimed this task.May 5 2023, 2:20 PM

matthiasmullie moved this task from Ready for Development to Doing on the Structured-Data-Backlog (Current Work) board.

mlitn opened https://gitlab.wikimedia.org/repos/structured-data/image-suggestions/-/merge_requests/29

Use suffix allowlist instead of denylist

gerritbot added a project: Patch-For-Review.May 8 2023, 1:17 PM

matthiasmullie moved this task from Doing to Code Review on the Structured-Data-Backlog (Current Work) board.May 8 2023, 1:18 PM

mfossati merged https://gitlab.wikimedia.org/repos/structured-data/image-suggestions/-/merge_requests/29

Use suffix allowlist instead of denylist

mfossati closed this task as Resolved.May 8 2023, 5:45 PM

mfossati updated the task description. (Show Details)

Maintenance_bot removed a project: Patch-For-Review.May 8 2023, 6:11 PM