Any client-side cropping we do in our article images / headers could benefit from this.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Spike | Tsevener | T225651 Investigate changes in iOS 13 | ||
| Resolved | Spike | HNordeenWMF | T229534 iOS13 [Feature] - Look into new image saliency API for better cropping |
Event Timeline
Comment Actions
@Tsevener Thank you Toni from 2019 for filing this 😆
Thanks to our iOS 13 deployment target, the thing we can now leverage to help our poor image cropping in Top read, Picture of the day, and article lead images is attention-based saliency: https://developer.apple.com/documentation/vision/cropping_images_using_saliency. Unlike our current approach in app, which relies on Core Image face detection, this approach would help us crop to the likely most interesting part of an image regardless of the image's subject/content (person, animal, landscape, architecture, etc.).
What remains to be seen is whether the Vision framework can be leveraged within a WidgetKit extension. When I get some free time I wanna toy around with this in the WidgetKit extension to find out, but truthfully even applying this just in the main app target itself, and just to an article's lead image itself, is a huge win in my book.
Comment Actions
suggested test steps:
- Open a biography article, for example "Angela Merkel", and check that the lead image is cropped around the face.
- Open an article with a non-portrait lead image, for example "Huhu beetle", and check that the image is top-aligned.
- Open an article with a group photo, for example "The Beatles", and check that the crop centers on the largest face.
- Go to the Explore feed and check that portrait thumbnails in "In the news" and "Top read" cards show faces, not cut-off heads.
- Scroll the Explore feed fast for several screens and check that thumbnails do not flicker or swap between cells.
- Open Saved and check that saved article thumbnails and reading list grid images crop around faces.
- Open Places, tap a pin for a person-related article, and check the callout thumbnail.
- Open a talk page and an article history diff for a biography and check the header image crop.
- Open the editor, tap "Insert media", search for "portrait", and check that result thumbnails are not face-cropped (detection is off there).
- Open the same biography article twice and check that the second load shows the crop instantly (cached result).
- Send the app to the background, return, and open a new biography article to check that detection still works.
- Add the Continue Reading widget with a biography as the last read article and check the widget thumbnail.
- Switch to dark theme and check that face-cropped images render the same.
Comment Actions
Build: 6167-beta-2026-09-16
Tested on:
iPhone 16, iOS 26.6
iPad Air, iPadOS 26.6
iPhone SE, iOS 26.5.2
| Requirement | ✅❓❌ | Notes |
| Open a biography article, for example "Angela Merkel", and check that the lead image is cropped around the face. | ✅ | Works as expected |
| Open an article with a non-portrait lead image, for example "Huhu beetle", and check that the image is top-aligned. | ✅ | Works as expected |
| Open an article with a group photo, for example "The Beatles", and check that the crop centers on the largest face. | ✅ | Works as expected |
| Go to the Explore feed and check that portrait thumbnails in "In the news" and "Top read" cards show faces, not cut-off heads. | ✅ | Works as expected |
| Scroll the Explore feed fast for several screens and check that thumbnails do not flicker or swap between cells. | ✅ | Works as expected |
| Open Saved and check that saved article thumbnails and reading list grid images crop around faces. | ✅ | Works as expected |
| Open Places, tap a pin for a person-related article, and check the callout thumbnail. | ✅ | Works as expected |
| Open a talk page and an article history diff for a biography and check the header image crop. | ✅ | Works as expected |
| Open the editor, tap "Insert media", search for "portrait", and check that result thumbnails are not face-cropped (detection is off there). | ✅ | Works as expected |
| Open the same biography article twice and check that the second load shows the crop instantly (cached result). | ✅ | Works as expected |
| Send the app to the background, return, and open a new biography article to check that detection still works. | ✅ | Works as expected |
| Add the Continue Reading widget with a biography as the last read article and check the widget thumbnail. | ✅ | Works as expected |
| Switch to dark theme and check that face-cropped images render the same. | ✅ | Works as expected |