Page MenuHomePhabricator

Implement Smart Image Cropping for 9:16-Friendly Detail View
Closed, ResolvedPublic5 Estimated Story Points

Assigned To
Authored By
KSarabia-WMF
Aug 18 2025, 5:58 PM
Referenced Files
F66008344: screenshot 144.png
Sep 11 2025, 1:44 PM
F66008343: screenshot 145.png
Sep 11 2025, 1:44 PM
F66008341: screenshot 142.jpg
Sep 11 2025, 1:44 PM
F66008337: screenshot 143.png
Sep 11 2025, 1:44 PM
F66008336: screenshot 141.png
Sep 11 2025, 1:44 PM
F66008328: screenshot 137.png
Sep 11 2025, 1:44 PM
F66008327: screenshot 138.png
Sep 11 2025, 1:44 PM
F66008326: screenshot 140.png
Sep 11 2025, 1:44 PM

Description

Description

We need to incorporate an automated image cropping mechanism into the image detail view of the image browing design, where image foci are centered and displayed in 9:16 (TikTok-friendly) dimensions. This will ensure images in the detail view are cropped intelligently rather than relying on simple center-cropping.

Scope / Details:

  • Integrate an SmartCropJS image cropping library to handle smart cropping (e.g., face/subject-aware).
  • Ensure carousel images render in TikTok-friendly aspect ratios.
  • Initial ticket should focus on basic implementation.
  • If time permits, create a follow-up ticket for refinement and QA polish.

Acceptance Criteria:

  • Images in detail view display in 9:16 TikTok-style dimensions.
  • Cropping is automated
  • T400703 has been updated to include a list of all 3rd-party libraries introduced to support this feature
  • Per last discussion with @egardner MVP now is just generating coordinates for object fit center
  • Per last discussion with team MVP now is applying the coordinates to the detail view image.

Follow-Up

  • Refine cropping accuracy based on QA feedback.
  • Explore alternative/improved libraries or approaches if needed.

Requirement

Scope: Mobile web detail view only (PatchDemo: https://patchdemo.wmcloud.org/wikis/8ab6c84ec2/wiki/Paris).

  • QA must focus on the detail (full-screen) image view, not the carousel.
  • Carousel images at the top of the page are no longer cropped to 9:16 (only very wide panoramas are cropped there).
  • In the detail view, images must be smart-cropped using SmartCropJS so that the main subject is centered when displayed in mobile-size viewports.
  • For larger screens, images should still be centered appropriately if cropped by viewport dimensions.
  • Strict 9:16 enforcement applies to the detail view on mobile; carousel remains responsive in its original aspect ratio.

BDD

Feature: Smart cropping in mobile detail view

  Scenario: Detail view on mobile
    Given I open an image detail view on a mobile-size viewport
    When SmartCropJS applies crop coordinates
    Then the image is displayed in 9:16 aspect ratio
    And the subject is centered in the viewport

  Scenario: Detail view on desktop
    Given I open an image detail view on a desktop-size viewport
    When the viewport dimensions cause cropping
    Then the subject remains centered on a reasonable focal point

  Scenario: Carousel images
    Given I view the carousel at the top of the page
    When images are rendered
    Then images display in their original aspect ratios
    And no strict 9:16 crop is applied (except very wide panoramas)

Test Steps

Test Case 1: Detail view on mobile

  1. Open https://patchdemo.wmcloud.org/wikis/408016161e/wiki/Paris in a mobile viewport.
  2. Tap an image to open the detail view.
  3. AC1: The image is displayed in 9:16 aspect ratio.
  4. AC2: SmartCropJS ensures the subject is centered (inspect applied crop coordinates).

Test Case 2: Detail view on desktop

  1. Open the same PatchDemo page on a desktop viewport.
  2. Open an image in the detail view.
  3. AC3: The image fills the viewport according to its aspect ratio.
  4. AC4: Cropping centers the subject if the viewport forces part of the image to be hidden.

Test Case 3: Carousel images

  1. Inspect the carousel at the top of the Paris page in both desktop and mobile.
  2. AC5: Images render in their original aspect ratios.
  3. AC6: No strict 9:16 crop is applied, except for panoramic images which may be cropped to fit.

QA Results - Patchdemo

Event Timeline

For the security review portion of this, we have a general task for that over at T400703. I don't think we need an additional review just for this one library. Maybe the acceptance criteria to this task can be revised from "security review is completed and signed off" to "T400703 has been updated to include a list of all 3rd-party libraries introduced to support this feature".

egardner renamed this task from Implement Smart Image Cropping for 9:16-Friendly Carousel to Implement Smart Image Cropping for 9:16-Friendly Detail View.Aug 19 2025, 3:22 PM

@JScherer-WMF my assumption so far has been that the detail view should just fill the viewport with the cropped image, regardless of the exact size/shape of that viewport. This means we'd need to support the Tiktok-style vertical aspect ratio, but that's not the only aspect ratio we'd see; the image would just always fill the entire viewport (and would be cropped in a way to center on whatever most salient features we can detect using smartcrop.js).

Is this correct, or do you want to enforce the 9:16 vertical format for all viewports (or all viewports below a certain breakpoint, etc).

Basically – do you want a responsive design or something that is more like the web UI of Instagram Stories (where stories are in a vertical format even on a large, wide desktop screen).

HSwan-WMF set the point value for this task to 5.Aug 19 2025, 5:15 PM
HSwan-WMF moved this task from Needs Refinement to Sprint 4 on the Reader Growth Team board.

Change #1180655 had a related patch set uploaded (by Kimberly Sarabia; author: Kimberly Sarabia):

[mediawiki/extensions/ReaderExperiments@master] [WIP] Add smartcrop library

https://gerrit.wikimedia.org/r/1180655

egardner updated the task description. (Show Details)

Change #1180655 merged by jenkins-bot:

[mediawiki/extensions/ReaderExperiments@master] Add smartcrop library

https://gerrit.wikimedia.org/r/1180655

Change #1182690 had a related patch set uploaded (by Kimberly Sarabia; author: Kimberly Sarabia):

[mediawiki/extensions/ReaderExperiments@master] Smart crop

https://gerrit.wikimedia.org/r/1182690

Change #1182690 merged by jenkins-bot:

[mediawiki/extensions/ReaderExperiments@master] make smartcrop work local and log object-fit coord

https://gerrit.wikimedia.org/r/1182690

Change #1184958 had a related patch set uploaded (by Kimberly Sarabia; author: Kimberly Sarabia):

[mediawiki/extensions/ReaderExperiments@master] Add SmartCrop source size to composable return

https://gerrit.wikimedia.org/r/1184958

Test wiki created on Patch demo by EGardner (WMF) using patch(es) linked to this task:
https://patchdemo.wmcloud.org/wikis/b7d2642e04/w/

Test wiki created on Patch demo by EGardner (WMF) using patch(es) linked to this task:
https://patchdemo.wmcloud.org/wikis/956643203e/w/

KSarabia-WMF updated the task description. (Show Details)
KSarabia-WMF moved this task from Code Review to QA on the Reader Growth Team (Sprint 5) board.
KSarabia-WMF added a subscriber: bvibber.

Test wiki created on Patch demo by ETadros (WMF) using patch(es) linked to this task:
https://patchdemo.wmcloud.org/wikis/8ab6c84ec2/w/

@KSarabia-WMF , I'm not clear on the cropping rules. Are the Carousel images at the top of the page supposed to be strictly displayed in 9:16? Does this carry over to the detailed view or are the detailed views in their original aspect ratios (which are apparently at the whim of the viewport dimensions? I'm looking at https://patchdemo.wmcloud.org/wikis/8ab6c84ec2/wiki/Paris, so unless I set up patch demo wrong, I'm not seeing a strict 9:16 AR.

@Edtadros You just want to QA the detail view in mobile viewport not the actual carousel.

@KSarabia-WMF , I'm not clear on the cropping rules. Are the Carousel images at the top of the page supposed to be strictly displayed in 9:16? Does this carry over to the detailed view or are the detailed views in their original aspect ratios (which are apparently at the whim of the viewport dimensions? I'm looking at https://patchdemo.wmcloud.org/wikis/8ab6c84ec2/wiki/Paris, so unless I set up patch demo wrong, I'm not seeing a strict 9:16 AR.

This task may have been written before we fully worked out what the desired layout was for larger screens (see T402964 for this). For QA purposes I think we're hoping to confirm that detail (full-screen) images are centered on a reasonable part when they are inevitably cropped on mobile-size viewports but also desktop ones.

Carousel images are not being cropped any longer (there was talk about doing that earlier on in the project but we only crop extremely wide panoramic images in the carousel now).

Change #1186624 had a related patch set uploaded (by Matthias Mullie; author: Matthias Mullie):

[mediawiki/extensions/ReaderExperiments@master] Optimize crop

https://gerrit.wikimedia.org/r/1186624

Change #1184958 merged by jenkins-bot:

[mediawiki/extensions/ReaderExperiments@master] Add SmartCrop source size to composable return

https://gerrit.wikimedia.org/r/1184958

Test wiki created on Patch demo by ETadros (WMF) using patch(es) linked to this task:
https://patchdemo.wmcloud.org/wikis/408016161e/w/

Edtadros moved this task from QA to Signoff on the Reader Growth Team (Sprint 5) board.
Edtadros subscribed.

@egardner / @KSarabia-WMF Thanks for your clarifications. I updated the Req, BDD, and Test steps. Everything looks good, and really slick.

Test Result - Patchdemo

Status: ✅ PASS
Environment: PatchDemo 408016161e
OS: macOS Sequoia 15.5
Browser: Chrome Canary (latest as of test date)
Device: MS
Emulated Device: various

Test Case 1: Detail view on mobile

  1. Open https://patchdemo.wmcloud.org/wikis/408016161e/wiki/Paris in a mobile viewport.
  2. Tap an image to open the detail view.
  3. AC1: The image is displayed in detail (full-screen).
  4. AC2: SmartCropJS ensures the subject is centered (inspect applied crop coordinates).

As best as it can it does keep the Eiffel tower in the image. Impressive honestly.

screenshot 139.png (1×834 px, 693 KB)
screenshot 140.png (535×991 px, 492 KB)
screenshot 138.png (1×722 px, 723 KB)
screenshot 137.png (1×874 px, 1 MB)

Test Case 2: Detail view on desktop

  1. Open the same PatchDemo page on a desktop viewport.
  2. Open an image in the detail view.
  3. AC3: The image fills the viewport according to its aspect ratio.
  4. AC4: Cropping centers the subject if the viewport forces part of the image to be hidden.

Pretty excellent cropping.

screenshot 141.png (1×1 px, 3 MB)
screenshot 143.png (1×1 px, 3 MB)
uncropped for reference
screenshot 142.jpg (2×4 px, 2 MB)

Test Case 3: Carousel images

  1. Inspect the carousel at the top of the Paris page in both desktop and mobile.
  2. AC5: Images render in their original aspect ratios.
  3. AC6: No strict 9:16 crop is applied, except for panoramic images which may be cropped to fit.
screenshot 145.png (513×1 px, 411 KB)
screenshot 144.png (569×1 px, 408 KB)

@egardner Can you make the call on next steps here? QA passed but back in doing?

Looks good, resolving!

Test wiki on Patch demo by ETadros (WMF) using patch(es) linked to this task was deleted:

https://patchdemo.wmcloud.org/wikis/408016161e/w/

Test wiki on Patch demo by ETadros (WMF) using patch(es) linked to this task was deleted:

https://patchdemo.wmcloud.org/wikis/8ab6c84ec2/w/

Test wiki on Patch demo by EGardner (WMF) using patch(es) linked to this task was deleted:

https://patchdemo.wmcloud.org//wikis/956643203e/w/

Test wiki on Patch demo by EGardner (WMF) using patch(es) linked to this task was deleted:

https://patchdemo.wmcloud.org//wikis/b7d2642e04/w/

Test wiki on Patch demo by EGardner (WMF) using patch(es) linked to this task was deleted:

https://patchdemo.wmcloud.org//wikis/956643203e/w/

Test wiki on Patch demo by EGardner (WMF) using patch(es) linked to this task was deleted:

https://patchdemo.wmcloud.org//wikis/b7d2642e04/w/