WIP: New: lazily load images
Replace WebView images with placeholders and don't download images until
- Small images, especially icons, are quickly downloaded and may appear in many places. Lazily loading these images degraded the experience with little gain so these continue to be eagerly loaded. The definition of small was taken from MobileFrontend: either defined dimension less than 50px. e.g., flags in the medal count for the "1896 Summer Olympics medal table."
- Placeholders are evaluated for loading on page load, table collapse and expand toggling, and resizing / orientation change and scrolling (each throttled to 100 ms). Eligibility is conditional on placeholders within the current screen or up to one screen ahead and affecting the layout; placeholders within collapsed tables will not be loaded. This differs from MobileFrontend which only loads up to a half screen ahead. The deviation was made to avoid considering the native component offsets on both Android and iOS apps.
- Only images are transformed. Videos with poster attributes and elements with downloaded resources in their styles among others are not considered.
- Lazy loading strives to avoid unnecessary layout changes, particularly when transitioning from placeholder to image, but they do occur when an image's width or height styles or attributes mismatch the actual dimensions of the image. e.g., see the image captioned "Obama and his wife Michelle at the Civil Rights Summit..." on the "Barack Obama" article. MobileFrontend has this same limitation. Image widening transforms must occur prior to the lazy load transform. srcset was not easily maintainable with lazy loading, and is expected to be removed soon, so this patch removes just removes it.
- The lead section, including any infoboxes or collapsed tables, is transformed which differs from the MobileFrontend implementation. Some pages such as "List of photographs of Abraham Lincoln," "List of works by Vincent van Gogh," and "Picasso's Blue Period" have the bulk of their images in the lead section and benefit greatly from lazy loading. Additionally, although the transform is designed to reduce bandwidth consumption, it may have the added benefit of reducing the memory profile and OutOfMemoryErrors and load and process time on older devices. A particularly dramatic example is the "List of works by Vincent van Gogh" article on the API 16 AOSP Nexus S emulator. Lastly, this difference makes the implementation more uniform and less surprising.
https://en.m.wikipedia.org/wiki/List_of_photographs_of_Abraham_Lincoln?oldid=779172663 https://en.m.wikipedia.org/wiki/List_of_works_by_Vincent_van_Gogh?oldid=781965885 https://en.m.wikipedia.org/wiki/Picasso's_Blue_Period?oldid=787521325
- An example of the transformation lifecycle is:
- Original: <img class=foo style='width: 100px; height: 200px' width=3 height=4 src=/ srcset=/ alt=bar>
- Pending: <span class='foo pagelib-lazy-load-placeholder pagelib-lazy-load-placeholder-pending' style='width: 100px' data-class=foo data-style='width: 100px; height: 200px' data-width=3 data-height=4 data-src=/ data-srcset=/ data-alt=bar> <span style='padding-top: 200%'></span> </span>
- Loading: <span class='foo pagelib-lazy-load-placeholder pagelib-lazy-load-placeholder-loading' style='width: 100px' data-class=foo data-style='width: 100px; height: 200px' data-width=3 data-height=4 data-src=/ data-srcset=/ data-alt=bar> <span style='padding-top: 200%'></span> </span>
<!-- Detached. --> <img class='foo pagelib-lazy-load-image-loading' style='width: 100px; height: 200px' width=3 height=4 src=/ srcset=/ alt=bar>
- Error: <span class='foo pagelib-lazy-load-placeholder pagelib-lazy-load-placeholder-error' style='width: 100px' data-class=foo data-style='width: 100px; height: 200px' data-width=3 data-height=4 data-src=/ data-srcset=/ data-alt=bar> <span style='padding-top: 200%'></span> </span>
- Loaded: <img class='foo pagelib-lazy-load-image-loaded' style='width: 100px; height: 200px' width=3 height=4 src=/ srcset=/ alt=bar>
The presentation of each state (except original) is styleable with CSS.
The original blobs of HTML are delivered a section at a time as text and then inflated into a DOM just as they were before. However, since images will begin downloading as soon as their source attribute is set, a new Document is now used to prevent images from downloading before the lazy load transform can run. When the new Document is attached to the Window, image resources are able to start downloading.
- Placeholder colors were taken from the Wikimedia User Interface Style Guide. Light mode matches the current MobileFrontend / Minerva implementation.
- Resource loading functionality may be verified using the Chrome DevTools network tab for modern devices or OkHttp logcat. All small images and large images within the first two screens forwards or backwards should load but those in the remaining sections should not.
If you're confident that OkHttp offline caching is working properly, the easiest way to test is:
- Clear your app cache.
- Visit the "List of works by Vincent van Gogh" article.
- Wait for the article to finish loading.
- Enable airplane mode.
- Expand the first collapsed table.
- Verify that no large images are shown in the first 50 or so images. Note: these images have now been attempted to load and will not be retried. They will always appear as placeholders until the page is refreshed.
- Disable airplane mode.
- Wait until a data connection is established.
- Start scrolling and verify that placeholders more than a screen away start loading images (lazy loading works).
- Enable airplane mode.
- Refresh the page.
- Verify that images previously loaded appear (offline caching works).
- This change has a surprising number of interactions. Some configurations for testing include: show / hide images, light / dark mode, expanding / collapsing infoboxes and tables, orientation changes, pressing back to show a previously visited page, page refresh, scrolling with find-in-page, MediaWiki / RESTBase page API, saved pages, cellular / Wi-Fi data connection, devices and API levels, namespaces, wikis, languages, and pages.
Some image heavy examples are: "Paris," "Pablo Picasso," "Barack Obama" (except the videos), and "Salvador Dalí" (Hebrew).
Shrink lazyLoadViewportDistanceMultiplier and increase the CSS animation times to more easily sneak up on images.
https://en.m.wikipedia.org/wiki/Paris?oldid=789426782 https://en.m.wikipedia.org/wiki/Pablo_Picasso?oldid=788122694 https://en.m.wikipedia.org/wiki/Barack_Obama?oldid=789232530 https://he.m.wikipedia.org/wiki/סלבדור_דאלי?oldid=21045644
- A demo for desktop is available in demo/LazyLoadTransform.html in the wikimedia-page-library repo. It's focused and free of some of the complexity concerns of the app.
Depends on: https://github.com/wikimedia/wikimedia-page-library/pull/33
Depends on: https://github.com/wikimedia/wikimedia-page-library/pull/56