The use of SSIM tests for Thumbor have served us well for a long time, but now that Thumbor has a more stable cadence of upgrades and updates to underlying libraries and Debian versions, the tests are becoming a little cumbersome. Security updates and particularly feature updates are invalidating our tests and forcing the similarity ratios to be gradually changed on a case by case basis. Doing this for fairly significant upgrades in the past made a certain amount of sense, but as we see minor security upgrades breaking tests, we're reaching a point of diminishing returns with our test set.

At the very least, we need to regenerate our sample converted thumbnails to get a better baseline for the tests, but in the longer term it might be worth reconsidering whether testing the exactness of our libraries makes sense when using Docker and contending with regular updates.