Page MenuHomePhabricator

Mass rendering tests / visual diff
Closed, ResolvedPublic

Description

Once we have proper CSS hooked up as discussed in bug 51245 we should start automated visual comparisons between PHP parser HTML + CSS and Parsoid HTML + CSS. The easiest way to scale & surface this is likely testreduce.

Some promising tools mentioned in a recent wikitech thread titled 'CSS regressions':


Version: unspecified
Severity: normal
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=62633

Details

Reference
bz62511

Related Objects

StatusSubtypeAssignedTask
OpenNone
ResolvedJdlrobson
Resolvedssastry
ResolvedArlolra
ResolvedArlolra
ResolvedEsanders
ResolvedCatrope
DeclinedNone
DeclinedNone
Resolvedssastry
Resolved bearND
Resolved bearND
Resolved bearND
Resolved bearND
OpenNone
ResolvedJhernandez
OpenNone
Resolved marcoil
ResolvedCatrope
Resolved marcoil
ResolvedArlolra
ResolvedArlolra
Resolved GWicke
Resolved GWicke
Resolved GWicke
Resolvedfgiunchedi
Resolvedfgiunchedi
ResolvedCmjohnson
ResolvedCmjohnson
ResolvedJoe
Resolvedfgiunchedi
Resolved GWicke
Resolved Jdouglas
Resolved GWicke
Resolved GWicke
ResolvedArlolra
Resolved GWicke
Resolvedmobrovac
Resolvedmobrovac
Resolvedmobrovac
Resolvedmobrovac
Duplicate Jdouglas
ResolvedAndrew
Resolved GWicke
Resolvedfgiunchedi
Resolvedfgiunchedi
Resolvedfgiunchedi
ResolvedEevans
Resolvedfgiunchedi
Resolved GWicke
Resolved GWicke
Resolvedfgiunchedi
Resolvedmobrovac
Resolved GWicke
Resolved GWicke
Resolved AlexMonk-WMF
Resolvedsanthosh
Resolvedssastry
ResolvedMholloway
ResolvedJackmcbarn
ResolvedRenxiaoyi
Resolvedcscott
ResolvedKelson
OpenNone
OpenNone
OpenNone
ResolvedArlolra
ResolvedArlolra
OpenNone
DeclinedNone
StalledRenxiaoyi
OpenNone
DeclinedNone
DeclinedNone
DeclinedNone
OpenNone
OpenNone
InvalidNone
InvalidNone
DuplicateNone
DuplicateNone
ResolvedJhernandez
ResolvedJdlrobson
DuplicatePeter
Resolvedbmansurov
DeclinedNone
DuplicateNone
Resolvednray
Resolvedphuedx
ResolvedAnomie
ResolvedAnomie
ResolvedAnomie
ResolvedEBernhardson
ResolvedAnomie
ResolvedAnomie
OpenNone
DuplicateNone
ResolvedNone
Resolvedphuedx
DeclinedNone
ResolvedPchelolo
ResolvedArlolra
ResolvedMooeypoo
ResolvedCatrope
Resolved GWicke
OpenNone
Resolved marcoil
Resolved marcoil
Resolved GWicke

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 2:59 AM
bzimport added a project: Parsoid-Tests.
bzimport set Reference to bz62511.

(In reply to Gabriel Wicke from comment #0)

Once we have proper CSS hooked up as discussed in bug 51245 we should start
automated visual comparisons between PHP parser HTML + CSS and Parsoid HTML
+ CSS. The easiest way to scale & surface this is likely testreduce.

Some promising tools mentioned in a recent wikitech thread titled 'CSS
regressions':

JS code driven by PhantomCSS (JS) and Resemble.js

C++ code .. probably standalone.

Ruby code driven by phantomjs + ImageMagick

nodejs library and has a dependency on a C library (cairo). Not sure if it will work with https://www.npmjs.org/package/image-diff which is ImageMagick binding.

It looks like huddle's solution will fit neatly within our nodejs setup. But, worth experimenting with a couple solns to see what works best in terms of: performance, quality, ease of use.

(In reply to ssastry from comment #1)

(In reply to Gabriel Wicke from comment #0)

Once we have proper CSS hooked up as discussed in bug 51245 we should start
automated visual comparisons between PHP parser HTML + CSS and Parsoid HTML
+ CSS. The easiest way to scale & surface this is likely testreduce.

Some promising tools mentioned in a recent wikitech thread titled 'CSS
regressions':

JS code driven by PhantomCSS (JS) and Resemble.js

Downloaded phantomcss and started looking at the code. PhantomCSS seems like a wrapper to work with casper to acquire screenshots, etc. capserjs can be run in nodejs using spookyjs. I'll try experimenting with this tomorrow.

But, I'm now wondering whether the generated images might be too big and/or if we'll get too many trivial rendering diffs, and whether running a normalizing html diff on PHP and Parsoid HTML should be considered as well.

Now deployed and mostly working ... still has some issues, but it is more in the realm of tweaking and bug fixing.