Page MenuHomePhabricator

Mass rendering tests / visual diff
Closed, ResolvedPublic

Description

Once we have proper CSS hooked up as discussed in bug 51245 we should start automated visual comparisons between PHP parser HTML + CSS and Parsoid HTML + CSS. The easiest way to scale & surface this is likely testreduce.

Some promising tools mentioned in a recent wikitech thread titled 'CSS regressions':


Version: unspecified
Severity: normal
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=62633

Details

Reference
bz62511

Related Objects

View Standalone Graph
This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.

Event Timeline

bzimport raised the priority of this task from to Normal.
bzimport set Reference to bz62511.

(In reply to Gabriel Wicke from comment #0)

Once we have proper CSS hooked up as discussed in bug 51245 we should start
automated visual comparisons between PHP parser HTML + CSS and Parsoid HTML
+ CSS. The easiest way to scale & surface this is likely testreduce.

Some promising tools mentioned in a recent wikitech thread titled 'CSS
regressions':

JS code driven by PhantomCSS (JS) and Resemble.js

C++ code .. probably standalone.

Ruby code driven by phantomjs + ImageMagick

nodejs library and has a dependency on a C library (cairo). Not sure if it will work with https://www.npmjs.org/package/image-diff which is ImageMagick binding.

It looks like huddle's solution will fit neatly within our nodejs setup. But, worth experimenting with a couple solns to see what works best in terms of: performance, quality, ease of use.

(In reply to ssastry from comment #1)

(In reply to Gabriel Wicke from comment #0)
> Once we have proper CSS hooked up as discussed in bug 51245 we should start
> automated visual comparisons between PHP parser HTML + CSS and Parsoid HTML
> + CSS. The easiest way to scale & surface this is likely testreduce.
>
> Some promising tools mentioned in a recent wikitech thread titled 'CSS
> regressions':
>
> * https://github.com/Huddle/PhantomCSS

JS code driven by PhantomCSS (JS) and Resemble.js

Downloaded phantomcss and started looking at the code. PhantomCSS seems like a wrapper to work with casper to acquire screenshots, etc. capserjs can be run in nodejs using spookyjs. I'll try experimenting with this tomorrow.

But, I'm now wondering whether the generated images might be too big and/or if we'll get too many trivial rendering diffs, and whether running a normalizing html diff on PHP and Parsoid HTML should be considered as well.

Now deployed and mostly working ... still has some issues, but it is more in the realm of tweaking and bug fixing.