Right now the threshold to decide how big a difference marks it as potential regression/improvement, is decided in the software. That could remain the same as default (with very low threshold), but would be nice to allow overriding in the YAML file.

That way, it can be configured by MediaWiki based on how stable and how fast its own CI hardware is. Rather than trying to make that prediction inside Fresnel. This also means it can then be tuned much more easily, without the overhead of needing to do a Fresnel release (npm publish, Dockerfile update, Jenkins job change, etc.).