Visual testing

An expectScreenshot step compares the page, or one element, with an accepted baseline image.

Baselines

1. The first run of a visual step saves its screenshot as the baseline and passes with a note.
2. Later runs compare pixel by pixel. If more than maxDiffRatio of the pixels differ, the step fails and the run shows the baseline, this run and a difference image.
3. When a change is intended, open the failed run and choose Accept as the new baseline. The old baseline stays in the history. Baselines are never replaced automatically.

Baselines are kept per step, viewport size, full page or element capture, and colour scheme, so desktop and mobile runs of one step never compare against each other.

Reducing noise

• mask paints over elements that change every run, such as dates, avatars and ads.
• threshold (0 to 1, default 0.1) is the colour distance treated as equal; anti-aliasing is ignored.
• maxDiffRatio (default 0.001, 0.1%) is the share of pixels allowed to change.
• Animations are disabled and the text caret hidden before capture, and web fonts are awaited.
• Run visual tests against stable data. Personalised or time dependent content will differ.

When the browser's major version changes between the baseline and a run, the result says so: rendering can change between Chromium releases, so review and accept a new baseline if the page looks right.