For years, visual regression has been a half-solved problem: you capture an image of the component, store it as a baseline, and compare pixel by pixel on every commit. If anything changes, the test fails and someone reviews it.
That model rests on an assumption that no longer holds: that code changes are few, deliberate, and human-authored. When an agent refactors thirty components in an afternoon, the pixel diff stops being a safety net and becomes noise.
The usual complaint against visual snapshots is flakiness: font rendering, antialiasing, a
blinking caret. That can be mitigated — tolerance thresholds, waiting for
document.fonts.ready, disabling animations:
// cypress/support/visual.ts
export function stabilize() {
cy.document().then((doc) => {
const style = doc.createElement('style');
style.innerHTML = `
*, *::before, *::after {
animation: none !important;
transition: none !important;
caret-color: transparent !important;
}
`;
doc.head.appendChild(style);
});
cy.document().its('fonts.status').should('equal', 'loaded');
}That fixes the flakiness. It does not fix the underlying problem, which is a different one: a pixel diff tells you that something changed, but not whether the change matters. And once the volume of changes multiplies, that distinction is the only thing that counts.
An agent that replaces a spacing utility with design system tokens is going to move things two pixels in forty snapshots. All of them fail. All of them are correct. The cost of reviewing them one by one exceeds the cost of not having the tests at all.
What has worked for me is to stop treating "the visual side" as one thing and split it into three layers with different tolerances.
Before looking at pixels, verify the invariants that can be expressed as assertions. They are not snapshots: they are properties.
it('keeps the accessibility contract of the stepper', () => {
cy.mount(<ProgressTracker steps={steps} />);
// There must be exactly one current step, and it must be announced.
cy.findAllByRole('button', { current: 'step' }).should('have.length', 1);
// Every node has an accessible name.
cy.findAllByRole('button').each(($el) => {
cy.wrap($el).should('have.attr', 'title').and('not.be.empty');
});
});A styling refactor does not break this layer, and an agent that removes an aria-current
"because it was unused" does. It is the layer that catches the most real regressions per
unit of maintenance.
Instead of comparing the whole image, you assert the computed values that are design decisions:
it('the current node uses the accent color, not just any blue', () => {
cy.mount(<ProgressTracker steps={steps} />);
cy.findByRole('button', { current: 'step' })
.should('have.css', 'background-color')
.then((color) => {
cy.window()
.then((win) =>
win.getComputedStyle(win.document.documentElement)
.getPropertyValue('--color-accent')
.trim(),
)
.then((token) => expect(normalizeColor(color)).to.equal(normalizeColor(token)));
});
});This catches the class of bug an agent introduces most often: hardcoding #2563eb instead
of using var(--color-accent). The result is visually identical today and broken the day
you change the theme.
The full capture is reserved for a small, explicit set of views whose appearance is itself the contract: the pricing page, the transactional email, the component a client signed off on in writing. Ten or fifteen, not four hundred.
That cap is about contract surfaces. It is not a budget for total coverage, and the distinction matters more than the numbers do. The wider sweeps are a different job with different economics: they exist to make change visible to a person, not to gate a merge on a threshold. In practice I run two of them — a component tier of 264 Storybook stories in Chromatic, and a route tier of around eighty page captures reaching whole app surfaces no component story renders. Neither contradicts the fifteen above, because neither is asserting that the pixels are correct: both post a report, and neither blocks a merge. What decides which layer a set belongs to is not how many snapshots it holds but whether a failure stops the pipeline. The mistake is collapsing the two: pointing a zero-tolerance gate at a wide sweep, or letting the contract set drift because it is buried in the sweep's noise. I have written about what the wide sweep caught that nothing else did — a print view receiving dark-scheme inks across 280 nodes — and it is exactly the kind of surface that is worth capturing and not worth gating.
Here is the uncomfortable part. In the classic model, the baseline is the truth and the change is suspect. With an agent in the pipeline, that inverts: the change is usually intentional and the baseline is what is out of date.
The practical consequence is that approving a baseline has to cost the same as reviewing a code diff, or the team will start approving in bulk without looking — which is exactly the failure the tests were meant to prevent.
What helps:
That last point is the one that has surprised me most: the agent's statement of intent, written before running the tests, works as an oracle. You do not need a human to judge every pixel; you need to compare two claims.
I ended up building a version of this into a real pipeline:
packages/visual-diff,
in this repo, runs on every PR and posts its report as a sticky comment — but it is
deliberately never a required check, on exactly the reasoning above: an approval nobody
has to read is not a safety net. The report groups by component tier and worst-diff-first
today, not by cause yet — that part of the idea above is still just the idea.
There is still a gap: changes that are individually correct and incoherent as a whole. Every component passes its tests and the full screen looks wrong. None of the three layers catches it, because the problem does not live in any one component.
For now I cover it with a handful of full-page snapshots on the critical flows, plus human review. It is not satisfying. If you have found something better, write to me.