As I’ve written about before, I’m the author/maintainer of a Haskell library for programmable CAD, called Waterfall-CAD.

Ever since I released this in 2023, it’s bothered me that I don’t really have tests for it.

Testing a CAD library like Waterfall-CAD is difficult, because the outputs of a Waterfall-CAD program are generally 3D models, and are difficult things to write good test assertions about.

In 2025, I added SVG support to Waterfall-CAD, converting the images in the README.md from screenshots of a mesh viewer, to vector diagrams generated directly within Haskell code.

While testing solid models seems inherently tricky, there’s an established field of “Visual Regression Testing” tools, so having SVG output seemed vastly more testable than 3d model output¹.

Visual Regression Testing

“Visual Regression Testing” is mostly used when testing UI code.

“Visual Regression Tools” work by storing a snapshot of an image generated by an application ², generating a visual diff of the current behaviour compared to that snapshot, and failing the test if the current behaviour looks significantly different. They also usually provide a mechanism to “accept” the current behaviour; overwriting the snapshots with new outputs, as well as a mechanism to visualize the difference between the expected and actual behaviour, highlighting the parts of the snapshot that have changed.

A Note on Terminology

“Golden Testing” ³ is the name of a testing technique where the expected output of a program is stored in a file, and the program is tested by comparing the current output to that file. “Golden Testing” tools also generally provide a mechanism to “accept” the current program behaviour, and “overwrite” the files with the current behaviour.

It seems useful to me to treat “Visual Regression Testing” as a special case of “Golden Testing” where the test files are images, and the “diff” is a visual diff, rather than a precise comparison of the binary structure of the files.

Going into this project, I thought that the term for the kind of test that I wanted to write was “Snapshot Testing”.

I’m now pretty sure that’s incorrect, “Snapshot Testing” seems to be used to describe exactly the same technique as “Golden Testing” (although the different terms are common in different programming language communities).

I’m currently using the term “Visual Regression Testing” for what I’m doing; I think this is synonymous with the (also commonly used) term “Visual Snapshot Testing”.

I still think it’s useful to think of this as a special case of “Golden Testing” or “Snapshot Testing”. Indeed, I’m using the Haskell Golden Testing library tasty-golden⁴ to implement these tests.

What I’m Doing Specifically

I’ve added a series of tests based on the diagrams that had already been generated for the README.

The images are stored as SVG, with some custom css so they render differently according to whether the README’s being viewed in dark mode or not.

I like having the images be dual purpose, because it’s one less thing to keep in sync.

The Rasterific-SVG library is used to convert both the SVG in the repo, and the one generated by the code under test, into a JuicyPixels image.

Once we’ve got two images, the code asserts that they both have the same size. If they do, it compares each pixel in the images, and counts the number of pixels that differ by a Manhattan distance of more than a certain amount. If this count of mismatched pixels is higher than some tolerance value, the test fails.

On test failure, a “diff image” is written, which highlights the pixels that differed.

You can see the code doing all this in the DiagramGoldenTests.hs file in the GitHub repo.

I made this image by deliberately failing the tests by changing the colour used for the hidden lines.

Is There an Open-Source Library In This?

I think there might be.

But at the moment, I’m a small distance away from having written it.

A proper library would have to support more of JuicyPixels’ Pixel formats.

It would need to make the diff visualization more customizable (you don’t want to show differences in red if the image already contains a whole lot of red).

It would also need to support loading images directly from disk, instead of the “via-svg” juggling that I’m doing.

At the moment, I’m leaning towards not doing any of that, but I’m torn, and if one person says “yeah, I’d use that”, there’s a good chance I can be persuaded.