My Experience with Snapshot Testing

Snapshot Testing — what is that?? I get questioned a lot about what it is and why we use it. It’s not as common as other types of testing such as Manual, Automated e2e, Regression, Integration, etc. In this article, i’ll discuss what snapshot testing is, when we have found it useful, when not so useful, some lessons learned, and existing tools.

What is Snapshot Testing?

Snapshot testing compares two images: a “Reference” image, which is the expected image of the application screen, to the “Current” image, which is the snapshot of what the screen is actually displaying. A “Difference” image is produced which displays where the two images are different.

To Use or Not To Use Snapshot Testing

Snapshot Testing is very helpful in determining whether there are unexpected UI changes.

  • To make sure the UI is correctly displayed still upon scrolling.
  • Truncation. In the iOS hierarchy tree, the full text is always returned regardless of whether or not it is truncated on the UI.
  • Colors. Obviously the snapshot tests will do color verification as shown in the example above. Some could argue why not do this color verification in the e2e tests instead. Some tools like Android Espresso allows you to do this; however you cannot using iOS XCUITest.

Lessons Learned

  • Snapshot Testing is not for everyone. However, if you want to ensure that you application maintains the expected design, use it. Otherwise, the quality of the design may degrade over time. Personally, when I notice an application that has a lot of design issues, I have less confidence in the functionality of the application. If they don’t take the time to care about the design, they also may not care much about the quality of the functionality.
  • If you have a small, simple application where UI changes are very isolated to only affecting the intended target then snapshot testing wouldn’t be worth it.
  • The reference images can take up a lot of space so ensure you are able to handle these and investigate if there are ways to generate smaller images that take up less space with the tool being used.
  • Don’t over snapshot test. We initially started out with snapshot testing all of our supported iOS devices. We realized that when an issue occurred, it was typically either on all devices, or a device that was at one end on a screen size scale. We decided to cut back and only snapshot test the smallest and largest screen size devices. This allowed us to have a lot of confidence that almost all UI issues would still be found while greatly reducing our snapshot testing run time.
  • Don’t snapshot frequently changing screens, data, or animations.
  • When possible, only snapshot smaller elements or sub-sections of the screen. Do not include sections that are shared across several pages. For example, if you have an iOS navigation bar at the top or tab bar at the bottom, you would not want this to be included in every single one of your snapshots. Isolate the navigation bar and tab bar to their own snapshots, and only take a snapshot of the view inside or even just certain elements on that screen. This will alleviate you from needing to redo ALL snapshots when a change is made just to one of those areas.

Snapshot Testing Tools

iOSSnapshotTestCase

  • This was originally developed by FaceBook and known as FBSnapshotTestcase but is now maintained by Uber.
  • This is what we are currently using for our iOS Snapshot Testing. It has a lot of contributors so issues or updates are frequently addressed.
  • We used this for a little while for Android Snapshot Testing however we ran into issues often where it would think the screens were different but they weren’t and the difference images would only show a gray screen. It doesn’t support a tolerance value either like iOSSnapshotTestCase does. We ended up spending too much time investigating failures then ended up not really being failures. We stopped Snapshot Testing for Android at this time until this tool is more mature or we come across another tool.

Test Engineer Expert, Dog Mom

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store