My Experience with Snapshot Testing

What is Snapshot Testing?

To Use or Not To Use Snapshot Testing

  • Elements that do not show up in the hierarchy tree. In Xcode, you can inspect the current tree structure by running po XCUIApplication(). However, we have ran into issues where a few elements were not appearing like nav bar text, or error message text. In these instances we had no way to access the element via our XCUITests however, a snapshot test allowed us to verify these elements.
  • To make sure the UI is correctly displayed still upon scrolling.
  • Truncation. In the iOS hierarchy tree, the full text is always returned regardless of whether or not it is truncated on the UI.
  • Colors. Obviously the snapshot tests will do color verification as shown in the example above. Some could argue why not do this color verification in the e2e tests instead. Some tools like Android Espresso allows you to do this; however you cannot using iOS XCUITest.

Lessons Learned

  • Snapshot Testing is not for everyone. However, if you want to ensure that you application maintains the expected design, use it. Otherwise, the quality of the design may degrade over time. Personally, when I notice an application that has a lot of design issues, I have less confidence in the functionality of the application. If they don’t take the time to care about the design, they also may not care much about the quality of the functionality.
  • If you have a small, simple application where UI changes are very isolated to only affecting the intended target then snapshot testing wouldn’t be worth it.
  • The reference images can take up a lot of space so ensure you are able to handle these and investigate if there are ways to generate smaller images that take up less space with the tool being used.
  • Don’t over snapshot test. We initially started out with snapshot testing all of our supported iOS devices. We realized that when an issue occurred, it was typically either on all devices, or a device that was at one end on a screen size scale. We decided to cut back and only snapshot test the smallest and largest screen size devices. This allowed us to have a lot of confidence that almost all UI issues would still be found while greatly reducing our snapshot testing run time.
  • Don’t snapshot frequently changing screens, data, or animations.
  • When possible, only snapshot smaller elements or sub-sections of the screen. Do not include sections that are shared across several pages. For example, if you have an iOS navigation bar at the top or tab bar at the bottom, you would not want this to be included in every single one of your snapshots. Isolate the navigation bar and tab bar to their own snapshots, and only take a snapshot of the view inside or even just certain elements on that screen. This will alleviate you from needing to redo ALL snapshots when a change is made just to one of those areas.

Snapshot Testing Tools

  • https://github.com/uber/ios-snapshot-test-case
  • This was originally developed by FaceBook and known as FBSnapshotTestcase but is now maintained by Uber.
  • This is what we are currently using for our iOS Snapshot Testing. It has a lot of contributors so issues or updates are frequently addressed.
  • https://github.com/Karumi/Shot
  • We used this for a little while for Android Snapshot Testing however we ran into issues often where it would think the screens were different but they weren’t and the difference images would only show a gray screen. It doesn’t support a tolerance value either like iOSSnapshotTestCase does. We ended up spending too much time investigating failures then ended up not really being failures. We stopped Snapshot Testing for Android at this time until this tool is more mature or we come across another tool.

--

--

--

Test Engineer Expert, Dog Mom

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Better iOS through Factory Method Pattern — A basic chat view

Swift New Concurrency Framework (Part 3)

MVC that is not massive

MongoDB Realm on XCode Playground: Installation

Keyboard Handling in iOS — Swift 5

Creating in-app messages with HarmonyOS

How to cleanup space on MacOS

Concurrency in Swift (Operations and Operation Queue Part 3)

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Ronell Lukasik

Ronell Lukasik

Test Engineer Expert, Dog Mom

More from Medium

Neuro-psychological testing

Memory Fragments

Architecture Design Competition ‘Shifting Horizons 2.0’

Chain of Responsibility Design Pattern