Helpful Fastlane Commands for iOS Automation Testing

What is it and how can it help you in QA? 🤔

Fastlane

• Used to automate building and releasing of iOS applications
• Specify version of Xcode to use
• Clear Derived Data
• Reset and launch iOS simulators
• Execute iOS test cases
• Output test results to Slack

Getting Started

Run the following but for additional installation instructions you can go here or for iOS specific instructions here.

• xcode-select — install
• brew install fastlane
• fastlane init

Running a Lane

lane :my_lane_name do
 ....
end

In order to run a lane all you need to do is run the following in a terminal where fastlane was installed in your project.

fastlane my_lane_name

Setting the Xcode Version

To set the version of Xcode that fastlane should use for any build steps, use the xcode_select action.

lane :my_lane_name do
 xcode_select("/Applications/Xcode_15.3.app")
end

When you run it, you will see output like the following:

Clear Derived Data

A lot of times you will want to clear the derived data so that you have a clean state prior to running any tests. Use the clear_derived_data action for this.

lane :my_lane_name do
 xcode_select("/Applications/Xcode_15.3.app")
 clear_derived_data
end

When you run it, you will see output like the following:

Resetting iOS Simulators

To reset all simultors and shutdown any booted simulators you can use FastlaneCore::Simulator.reset_all

lane :my_lane_name do
 FastlaneCore::Simulator.reset_all
end

When you run it, you will see output like the following:

💡If you need more information about the simulators you can use xcrun simctl commands. To get the list of simulators and their names or UUIDs, run xcrun simctl list. 💡

You can use reset instead of reset_all if you want to only reset only particular device. You can use the UDID of the device or do it by name.

lane :my_lane_name do
 FastlaneCore::Simulator.reset(udid: 'DC97769E-A28B-4046-82FF-8DE20C749629')
end

If you do Parallel Testing, this will not reset those cloned simulators. In order to do that, you can use the sh method. You can then use the xcrun simctl commands.

lane :my_test_lane_name do
 sh "xcrun simctl shutdown all && xcrun simctl erase all && xcrun simctl --set testing shutdown all && xcrun simctl --set testing erase all"
end

Running Tests

There are a few ways in which you can run your tests. I will show 2 ways here. You can use the sh method and use the xcodebuild command. There are a lot of different options you can use to configure the runs so run xcodebuild -help in the terminal for more information on all of those but here is an example:

lane :my_test_lane_name do
 sh "xcodebuild -quiet -workspace ../myworkspace.xcworkspace -scheme MYSCHEMENAME -sdk iphonesimulator -derivedDataPath '#{derivedDataPath}' -destination 'platform=iOS Simulator,name=iPhone 15 Pro Max,OS=17.4' -testPlan TESTPLANNAME -parallel-testing-enabled YES -parallel-testing-worker-count 2 -enableCodeCoverage YES -resultBundlePath TestResults test | xcbeautify"
end

Another way is to use fastlane scan.

lane :my_test_lane_name do
  scan(
      workspace: "myworkspace.xcworkspace",
      scheme: "MYSCHEME",
      device: "iPhone 15 Pro Max (17.4)",
      testplan: "test_plan_name",
      parallel_testing: "YES",
      concurrent_workers: 3,
      number_of_retries: 1,
      result_bundle: true,
      output_remove_retry_attempts: true
  )
end

Using scan will output the number of tests ran and number of failures like shown below.

👉 You may need to include the following in order to not mask errors. See https://stackoverflow.com/questions/68465355/what-is-the-meaning-of-set-o-pipefail-in-bash-script or https://gist.github.com/mohanpedala/1e2ff5661761d3abd0385e8223e16425

lane :my_test_lane_name do
 sh "set -o pipefail"
end

Slack

If your team using Slack for communication, you can integrate with it to show when the lane has ran and completed.

If you are using scan to run the tests, you can add the following fields:

lane :my_test_lane_name do
  scan(
      workspace: "myworkspace.xcworkspace",
      scheme: "MYSCHEME",
      device: "iPhone 15 Pro Max (17.4)",
      testplan: "test_plan_name",
      parallel_testing: "YES",
      concurrent_workers: 3,
      number_of_retries: 1,
      result_bundle: true,
      output_remove_retry_attempts: true
      slack_url: "https://hooks.slack.com/services/...",
      slack_username: "Gitlab Bot",
      slack_message: "Slack message you want displayed",
      slack_icon_url: "https://about.gitlab.com/images/press/logo/png/gitlab-logo-gray-stacked-rgb.png",
      slack_channel: "name_of_slack_channel_to_display_results"
  )
end

If you aren’t using scan, you can also output the results by using slack:

slack(
    slack_url: "https://hooks.slack.com/services/...",
    message: "Slack message you want displayed",
    success: true,
    default_payloads: [:git_author, :last_git_commit],
    use_webhook_configured_username_and_icon: false,
    username: "Gitlab Bot",
    icon_url: "https://about.gitlab.com/images/press/logo/png/gitlab-logo-gray-stacked-rgb.png"
   )

In Slack, you will then see a message like this appear after the lane completes the run.

👉 If you are using Gitlab then you can also have it include a link to the job that ran in the slack message field.

slack_message: "CI Job: https://gitlab.com/-/jobs/" +  ENV['JOB_ID'],

The JOB_ID comes from your gitlab.yml file. You will need to include this

script:
  - export JOB_ID=$CI_JOB_ID

or if you want to use the branch instead:

script:
  - export COMMIT_REFERENCE=$CI_COMMIT_REF_NAME

👉 You can find pre-defined git variables here.

Result Bundles

The Result Bundle that you have seen above result_bundle: true (scan) or -resultBundlePath TestResults test (xcodebuild) contains the test failures and successes. In order to retrieve those results, you can do something like the following in your fastfile:

sh "xcrun xcresulttool get --path TestResults.xcresult --format json >results.json"
json = File.read('results.json')
result = JSON.parse(json)
numTests = result["metrics"]["testsCount"]["_value"]
numTestFailures = result["metrics"]["testsFailedCount"] ? result["metrics"]["testsFailedCount"]["_value"] : 0

When ran, you should see the file get generated. When you open the json file, if you search for metrics, you will see that it has the testCount, totalCoveragePercentage, and testsFailedCount (if there were failures).

UI Message

UI.message("---------------")
UI.message("Tests failed: " + testFailures.to_s)
UI.message("---------------")

You can use the UI class to print out results then if needed. See https://docs.fastlane.tools/advanced/actions/ and the section on Interacting with the user.

• https://www.rubyguides.com/2018/09/ruby-conversion-methods/

When ran, it will then look like this: