iOS Unit Testing for the User Interface

Out of the box Apple testing solutions

For Apple platforms, the XCUITest suite is the standard for testing a user interface, so let's take a look at its architecture. A developer's application (the system under test, or SUT) is launched within Apple's test framework, which then uses a second test runner application to control it through a private low-level system interface in Apple's framework (e.g., UIKit) and execute the tests from within the test runner's process. This private inter-application communication allows the test runner to query the SUT for the current user interface tree and return simulated events such as user touches and key presses. 

There are benefits and drawbacks to this pattern. The test runner is very loosely coupled to the SUT, which is running exactly as it does normally. However all this inter-application communication is inefficient and slow, and the very loose coupling makes the tests susceptible to timing and synchronization problems. In addition, a test sequence normally needs to work through the SUT exactly as a user does to get to the component of interest for the specific test. This can be repetitive and time consuming for both test development and execution.

Alternately, Apple's test framework provides a unit testing framework with a different architecture. For unit tests, the test framework launches the SUT within the process of the test runner, so the tests and the SUT are both running in the same process, eliminating the need for inter-application communication. The SUT first receives the typical application launch lifecycle events, but then each unit test function is called directly by the test runner. This pattern also has benefits and drawbacks. Tests run quickly, with direct access to the application state and code, but they are tightly coupled to the test framework which lacks the ability to simulate user events.

UIUTest is a test framework that works with Apple's unit test platform in iOS to facilitate many UIKit user interface style tests in-process where they're faster, more efficient and less fragile than traditional user interface tests. This makes automated tests more efficient and allows them to be deployed in situations where they frequently wouldn't be as efficient. It also encourages better functional separation within a codebase and makes development patterns like Test-Driven Development (TDD) practical for the user interface. Importantly, UIUTest takes care to ensure the integrity of tests by accounting for the state of an application's view hierarchy in the same way conventional UI tests do.

UIUTest basics

UIUTest consists mostly of Swift extensions to existing UIKit classes. In some cases, it uses the low-level Objective-C runtime to swizzle existing methods to track activity and save that information via associated objects. View controllers are generally the starting point of UIUTest user interface tests, and tools are provided to load and initialize them directly for testing via either programmatic creation or instantiation from storyboards. Extensions to controls, text fields, gestures and other UIKit elements provide tools for simulating user touches and other events. 

Although it's not a requirement, typical UIUTest tests follow the established XCUITest pattern of referencing views via accessibility identifiers. However, since tests have direct access to the view hierarchy, this is much more efficient than it is in XCUITest. One word of caution though, in order to simulate UIKit behavior it's sometimes necessary for UIUTest to exploit private UIKit implementation details. Apple doesn't always export everything needed to simulate the full behavior of some elements in the public API. These details seem stable, but it's always possible an OS update will require an update to UIUTest. Also, you'll want to be careful to only include UIUTest in your test targets to avoid those pesky app review questions about SPI use when submitting to the App Store.

Let's get started with a trivial example. Here's a view controller described elsewhere in a storyboard, with a button that changes its label from "Ready" to "Done" when tapped.

    class TrivialViewController: UIViewController {
        @IBAction private func buttonTapped(sender: UIButton) {
            sender.setTitle("Done", for: .normal)
        }
    }

We can test this within our unit test target like this:

    func testReadyButton() {
        let viewController = UIViewController.loadFromStoryboard(identifier: "Trivial")             // 1
        let button = viewController?.view?.viewWithAccessibilityIdentifier("ready") as? UIButton    // 2
        XCTAssertEqual(button?.currentTitle, "Ready")                                               // 3
        button?.simulateTouch()                                                                     // 4
        XCTAssertEqual(button?.currentTitle, "Done")                                                // 5
    }

1. Load the view controller with the identifier "Trivial" from the default (i.e., main) storyboard (if one exists).
2. Obtain a button with the accessibility identifier "ready" from the controller's view hierarchy.
3. Assert the button exists with the initial title "Ready."
4. Simulate a user touch on the button.
5. Assert the button title is now "Done."

This single unit level test is actually testing several things. Our storyboard defines a view controller with the expected identifier, which contains a button with another identifier. The initial label of the button is what we expect and tapping the button changes the label as it should. However, also note what we're not testing here. This is admittedly a trivial case, but here the public interface is simply the identifiers of our elements and that's all we're using to test the behavior we expect. The button action method is private to the view controller to indicate it's an internal implementation detail that, following best testing practices, we don't care about in our test. We're entirely free to refactor the implementation without modifying the test, giving us a high degree of confidence we haven't caused a regression of any kind.

It's also worth noting our test is instantiating a single view controller and testing it independently from the rest of the application. This not only makes tests more modular, but it can make them much faster and easier to write since the normal application flow needed to get to this view controller is eliminated.

A more realistic example

Here's a simple authentication screen containing elements typical in many applications. The view controller for this screen implements some basic presentation logic we want to test.

• If either the user name or password field is blank, the sign-in button is disabled. Otherwise, it is enabled.
• The password field is initially a secure text field to obscure the password text.
• Tapping the show/hide button makes the text field insecure to show the password text, and tapping the button again reverts it.
• When the password field is secure, the text of the show/hide button is "Show," when it is insecure the button text is "Hide."
• The user name and password fields limit text input length to some reasonable arbitrary value as an extra security measure.
• Tapping the sign-in button calls an injected authentication module which returns a result used to show or hide an error message.

The public interface to this screen is two text fields, two buttons and an injected authenticator, so ideally we want to test all of the behavior above without using anything else. For the sake of brevity, we'll implement a couple of tests here assuming the controls are loaded elsewhere. The other tests are valuable but are left as they say, as an exercise for the reader.

    func testSecurePasswordField() {
        XCTAssertTrue(passwordField.isSecureTextEntry)
        XCTAssertEqual(showPasswordButton.currentTitle, "SHOW")

        showPasswordButton.simulateTouch()

        XCTAssertFalse(passwordField.isSecureTextEntry)
        XCTAssertEqual(showPasswordButton.currentTitle, "HIDE")

        showPasswordButton.simulateTouch()

        XCTAssertTrue(passwordField.isSecureTextEntry)
        XCTAssertEqual(showPasswordButton.currentTitle, "SHOW")
    }

Here we test the behavior of the secure password field and show/hide button. This example is similar to the trivial example above, but checks the state of password security and taps the button a second time to check that conditions revert to the original state. Since this is a unit test we have the luxury of creating a mock authenticator and injecting that dependency for our tests. Therefore we can easily check for the proper strings being authenticated and filter out any network interaction we normally expect when testing a user interface like this. 

In our final example, we use such a mock authenticator and test a valid user name with an invalid password.

    func testValidUserNameWithInvalidPassword() {
        let mockAuthenticator = MockAuthenticator()
        viewController.setAuthenticator(mockAuthenticator)
        userNameField.text = MockAuthenticator.validUser
        passwordField.setTextAndNotify(MockAuthenticator.invalidPassword)   // 1

        authenticateButton.simulateTouch()

        XCTAssertEqual(mockAuthenticator.calledWithUserName, MockAuthenticator.validUser)
        XCTAssertEqual(mockAuthenticator.calledWithPassword, MockAuthenticator.invalidPassword)
        XCTAssertNil(viewController.mostRecentlyPerformedSegue)             // 2
        XCTAssertFalse(invalidCredentialsLabel.isHidden)
    }

By now you can probably see where this is going, we enter a valid user name and an invalid password and tap the authenticate button. Then check that the authenticator has been called with the expected values, and the view controller hasn't tried to segue to another view controller but is showing the invalid credentials label. However, there are a couple of new twists here. 

Before we tap the authenticate button, we need it to be enabled so it will respond, and for that to happen we need the password text entry to inform its delegate it has changed. When setting the text of a UIKit element programmatically we don't normally get the same behavior as from actual user actions on a device, so on the line marked "1" we use a UIUTest method to both set the text and send the appropriate UIKit messages. Since both text fields now contain text, we expect the button to be enabled and invoke the authenticator. 

The second twist on the line marked "2" is that UIViewController has been extended by UIUTest to track segue activity. Since we're using invalid credentials here, we don't expect it to segue to a new view controller, but elsewhere we can write a test that expects the proper segue for valid credentials.

Under the hood

In order to faithfully simulate user behavior, it's important to handle simulated actions identically to real actions. For example, tapping on a button doesn't always invoke that button's action. The button may be hidden, disabled or obscured by another view or window. When simulating a tap on an element like a button, UIUTest performs hit testing to ensure the element hit by a tap on the center of the button is in fact part of the button in question. Therefore a simulated tap on a button obscured by the soft keyboard, for example, will do nothing as expected.

Another challenge of simulating iOS is reverse-engineering the various notification patterns supported by UIKit elements. These patterns include delegates, NotificationCenter, target/action and KVO. Some UIKit elements support one of these, while others support another or even multiple patterns. UIUTest handles these notifications, in the proper order, but if the code being tested relies on certain notifications it's important to invoke methods such as UITextField.setTextAndNotify described in the example above. Simply setting element values programmatically in UIKit typically doesn't invoke the notifications which are issued when an action is performed directly by the user.

Other issues that can sometimes appear in tests include animations, run loop processing and asynchronous thread delays. These can be mitigated with techniques such as disabling UIView animations, allowing the run loop to run briefly before asserting results and ensuring any background thread processing is accounted for.

Conclusion

Testing user interface elements in-process with UIUTest can provide several benefits. Typically these tests run much faster and are generally easier to write. This encourages more complete automated testing, which can mean higher quality code. Going directly to the specific screen required for a test and injecting the necessary state allows tests to be written before the code, allowing a TDD methodology which can also result in more complete tests. 

Writing tests first can be especially useful when fixing bugs, since the conditions required to reproduce the bug can be configured within the test, allowing the failure to be observed before it is fixed and then confirming the fix. For these and other reasons, UIUTest can be a useful addition to an iOS developer's toolbox.

UIUTest is open source under the MIT license. It can be found at https://github.com/nallick/UIUTest.