Creating a Highly Testable Networking Layer in Combine
In this article
For most teams supporting existing iOS projects (n-2 version back), it just recently became reasonable to use Combine when creating a network layer, since it requires iOS 13+. This sample project is based on working with challenges with a real client. The result was worth sharing!
Let's first outline a few qualifiers on what we want to accomplish:
- The networking layer should not make assumptions about what to add (it should not always add
content-type: "application/json", for example). - It should be easy to add headers to the request.
- It should be easy to test with complex scenarios.
- It can assume HTTP/HTTPS is being used.
- It should turn a response into a
Result, specifically aResult<DeserializedType, CustomError>. - It should be able to handle using a JWT refresh token to get a new access token (only once!) when an unauthorized response is received.
- It should end up being highly readable.
- It should be very easy to inject the desired
Resultin tests that are setting up responses from the server.
Before we dive into all the details, there are a few dependencies used.
Note: None of these are strictly necessary, most of them are built to help with testing.
- Swinject: A swift dependency injection framework. Its use here is lightweight, to say the least. It's combined with a Swift5 property wrapper
@DependencyInjectedfor readability. This could easily be removed from the project - OHHTTPStubs: HTTP Stubbing library, the only * is that we wrapped it in a fluent API for convenience and readability. Learn more about writing a fluent API wrapper around OHHTTPStubs.
- Cuckoo: A swift mocking framework (based on codegen). It has some limitations but ended up really speeding up mocking.
- Fakery: A random data generator, entirely unnecessary but was a way to generate test data.
Steps to a testable networking layer
In true TDD fashion, let's start by looking at some tests. So we know we want to support all the common HTTP Verbs, and we know we have at least 1 error condition we need to deal with in regards to malformed URLs.
Let's also make this protocol-based, so that consumers can make either a struct or a class capable of performing REST calls.
extension API { //this is literally an empty struct in the prod code, I like the namespacing aspect
struct JSONPlaceHolder: RESTAPIProtocol { //struct that conforms to the protocol, while network calls are stubbed JSONPlaceHolder is a nice little fake REST API you can call.
var baseURL: String = "https://jsonplaceholder.typicode.com"
}
}
class APITests:XCTestCase {
var subscribers = Set<AnyCancellable>() //store ongoing subscribers (ongoing calls) here, for simplicities sake.
override func setUp() {
//NOTE: Subscribers are cancelled on deinit, so realistically the removeAll call is all that is needed to stop any ongoing calls.
subscribers.forEach { $0.cancel() }
subscribers.removeAll()
}
func testAPIMakesAGETRequest() {
let json = """
[
{
userId: 1,
id: 1,
title: "sunt aut facere repellat provident occaecati excepturi optio reprehenderit",
body: "quia et suscipit suscipit recusandae consequuntur expedita et cum reprehenderit molestiae ut ut quas totam nostrum rerum est autem sunt rem eveniet architecto"
},
]
""".data(using: .utf8)!
//This is the fluent wrapper around OHHTTPStubs, you'll also note the convenience initializer for URLRequest
StubAPIResponse(request: .init(.get, urlString: "https://jsonplaceholder.typicode.com/posts"),
statusCode: 200,
result: .success(json))
let api = API.JSONPlaceHolder()
var GETFinished = false
api.get(endpoint: "posts")
.sink(receiveCompletion: { (completion) in
switch completion {
case .finished: GETFinished = true
case .failure: XCTFail("Call should succeed")
}
}) { (value) in
XCTAssertEqual((value.response as? HTTPURLResponse)?.statusCode, 200)
XCTAssertEqual(String(data: value.data, encoding: .utf8), String(data: json, encoding: .utf8))
}.store(in: &subscribers)
waitUntil(GETFinished)
XCTAssert(GETFinished)
}
func testAPIThrowsErrorWhenGETtingWithInvalidURL() {
var api = API.JSONPlaceHolder()
api.baseURL = "FA KE"
var GETFinished = false
api.get(endpoint: "notreal")
.sink(receiveCompletion: { (completion) in
GETFinished = true
switch completion {
case .finished: XCTFail("Should have thrown error")
case .failure(let error):
XCTAssertEqual((error as? API.URLError), API.URLError.unableToCreateURL)
}
}, receiveValue: { _ in })
.store(in: &subscribers)
waitUntil(GETFinished)
XCTAssert(GETFinished)
}
//do the same for PUT, POST, PATCH, and DELETE calls
}
Great! We know what we want the API to look like. When you call get you receive an erased dataTaskPublisher AnyPublisher<(data: Data, response: URLResponse), Error>. As a quick note for those who aren't aware, when you use Combine and start chaining publishers together, you start getting very complex objects. eraseToAnyPublisher() is a method that, well, erases that complex type down to an AnyPublisher of some kind.
Let's start by creating a convenient way to create that erased DataTaskPublisher:
extension URLSession {
typealias ErasedDataTaskPublisher = AnyPublisher<(data: Data, response: URLResponse), Error>
func erasedDataTaskPublisher(
for request: URLRequest
) -> ErasedDataTaskPublisher {
dataTaskPublisher(for: request)
.mapError { $0 }
.eraseToAnyPublisher()
}
}
Now let's create our protocol:
protocol RESTAPIProtocol {
//we'll get into this later, but this is a way to add headers to a request as it goes out
typealias RequestModifier = ((URLRequest) -> URLRequest)
var baseURL:String { get }
var urlSession:URLSession { get }
}
extension RESTAPIProtocol {
var urlSession: URLSession { URLSession.shared }
func get(endpoint:String, requestModifier:@escaping RequestModifier = { $0 }) -> URLSession.ErasedDataTaskPublisher {
guard let url = URL(string: "\(baseURL)")?.appendingPathComponent(endpoint) else {
//malformed URL, fail.
return Fail<URLSession.DataTaskPublisher.Output, Error>(error: API.URLError.unableToCreateURL).eraseToAnyPublisher()
}
let request = URLRequest(url: url)
return createPublisher(for: request, requestModifier: requestModifier)
}
func put(endpoint:String, body: Data?, requestModifier:@escaping RequestModifier = { $0 }) -> URLSession.ErasedDataTaskPublisher {
guard let url = URL(string: "\(baseURL)")?.appendingPathComponent(endpoint) else {
return Fail<URLSession.DataTaskPublisher.Output, Error>(error: API.URLError.unableToCreateURL).eraseToAnyPublisher()
}
var request = URLRequest(url: url)
request.httpMethod = "PUT"
request.httpBody = body
return createPublisher(for: request, requestModifier: requestModifier)
}
//similar methods for post, patch, and delete
func createPublisher(for request:URLRequest, requestModifier:@escaping RequestModifier) -> URLSession.ErasedDataTaskPublisher {
//Start by publishing the request, and give it an error type
//Then map it into the dataTaskPublisher
//Run it through the request modifier, so people can add headers, or otherwise modify the request
Just(request).setFailureType(to: Error.self)
.flatMap { request -> URLSession.ErasedDataTaskPublisher in
return self.urlSession.erasedDataTaskPublisher(for: requestModifier(request))
}.eraseToAnyPublisher()
}
}
Alright, so you may have noticed a "requestModifier" that's in there. The idea is we want to give easy access to be able to change aspects of the request, for example, the headers.
To do this, we'll create some extensions on URLRequest. (I'll spare you the tests as they are very straightforward.)
import Foundation
extension URLRequest {
func addingBearerAuthorization(token: String) -> URLRequest {
var request = self
request.setValue("Bearer \(token)", forHTTPHeaderField: "Authorization")
return request
}
func acceptingJSON() -> URLRequest {
var request = self
request.setValue("application/json", forHTTPHeaderField: "Accept")
return request
}
func sendingJSON() -> URLRequest {
var request = self
request.setValue("application/json", forHTTPHeaderField: "Content-Type")
return request
}
}
Excellent! Now it's trivial for us to modify the request to add authorization and accept (and send) JSON.
api.get(endpoint: "posts") {
$0.addingBearerAuthorization(token: "1234")
.acceptingJSON()
.sendingJSON()
}
.sink {
//handle success, or failure from the request
}.store(in: &subscribers) //or assign to a variable
Next, let's deal with something more complicated: receiving a 401, then using a refresh token to get a new access token and retry the same call, but only do that once.
In this article about creating your own custom combine operator, we created RetryOn which let us retry with a specific error, and even chain an additional publisher on before retrying. Let's use it here.
extension URLSession.ErasedDataTaskPublisher {
func retryOnceOnUnauthorizedResponse(chainedRequest:AnyPublisher<Output, Error>? = nil) -> AnyPublisher<Output, Error> {
tryMap { data, response -> URLSession.ErasedDataTaskPublisher.Output in
if let res = response as? HTTPURLResponse,
res.statusCode == 401 {
throw API.AuthorizationError.unauthorized
}
return (data:data, response:response)
}
.retryOn(API.AuthorizationError.unauthorized, retries: 1, chainedPublisher: chainedRequest)
.eraseToAnyPublisher()
}
}
We can pretend that we were dealing with an API — we'll call it the Identity Service. Now this theoretical Identity Service we're dealing with sticks everything in a top-level object called "result".
What we'd really like is to be able to unwrap that result object so that we can serialize to Codable objects of our own. Let's create a Combine operator for that. (Again, this is fairly trivial to test, but I'll spare you in this article.)
extension URLSession.ErasedDataTaskPublisher {
func unwrapResultJSONFromAPI() -> Self {
tryMap {
if let json = try JSONSerialization.jsonObject(with: $0.data, options: []) as? [String:Any],
let result = (json["result"] as? [String:Any]) {
let data = try JSONSerialization.data(withJSONObject: result, options: [])
return (data:data, response: $0.response)
}
return $0
}.eraseToAnyPublisher()
}
}
Great! Let's finally get to creating our Identity Service. What does it look like?
Again, let's start with the tests.
class IdentityServiceTests: XCTestCase {
var ongoingCalls = Set<AnyCancellable>()
override func setUp() {
ongoingCalls.removeAll()
}
func testIdentityServiceUsesURLSessionDefaultConfiguration() {
XCTAssertEqual(API.IdentityService().urlSession, URLSession.shared)
}
func testProfileIsFetchedFromAPI() {
StubAPIResponse(request: .init(.get, urlString: "\(API.IdentityService().baseURL)/me"),
statusCode: 200,
result: .success(validProfileJSON.data(using: .utf8)!))
let api = API.IdentityService()
var called = false
api.fetchProfile.sink { (result) in
switch result {
case .success(let profile):
XCTAssertEqual(profile.firstName, "Joe")
XCTAssertEqual(profile.lastName, "Blow")
XCTAssertEqual(profile.preferredName, "Zarathustra, Maestro of Madness")
XCTAssertEqual(profile.email, "Tyler.Keith.Thompson@gmail.com")
XCTAssertEqual(profile.dateOfBirth, DateFormatter("yyyy-MM-dd'T'HH:mm:ss").date(from: "1990-03-26T00:00:00"))
XCTAssertEqual(profile.createdDate, DateFormatter("yyyy-MM-dd'T'HH:mm:ss.SSS").date(from: "2018-07-26T19:33:46.6818918"))
XCTAssertEqual(profile.address?.line1, "111 Fake st")
XCTAssertEqual(profile.address?.line2, "")
XCTAssertEqual(profile.address?.city, "Denver")
XCTAssertEqual(profile.address?.state, "CA")
XCTAssertEqual(profile.address?.zip, "80202")
case .failure(let error):
XCTFail(error.localizedDescription)
}
called = true
}.store(in: &ongoingCalls)
waitUntil(0.3, called)
XCTAssert(called)
}
func testFetchProfileThrowsAPIBorkedError() {
StubAPIResponse(request: .init(.get, urlString: "\(API.IdentityService().baseURL)/me"),
statusCode: 200,
result: .success(Data("Invalid".utf8)))
let api = API.IdentityService()
var called = false
api.fetchProfile.sink { (result) in
switch result {
case .success(_): XCTFail("Should not have a successful profile")
case .failure(let error):
XCTAssertEqual(API.IdentityService.FetchProfileError.apiBorked, error)
}
called = true
}.store(in: &ongoingCalls)
waitUntil(0.3, called)
XCTAssert(called)
}
func testFetchProfileRetriesOnUnauthorizedResponse() {
StubAPIResponse(request: .init(.get, urlString: "\(API.IdentityService().baseURL)/me"),
statusCode: 401)
.thenRespondWith(request: .init(.post,
urlString: "\(API.IdentityService().baseURL)/auth/refresh"),
statusCode: 200,
result: .success(validRefreshResponse))
.thenVerifyRequest { request in
XCTAssertEqual(request.httpMethod, "POST")
XCTAssertEqual(request.bodySteamAsData(), try? JSONSerialization.data(withJSONObject: ["refreshToken":User.refreshToken], options: []))
}
.thenRespondWith(request: .init(.get, urlString: "\(API.IdentityService().baseURL)/me"),
statusCode: 200,
result: .success(validProfileJSON.data(using: .utf8)!))
.thenVerifyRequest { request in
XCTAssertEqual(request.httpMethod, "GET")
XCTAssertEqual(request.value(forHTTPHeaderField: "Accept"), "application/json")
XCTAssertEqual(request.value(forHTTPHeaderField: "Content-Type"), "application/json")
XCTAssertEqual(request.value(forHTTPHeaderField: "Authorization"), "Bearer \(User.accessToken)")
}
let api = API.IdentityService()
var called = false
api.fetchProfile.sink { (result) in
switch result {
case .success(let profile): XCTAssertEqual(profile.firstName, "Joe")
case .failure(_):
XCTFail("Should not have an error")
}
called = true
}.store(in: &ongoingCalls)
waitUntil(called)
XCTAssert(called)
}
func testFetchProfileFailsOnUnauthorizedResponseIfRefreshFails() {
StubAPIResponse(request: .init(.get, urlString: "\(API.IdentityService().baseURL)/me"),
statusCode: 401)
.thenRespondWith(request: .init(.post, urlString: "\(API.IdentityService().baseURL)/auth/refresh"),
statusCode: 200,
result: .success(Data("".utf8)))
.thenRespondWith(request: .init(.get, urlString: "\(API.IdentityService().baseURL)/me"),
statusCode: 200,
result: .success(validProfileJSON.data(using: .utf8)!))
let api = API.IdentityService()
var called = false
api.fetchProfile.sink { (result) in
switch result {
case .success(_): XCTFail("Should not have successful response")
case .failure(let error):
XCTAssertEqual(API.IdentityService.FetchProfileError.apiBorked, error)
}
called = true
}.store(in: &ongoingCalls)
waitUntil(called)
XCTAssert(called)
}
}
extension IdentityServiceTests {
var validRefreshResponse:Data {
Data("""
{
"result" : {
"accessToken" : "\(UUID().uuidString)"
}
}
""".utf8)
}
var validProfileJSON:String {
"""
{
"self": {
"firstName": "Joe",
"lastName": "Blow",
"preferredName": "Zarathustra, Maestro of Madness",
"email": "Tyler.Keith.Thompson@gmail.com",
"dateOfBirth": "1990-03-26T00:00:00",
"gender": "male",
"phoneNumber": "3033033333",
"address": {
"line1": "111 Fake st",
"line2": "",
"city": "Denver",
"stateOrProvince": "CA",
"zipCode": "80202",
"countryCode": "US"
}
},
"isVerified": true,
"username": "Tyler.Keith.Thompson@gmail.com",
"termsAcceptedDate": "2018-07-26T19:33:46.8381401",
"isTermsAccepted": true,
"createdDate": "2018-07-26T19:33:46.6818918",
}
"""
}
}
Some things to look for: we created a struct called IdentityService and namespaced it under API. Because Cuckoo (our mocking framework) can't mock structs, we also created IdentityServiceProtocol so that a mock could be created.
As long as we already had to make a protocol, we may as well have it do all the heavy lifting related to the Identity Service.
protocol IdentityServiceProtocol: RESTAPIProtocol {
var fetchProfile: AnyPublisher<Result<User.Profile, API.IdentityService.FetchProfileError>, Never> { get }
}
extension IdentityServiceProtocol {
var baseURL: String {
"https://some.identityservice.com/api"
}
var fetchProfile: AnyPublisher<Result<User.Profile, API.IdentityService.FetchProfileError>, Never> {
self.get(endpoint: "/me", requestModifier: {
$0.addingBearerAuthorization(token: User.accessToken)
.acceptingJSON()
.sendingJSON()
}).retryOnceOnUnauthorizedResponse(chainedRequest: refresh)
.unwrapResultJSONFromAPI()
.map { $0.data }
.decodeFromJson(User.Profile.self) //codable object
.receive(on: DispatchQueue.main)
.map(Result.success) //Notice that we choose to return a `Result` type for convenience. This will make a lot of sense later.
.catch { error in Just(.failure((error as? API.IdentityService.FetchProfileError) ?? .apiBorked)) }
.eraseToAnyPublisher()
}
//this can be left private, nothing using the identity service should really know that refresh is happening. Moreover the tests don't need access because they can stub responses, including an unauthorized response.
private var refresh:URLSession.ErasedDataTaskPublisher {
post(endpoint: "/auth/refresh", body: try? JSONSerialization.data(withJSONObject: ["refreshToken":User.refreshToken], options: []), requestModifier: {
$0.acceptingJSON()
.sendingJSON()
}).unwrapResultJSONFromAPI()
.tryMap { v -> URLSession.ErasedDataTaskPublisher.Output in
let json = try? JSONSerialization.jsonObject(with: v.data, options: []) as? [String:Any]
guard let accessToken = json?["accessToken"] as? String else {
throw API.AuthorizationError.unauthorized
}
User.accessToken = accessToken
return v
}.eraseToAnyPublisher()
}
}
extension API {
struct IdentityService: IdentityServiceProtocol {
enum FetchProfileError: Error {
case apiBorked
}
}
}
Great — we have a networking layer and we can stub responses from a server. But what does it look like if we want to use this in a view? How easy is it to create the responses we want? Do we have to stub all the way back to JSON from the server? We can do better.
Let's start by thinking about how we want to pull in our IdentityService to our view. I personally am a fan of Swinject and so created a fairly simple property wrapper around it. To understand it, let's finally take a look at that base API struct we've been using for namespacing.
struct API {
enum URLError: Error {
case unableToCreateURL
}
enum AuthorizationError:Error {
case unauthorized
}
static var container = Container()
}
That Container is for Swinject, it's where our dependencies (to be injected) are stored. So how would we create a property wrapper around this?
@propertyWrapper
public struct DependencyInjected<Value> {
let name:String?
let container:Container
public init(wrappedValue value: Value?) {
name = nil
container = API.container
}
public init(wrappedValue value: Value? = nil, name:String) {
self.name = name
container = API.container
}
public init(wrappedValue value: Value? = nil, container containerGetter:@autoclosure () -> Container, name:String? = nil) {
self.name = name
container = containerGetter()
}
public lazy var wrappedValue: Value? = {
container.resolve(Value.self, name: name)
}()
}
Great! We now have an easy way to pull in dependencies, and in this case, they are lazy as well. Meaning we can be rest assured we use the same instance.
I wanted to be able to run this project on my Mac (for speed) and not have the overhead of the iOS simulator. So this isn't a real UIViewController, but it doesn't have to be in order to showcase how this is used.
class ProfileViewController {
@DependencyInjected var identityService:IdentityServiceProtocol?
var nameLabelText:String = ""
var currentNetworkCalls = Set<AnyCancellable>() //these get cleaned up when the ViewController does, canceling all network calls along the way
func fetchProfile() {
identityService?.fetchProfile.sink { [weak self] result in
if case .success(let profile) = result {
self?.nameLabelText = profile.firstName
}
}.store(in: ¤tNetworkCalls)
}
}
So our IdentityServiceProtocol is @DependencyInjected (from API.Container). If our view gets collected by ARC, all ongoing network calls get canceled — cool!
Finally, we have a pretty simple check for a serialized User.Profile object and we display their first name on a completely fake label. How easy is it to test this, though?
This is where Cuckoo comes in handy as a mocking framework. It made this incredibly straightforward.
class ProfileViewControllerTests: XCTestCase {
func testFetchingProfileFromAPI() {
let mock = MockIdentityServiceProtocol()
.registerIn(container: API.container)
let expectedProfile = User.Profile.createForTests()
stub(mock) { stub in
//notice that because we return a `Result` publisher already, it's very easy to just create a successful response, or an error for that matter.
_ = when(stub.fetchProfile.get
.thenReturn(Result.Publisher(.success(expectedProfile)).eraseToAnyPublisher()))
}
let testViewController = ProfileViewController()
testViewController.fetchProfile()
verify(mock, times(1)).fetchProfile.get()
XCTAssertEqual(testViewController.nameLabelText, expectedProfile.firstName)
}
//let's make sure the calling code uses `[weak self]`
func testFetchingProfileDoesNotRetainAStrongReference() {
let mock = MockIdentityServiceProtocol()
.registerIn(container: API.container)
stub(mock) { stub in
_ = when(stub.fetchProfile.get
.thenReturn(Result.Publisher(.success(User.Profile.createForTests()))
.delay(for: .seconds(10), scheduler: RunLoop.main)
.eraseToAnyPublisher()))
}
var testViewController:ProfileViewController? = ProfileViewController()
weak var ref = testViewController
testViewController?.fetchProfile()
testViewController = nil
verify(mock, times(1)).fetchProfile.get()
XCTAssertNil(ref)
}
}
Hopefully, you've now got a good idea of how to write a pretty straightforward networking layer in Combine. At the end of this experiment, we had 99.3 percent code coverage and were able to test both the networking side of this and easily inject our own Result into our UIViewController.
It's also worth noting that we had a mutation score of 100 percent. What this means is that the code was not only covered, but that those unit tests are valuable — in other words, they can detect breaking changes.
Talk with us today
Feel free to share your experiences with Combine in the comment section below, or reach out to us directly to start discussing your unique considerations.