iOS Continuous Integration Guide

iOS Continuous Integration Guide

This guide will help you set up a server to test, build, and deploy your iOS apps automatically.

Here at Casa, we build solutions to help simplify the use of cryptocurrencies like Bitcoin. Due to the nature of our products, it is extremely important for all code that gets introduced to be peer reviewed. This alleviates any possibility of insecure code finding its way into our products.

We move quickly and we ship product constantly. Shipping code quickly introduces risks. It’s important to put in place safeguards to protect developers from accidentally introducing bugs. That’s why we recommend Continuous Integration.

What Is Continuous Integration and Why Use It?

Continuous Integration (CI) is an automated way for you to test code and push builds for testing and App Store deployment. It gives us several amazing benefits:

Strategy Overview

There are two main recommended paths toward Continuous Integration for iOS. The path supported and recommended by Apple relies on Xcode Server, which allows you to configure and run ‘Bots’ to execute tests within your project. However, at the time of writing (Xcode 10.1), these integration bots do not support pushing the builds to the app store and I have had no luck working around this restriction.

Here is a link to Apple’s guide to CI with Xcode Server: https://developer.apple.com/library/archive/documentation/IDEs/Conceptual/xcode_guide-continuous_integration/index.html

The second path, outlined below, involves using a Jenkins server to handle task management and a library called Fastlane that helps simplify all the building, signing, and uploading of the builds. Jenkins is listening for new events and then launching our builds while Fastlane is actually performing the heavy lifting and managing all of the command line utilities to perform the actions that we need locally on the server.

Jenkins will be set up with two tasks:

Fastlane will be configured with a few simple jobs to make this possible:

Deployments

Now, to deploy a new Prototype and Dev build, we need to create a Pull Request from our feature branch into Development and merge it after approval. The Jenkins server will detect this change and launch a new Project-Dev task, which will run tests and push to the store.

To push to staging/prod, we need to PR from Development into Master. This will launch a new Project-Prod task that will run tests and push to the TestFlight.

Note: In addition to pushing to the correct git branch, you must also ensure that for each target, the app store listing has a new version pending with the correct new version number in the target. This is done by logging into appstoreconnect.apple.com , going to the given app, and selecting add new version . The version found in the info.plist for the given target in Xcode must be updated to match this version.

To check on the status of a build, you can log in to Jenkins from a remote computer by opening the IP address of the remote machine at port 8080 in any web browser (http://your.ip.address:8080). Once logged in, you may view the status of any previous jobs, view the console from any running jobs, and manually launch a new job.

Step-by-step Setup:

Access Remote Server (via macstadium.com)

Create a new account at macstadium.com. This will give you access to a fresh Mac Mini that is all set up for remote access.

To access the machine remotely, you may use Apple’s ‘Screen Sharing’ software and enter the IP address for the new machine as found in the Subscriptions tab in macstadium. The username will be administrator and the default password will be provided through their portal. Once logged in, change this default password.

Next, make sure the following are installed:

xcode-select --install

\curl -sSL https://get.rvm.io | bash -s stable --ruby

/usr/bin/ruby -e ‘$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)’

gem install cocoapods

sudo gem install fastlane -NV

Install Jenkins Server

While logged in as the administrator, install Jenkins. While there is a dmg installer available, I was unable to get Jenkins set up with the installer and instead installed using Homebrew.

brew install jenkins

When installing, you only need a minimal set of plugins, so don’t install with all the defaults. Instead, look through the plugins and only install ones that look useful, like GitHub.

Once the installer is run, you have to unlock it. Do this by running jenkins from the command line to launch Jenkins. Then, from a web browser, navigate to localhost:8080 to open the Jenkins application, which will direct you to access a local password file to unlock it.

Next, you’ll want to create a new Jenkins User. Open the Jenkins console in a web browser on the remote machine and hit Jenkins → Manage Jenkins → Manage Users → Add New. Create a new user and assign them a password.

Close the command line console to close Jenkins. Note that by default the workspace will be set at ~/.jenkins. You may wish to rename it, mv ~/.jenkins ~/jenkins to make the workspace more visible.

Configure Jenkins to Launch on Boot

In the current configuration, Jenkins will only run when the administrator is logged in and must be manually started by the user. We’ll want Jenkins to launch without requiring a login every time the computer boots.

To help make this task a bit easier, I enlisted help from a program called LaunchControl, available here for $10:

http://www.soma-zone.com/LaunchControl/

Install, purchase, and open LaunchControl on the remote computer, and let’s create a new Global Daemon.

On the top-left of the screen, you’ll see a drop down with User Agents selected. Change that to Global Daemons and select File→New.

We’ll configure this job as shown below:

Integrate Fastlane

We must now set up Fastlane in our local Xcode project.

From the project directory, on your mac, run

sudo gem install fastlane -NV
fastlane init

(this assumes you already have ruby installed on your local machine. If you don’t, see above to install rvm and ruby)

You’ll also want to add these lines into your ~/.bash_profile

export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

This will generate all the Fastlane files and integrate it into your project. You’ll likely want to add fastlane/fastfile into your project so you can edit it in Xcode and then add some of the Fastlane files to your .gitignore:

fastlane/report.xml
fastlane/Preview.html
fastlane/screenshots
fastlane/test_output

You may also want to follow the instructions here to add a gemfile and lock the versioning of Fastlane: https://docs.fastlane.tools/getting-started/ios/setup/

Now in Xcode, we can create our Fastlane tasks. Open the fastfile generated by Fastlane and add some tasks:

desc "Push a new proto dev build to the App Store"
    lane :release_dev do
        set_info_plist_value(path: "Project/Info-Proto.plist", key: "commit_hash", value: last_git_commit[:commit_hash] )

build_app(workspace: "Project.xcworkspace", scheme: "Project-Dev")

upload_to_app_store(skip_metadata: true, skip_screenshots: true, app_identifier: "com.Dev", username: "user_name@apple.com")
    end

desc "Run automated tests"
    lane :test do
        run_tests(workspace: "Project.xcworkspace",
                  devices: ["iPhone 6"],
                  scheme: "Project-Dev", clean: true)
    end

desc "Run automated ui tests"
    lane :test_ui do
        run_tests(workspace: "Project.xcworkspace",
                   devices: ["iPhone 6"],
                   scheme: "Project-Proto", clean: true)
    end

Note the set_info_plist command is going to take the commit hash of the current build that we are compiling and add it into our project at build time so we can access it and display it in code within our project.

Now commit this code and push it up to the server.

git checkout -b 'fastlane'
git add .
git commit -am 'add fastlane'

Push this new branch and merge it into dev.

Configure GitHub Webhooks

Generate a new secret key for the webhook. Open the Jenkins console from a web browser and go to Manage Jenkins → Configure Jenkins → GitHub. If you do not see the GitHub section ensure the GitHub plugin is installed.

Select Advanced Settings and you’ll see a drop-down for Shared Secret. Select ‘add’ and follow the steps to generate and store a new shared secret.

We need to configure GitHub to send webhooks for our repository to Jenkins. Go to github.com and navigate to the repo settings → webhooks → add webhook

Configure Jenkins to Pull from GitHub

From the remote server, we need to add an SSH key to the admin user and add the key to our GitHub repo as a deploy key. Go to the repo settings page, under deploy keys and follow this guide to set up a new SSH key to the server and add it as a deploy key. Note that this deploy key must have write access in order to push tags to GitHub to tag our builds.

https://developer.github.com/v3/guides/managing-deploy-keys/#deploy-keys

Configure Jenkins Tasks

Now we need to get our project compiling and running tests on the server.

Log in to the remote server via screen sharing. Open Jenkins via localhost:8080.

Select ‘New Item’ and create a free style project. Call it Project-Dev.

pod repo update
pod install
fastlane test_ui
fastlane test
fastlane release_dev

We’ll want to then create a task for production releases. You can do this by simply copying the settings from the dev build, but change the names of the branches and the fastlane targets to execute.

Configure Email Notifications

To have Jenkins send emails after every build, first create a new gmail account exclusively for sending these build summary emails.

Next, configure Jenkins with a default email configuration

Go to Configure Jenkins — Manage Configuration

Under Extended E-mail Notification:

Scroll up to Jenkins Location

Now configure your tasks:

Install Certificates and Provision Profiles

This should now compile our project, but builds cannot successfully upload because the server does not have the certificates required to sign our code and to push the builds to the app store.

Certificate and provisioning profile management in iOS can be quite a pain, but here are the basic steps to get this working.

The project is now set up to use our new profiles and the profiles are stored on the admin keychain on the server, but we need to move them to the system keychain so that the Jenkins server can access them without the admin user being signed in.

On the server, open keychain access. From the Login keychain, open the certificates tab and find the certificates you downloaded previously.

Copy those certificates by clicking them to expand them and highlighting both the certificate and the associated private key. Then select Edit→ Copy. Then Paste those into the System keychain.

App Store Access

The last thing we need is access to upload the build to the app store. We’ll create a new user in appstore connect and add that user’s credentials to the server’s keychain.

echo ‘[server admin password]’ | sudo -S fastlane fastlane-credentials add --username [itunes username] --password [itunes password]

Enable Server Side Build Versioning

Let’s allow the CI system to increment our build numbers for us. That way, when we push new code, we won’t also have to increment the build numbers manually. Note that if we try to push builds to the app store with the same build number, the upload will already exist, and hence the build will fail.

First, we need to make sure our build number versioning is set to use whole numbers. If we are using something like 1.0.0.1, this system won’t work since we will have to do a bunch of parsing to successfully increment the build number.

Now we just add these ‘lanes’ to Fastlane in our fastfile.

lane :build_number_prod do
        version_number = get_version_number(:target => "Project-Prod")
        last_build_dev = latest_testflight_build_number(:app_identifier => "com.Proto", :version => version_number, :initial_build_number => 0, username: "you@apple.com")
        last_build_prod = latest_testflight_build_number(:app_identifier => "com.project.Staging", :version => version_number, :initial_build_number => 0, username: "you@apple.com")
        last_build = last_build_dev > last_build_prod ? last_build_dev : last_build_prod
        set_info_plist_value(path: "Project/Info-Staging.plist", key: "CFBundleVersion", value:  (last_build + 1).to_s)
        set_info_plist_value(path: "Project/Info-Prod.plist", key: "CFBundleVersion", value:  (last_build + 1).to_s)
        #tag our new build
        tag = version_number.to_s + "." + (last_build + 1).to_s
        if git_tag_exists tag: tag
            tag = tag + "-" + Time.now.to_i.to_s
        end
        git_add_tag tag: tag
        push_git_tags, remote: "Org/repo"
    end

lane :build_number_dev do
        version_number = get_version_number(:target => "Project-Dev")
        last_build_dev = latest_testflight_build_number(:app_identifier => "com.Proto", :version => version_number, :initial_build_number => 0, username: "you@apple.com")
        last_build_prod = latest_testflight_build_number(:app_identifier => "com.Staging", :version => version_number, :initial_build_number => 0, username: "you@company.com")
        last_build = last_build_dev > last_build_prod ? last_build_dev : last_build_prod
        set_info_plist_value(path: "Project/Info-Dev.plist", key: "CFBundleVersion", value:  (last_build + 1).to_s)
        set_info_plist_value(path: "Project/Info-Proto.plist", key: "CFBundleVersion", value:  (last_build + 1).to_s)
        #tag our new build
        tag = version_number.to_s + "." + (last_build + 1).to_s
        if git_tag_exists tag: tag
            tag = tag + "-" + Time.now.to_i.to_s
        end
        add_git_tag tag: tag
        push_git_tags, remote: "Org/repo"
    end

These tasks will essentially check TestFlight for the latest build number that has been pushed for the current version, then increment our build accordingly. It will then add a unique tag for this build in GitHub.

Once these lanes are created, modify your Jenkins tasks and add the commands fastlane build_number_dev to the dev build before fastlane release_proto and add fastlane build_number_prod to the prod build before fastlane release_staging

Locking It Down

The server is currently a little too easy to access. We need to turn off VNC / remote desktop access by default and require logins to come into the machine via SSH instead, where the SSH key is a hardware authentication device like a YubiKey.

Restrict SSH logins and require all logins to use a Hardware Token from our YubiKey:

PermitRootLogin=prohibit-password
PubkeyAuthentication=yes
ChallengeResponseAuthentication=no
PasswordAuthentication=no

Now log in to the server via Screen Sharing (VNC) and create these scripts to start and stop VNC. Save them in the administrators’ home folder and make sure they have executable permissions.

~/start_vnc.sh

#!/bin/bash
sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -activate -configure -allowAccessFor -specifiedUsers
sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -configure -users administrator -access -on -privs -all -setmenuextra -menuextra no -restart -agent

~/stop_vnc.sh

#!/bin/bash
sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -deactivate -configure -access -off

Now we’ll want to run the stop_vnc command on boot, so open launch control and add a new Global Daemon job.

Now load the job and start it. This should kill your VNC connection. To launch a new one, just log in over SSH and run the start_vnc.sh command.

Conclusion

Now, after a long setup process, we have a very nice CI server all set up and ready to go.

With these goals accomplished you may now do what you came here to do: crank out code!