[Thanks to @Swift, co-founder & CEO of Major League Hacking, for sharing this excellent Clarifai iOS SDK tutorial he created.]
*** Please note that support for our mobile SDK has been depreciated ***
Everyday Carry
An iOS App That Sees Your Stuff
We all have those possessions that go everywhere with us. For some it’s a notebook, a satchel bag, or a laptop. For many, it might be all three and more. These everyday items are like the background actors of our daily lives. We might completely forget about them as they blend in. Thanks to computer vision technology, we can pick them out in our photos.
Let’s elevate these everyday items to give them the attention they deserve in this tutorial on using the Clarifai Apple iOS SDK to predict the content of an image.
This tutorial will guide you through the process of integrating the Clarifai SDK within your application. All code examples will be given in the Swift programming language.
The Clarifai iOS SDK provides an easy way to add image recognition to your app. In this tutorial, you’ll create a sample app that predicts the contents of “Everyday Carry,” the objects people carry around every day. This tutorial demonstrates how you could build an app that parses these images and outputs what’s in the photos.
Set up Your XCode Project
Important note: Before doing any of the following steps please ensure that you have git-lfs installed on your system. It can be installed via Homebrew or Macports. Follow the link for more information on installing git-lfs https://git-lfs.github.com
1. Before doing anything sign up for a free account on the Clarifai website to get an API Key that is necessary to run your application.
2. Clone the Clarifai iOS SDK repository from GitHub using the following command:
- git clone https://github.com/Clarifai/clarifai-apple-sdk.git
3. Once you have cloned the repository, create a directory within your project’s root directory titled Clarifai then move the Clarifai-Apple-SDK.framework to the directory that you just created within your project’s root directory.
4. From the cloned repository move the setup_framework.sh into the Clarifai directory that you created with your project’s root directory.
5. Now that you have all of the necessary file open up your XCode project. From your XCode project go to the project configurations, General tab, and click the + button under the Embedded Binaries section. Navigate to the directory where you cloned the repository and select the Clarifai-Apple-SDK.framework
6. Include the following required dependencies to Linked Frameworks and Libraries:
- Accelerate.framework
- CoreGraphics.framework
- Foundation.framework
- libc++
- libsqlite3
- libz
- UIKit.framework
7. Create a new Run Script Build Phase (XCode > Editor > Add Build Phase > Add Run Script Build Phase). In the execution line enter the line from setup_framework.sh:
Note: Make sure the position of the new Run Script is after Compile Sources and before Link Binary With Libraries. If needed drag and drop it to the right position.
Connect to the Clarifai SDK
The basic usage of the SDK is pretty simple. You provide an image to the SDK and it outputs a list of predictions about what is contained within the image as well as a number representing the degree of confidence. Let’s first start by looking at how to start up the SDK.
Initialize the SDK
To start up the SDK you call the start function with the API Key from the account that you created with Clarifai. Reference startSDK.swift.
Choose Image and Predict Content
Note about prediction models: By default, the SDK uses the General Model to determine its predictions. This is the most comprehensive and is a great all-purpose solution; however, if you have more specific needs Clarifai offers more focused models. More information about the different models that Clarifai offers can be found here. Choose an image and initialize a DataAsset object with the image. The DataAsset object contains the image as well as the metadata associated with the image.
From the DataAsset object create an input object. An input object contains the data asset, temporal information, and is fundamental to be used by models to train on or predict
Now that we have our inputs, we can use the predict function to predict the content of our image.
Resources and Next Steps
Now that you’ve seen one example of Clarifai’s computer vision, you might want to explore more. Perhaps you want to find cats in Instagram photos, pick out the food in your photos, or catalog your personal inventory. A good place to start is Clarifai’s Apple iOS GitHub repository or the Clarifai Developer Guide.
