This post is a detailed walkthrough of how to perform a simple image prediction using Clarifai's python API. Everything contained here is in the accompanying Jupyter notebook hosted on Github at the following links:
Let's get started! If you want to jump straight into the notebook, the only change you need to make is to add your own PAT. Once that is in place you can run the whole thing to get your predictions.
Installing the Clarifai gRPC client and dependencies
First, we need to install Clarifai's gRPC software and dependencies. Clarifai's current version requires protobuf 3.20.3 as a minimum, but we can also upgrade protobuf to the latest version.
Protobuf, or "Protocol Buffers," is a free and open-source cross-platform data format used to serialize structured data. It's used by Clarifai's gRPC (a high performance remote procedure call framework) to communicate with Clarifai's servers.
Next we need install all the other dependencies used in the notebook. We'll install:
os to access file system commands like listing the contends of a directory
BytesIO to steam files as a stream of bytes to be processed
skimage we'll run predict on the example images in skimage's data module
matplotlib.pyplot to display the results as images
PIL "pillow," an "image processing library" used to send images to Clarifai
numpy to create a range of numbers
inline as a "magic function" in python to have the resulting generated images stored in the notebook.
Initializing the Clarifai gRPC-based Client
In this step we just need to import all the relevant parts of the Clarifai packages and instantiate them to eventually create the connection with Clarifai's servers.
Setup the Authorization
Clarifai uses personal access tokens -PATs - to identify you as a user and let you access services.
We fill in the placeholder for PAT here
To create or find a PAT, in Clarifai Community, click on the circular icon for your username in the top right, select “Security”, and then create a PAT or copy an existing one if you’ve already created one. You can follow the below screenshots to see where to find and create PATs.
Now that we've setup the PAT, we also need to specify the application and account that owns the model we're going to use. Since we're choosing a model from Clarifai / Main, we use
Collecting Inputs (Images in this example)
Since we're running predict on multiple images in this example, we iterate through all the files in the skimage data module, then only keep the ones we have in the "descriptions" list. That's it! You can modify this notebook to pull in your own images and classes to try it yourself. Please send any questions our way; our community Slack channel can be found here
Preparing Inputs (Images in this example)
In this step we convert our images into a byte stream using BytesIO and the userDataObject called resources_pb2.
Clarifai's General Model is a visual classifier for identifying a variety of concepts, common objects, etc. It is a great all-purpose solution for most visual recognition needs with industry-leading performance. It works on both images and videos.
Here we plot diagrams that show the identified concepts compared to the probability of them appearing in the image.