Getting Started
The Clarifai API offers image and video recognition as a service. Whether you have one image or billions, you are only steps away from using artificial intelligence to recognize your visual content.
The API is built around a simple idea. You send inputs (an image or video) to the service and it returns predictions.
The type of prediction is based on what model you run the input through. For example, if you run your input through the 'food' model, the predictions it returns will contain concepts that the 'food' model knows about. If you run your input through the 'color' model, it will return predictions about the dominant colors in your image.
Before you get started, if you haven't created an account and received your free API key, please do so before proceeding with this guide. You can begin making API calls for free, a credit card is not required.
Please note that your account will be limited to 100 API calls until you verify your email address. After verification, you will receive the full amount of API calls under your plan.
All API access is over HTTPS, and accessed via the https://api.clarifai.com domain. The relative path prefix /v2/ indicates that we are currently using version 2 of the API.
In the below examples, we use single brackets {variable} to indicate that this is a variable you should replace with a real value.
API Clients
We recommend using a client library to access the API. We offer official clients in various languages. View our API Clients.
For REST documentation please see the cURL examples.
Client Installation Instructions
// The JavaScript client works in both Node.js and the browser.
// Install the client from NPM
npm install clarifai
// Require the client
const Clarifai = require('clarifai');
// initialize with your api key. This will also work in your browser via http://browserify.org/
const app = new Clarifai.App({
apiKey: 'YOUR_API_KEY'
});
// You can also use the SDK by adding this script to your HTML:
<script type="text/javascript" src="https://sdk.clarifai.com/js/clarifai-latest.js"></script>
# Pip install the client:
# pip install clarifai
from clarifai.rest import ClarifaiApp
# Create your API key in your account's `Manage your API keys` page:
# https://clarifai.com/developer/account/keys
app = ClarifaiApp(api_key='YOUR_API_KEY')
# You can also create an environment variable called `CLARIFAI_API_KEY`
# and set its value to your API key.
# In this case, the construction of the object requires no `api_key` argument.
app = ClarifaiApp()
// Our API client is hosted on jCenter, Maven Central, and JitPack.
///////////////////////////////////////////////////////////////////////////////
// Installation - via Gradle (recommended)
///////////////////////////////////////////////////////////////////////////////
// Add the client to your dependencies:
dependencies {
compile 'com.clarifai.clarifai-api2:core:2.3.0'
}
// Make sure you have the Maven Central Repository in your Gradle File
repositories {
mavenCentral()
}
///////////////////////////////////////////////////////////////////////////////
// Installation - via Maven
///////////////////////////////////////////////////////////////////////////////
/*
<!-- Add the client to your dependencies: -->
<dependency>
<groupId>com.clarifai.clarifai-api2</groupId>
<artifactId>core</artifactId>
<version>2.3.0</version>
</dependency>
*/
///////////////////////////////////////////////////////////////////////////////
// Initialize client
///////////////////////////////////////////////////////////////////////////////
new ClarifaiBuilder("YOUR_API_KEY")
.client(new OkHttpClient()) // OPTIONAL. Allows customization of OkHttp by the user
.buildSync(); // or use .build() to get a Future<ClarifaiClient>
// Within Visual Studio IDE:
Install-Package Clarifai
// With the dotnet command line tool:
dotnet add package Clarifai
///////////////////////////////////////////////////////////////////////////////
// Initialize client
///////////////////////////////////////////////////////////////////////////////
using System.Threading.Tasks;
using Clarifai.API;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
}
}
}
// Note: See the StackOverflow answer below on how to use async/await from the Main method in C# < 7.1:
// https://stackoverflow.com/a/24601591
// Set `<LangVersion>7.1</LangVersion>` in your .csproj under `<PropertyGroup/>`
// Installation via CocoaPods - https://cocoapods.org
// 1. Create a new XCode project, or use a current one.
// 2. Add the following to your Podfile:
// pod 'Clarifai'
// 3. Install dependencies and generate workspace.
// pod install
// 4. Open the workspace in Xcode
// open YOUR_PROJECT_NAME.xcworkspace
// 5. You are now able to import ClarifaiApp.h and any other classes you need!
#import ClarifaiApp.h
// Note: if you are using Swift in your project, make sure to include use_frameworks! in your Podfile. Then import Clarifai as a module.
import Clarifai
// Install cURL: https://curl.haxx.se/download.html
- JS
- Py
- Java
- C#
- ObjC
- cURL
Glossary
Our Glossary page will be continously updated with our domain specific terminolgy. You can access it here.
Status Codes
All of our API endpoints return back a status code and description. To see a full list of our API status codes and their descriptions you can visit thefollowing page.
Explorer
In addition to the API, Clarifai now offers a web site that allows you to preview your Clarifai applications. You can view all the inputs you have added, perform searches and train new models.
You can access the Explorer here
For a step-by-step guide on building and training a custom model using Explorer, checkout this walkthrough.
Authentication
Note: API Keys have replaced Access Tokens/Client ID & Secret
API Keys will now be used to authorize your Clarifai applications. You can go to API Keys page to create a new key. This change will not break existing applications - Access Tokens will be supported until late 2017. You can find more information on our blog.
Authentication to the API is handled through API Keys. You can limit the scope of an API Key, which enables the key to perform very specific operations on a given app, keeping your app secure.
To create an API Key, you can head over to the Keys Dashboard, and specify a) which app you would like to access, and b) what scopes you want the key to have.
Authorize Applications
Select the application that you want to authorize using this key. An API Key cannot be used across multiple apps.
Key Description
Use this field to create a descriptive name for your API Key. As you start to create a lot of keys, you would need this field to keep track of what key is for what purpose. Note: this field is not used for any identification purposes.
Scopes
You will see a list of scopes that you can add on to your API Key for the app that was selected. We recommend generating different keys for very specific purposes - this is the best approach to keep your app secure from malicious attacks. Do NOT share your API Key with other users. Here's a list of scopes that your key can have:
Note: API Keys do not expire. In case your API Key gets compromised, you should delete that key, and create a new one with the same scopes.
Authorize API Calls
After creating your API Key, you are ready to make API calls. If you are using a client, authentication will be handled for you. If you are using the REST API, you will need to add the Authorization header as described in the cURL example.
const app = new Clarifai.App({apiKey: 'YOUR_API_KEY'});
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
new ClarifaiBuilder("YOUR_API_KEY").buildSync();
using System.Threading.Tasks;
using Clarifai.API;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
}
}
}
ClarifaiApp *app = [[ClarifaiApp alloc] initWithApiKey:@"YOUR_API_KEY"];
curl -X POST \
-H 'Authorization: Key YOUR_API_KEY' \
-H "Content-Type: application/json" \
-d '- JS
- Py
- Java
- C#
- ObjC
- cURL
If the API Key does not have the required scope(s), you will receive one of the following responses:
- Rejected Request: if a large portion of the response requires a scope that is missing
- Redacted Response: if majority of the response has the required scope(s), a small portion of the response will be redacted to not reveal unwarranted information
Predict
The Predict API analyzes your images or videos and tells you what's inside of them.
The API will return a list of concepts with corresponding probabilities of how likely it is these concepts are contained within the image or video.
When you make a prediction through the API, you tell it what model to use. This can be one Clarifai's pre-built models or a custom one trained by you. A model contains a group of concepts. A model will only 'see' the concepts it contains. You can also specify more parameters for predictions.
We recommend specifying the version id parameter in your predict calls. If no version id is specified, predictions will occur on the most recent version of the model. More information can be found in the Advanced Predictions section.
Images
Via URL
To get predictions for an input, you need to supply an image and the model you'd like to get predictions from. You can supply an image either with a publicly accessible URL or by directly sending bytes. You can send up to 128 images in one API call. You specify the model you'd like to use with the {model-id} parameter.
Below is an example of how you would send image URLs and receive back predictions from the general model.
You can learn all about the different public models available later in the guide.
app.models.predict({Clarifai.GENERAL_MODEL}, "https://samples.clarifai.com/metro-north.jpg").then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
from clarifai.rest import Image as ClImage
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('general-v1.3')
image = ClImage(url='https://samples.clarifai.com/metro-north.jpg')
model.predict([image])
client.getDefaultModels().generalModel().predict()
.withInputs(ClarifaiInput.forImage("https://samples.clarifai.com/metro-north.jpg"))
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Inputs;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.PublicModels.GeneralModel.Predict(
new ClarifaiURLImage("https://samples.clarifai.com/metro-north.jpg"))
.ExecuteAsync();
}
}
}
ClarifaiImage *image = [[ClarifaiImage alloc] initWithURL:@"https://samples.clarifai.com/metro-north.jpg"];
[_app getModelByName:@"general-v1.3" completion:^(ClarifaiModel *model, NSError *error) {
[model predictOnImages:@[image]
completion:^(NSArray<ClarifaiSearchResult *> *outputs, NSError *error) {
NSLog(@"outputs: %@", outputs);
}];
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
}
}
]
}'\
https://api.clarifai.com/v2/models/aaa03c23b3724a16a56b629203edc62c/outputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Response
{
"status": {
"code": 10000,
"description": "Ok"
},
"outputs": [
{
"id": "ea68cac87c304b28a8046557062f34a0",
"status": {
"code": 10000,
"description": "Ok"
},
"created_at": "2016-11-22T16:50:25Z",
"model": {
"name": "general-v1.3",
"id": "aaa03c23b3724a16a56b629203edc62c",
"created_at": "2016-03-09T17:11:39Z",
"app_id": null,
"output_info": {
"message": "Show output_info with: GET /models/{model_id}/output_info",
"type": "concept"
},
"model_version": {
"id": "aa9ca48295b37401f8af92ad1af0d91d",
"created_at": "2016-07-13T01:19:12Z",
"status": {
"code": 21100,
"description": "Model trained successfully"
}
}
},
"input": {
"id": "ea68cac87c304b28a8046557062f34a0",
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
}
},
"data": {
"concepts": [
{
"id": "ai_HLmqFqBf",
"name": "train",
"app_id": null,
"value": 0.9989112
},
{
"id": "ai_fvlBqXZR",
"name": "railway",
"app_id": null,
"value": 0.9975532
},
{
"id": "ai_Xxjc3MhT",
"name": "transportation system",
"app_id": null,
"value": 0.9959158
},
{
"id": "ai_6kTjGfF6",
"name": "station",
"app_id": null,
"value": 0.992573
},
{
"id": "ai_RRXLczch",
"name": "locomotive",
"app_id": null,
"value": 0.992556
},
{
"id": "ai_VRmbGVWh",
"name": "travel",
"app_id": null,
"value": 0.98789215
},
{
"id": "ai_SHNDcmJ3",
"name": "subway system",
"app_id": null,
"value": 0.9816359
},
{
"id": "ai_jlb9q33b",
"name": "commuter",
"app_id": null,
"value": 0.9712483
},
{
"id": "ai_46lGZ4Gm",
"name": "railroad track",
"app_id": null,
"value": 0.9690325
},
{
"id": "ai_tr0MBp64",
"name": "traffic",
"app_id": null,
"value": 0.9687052
},
{
"id": "ai_l4WckcJN",
"name": "blur",
"app_id": null,
"value": 0.9667078
},
{
"id": "ai_2gkfMDsM",
"name": "platform",
"app_id": null,
"value": 0.9624243
},
{
"id": "ai_CpFBRWzD",
"name": "urban",
"app_id": null,
"value": 0.960752
},
{
"id": "ai_786Zr311",
"name": "no person",
"app_id": null,
"value": 0.95864904
},
{
"id": "ai_6lhccv44",
"name": "business",
"app_id": null,
"value": 0.95720303
},
{
"id": "ai_971KsJkn",
"name": "track",
"app_id": null,
"value": 0.9494642
},
{
"id": "ai_WBQfVV0p",
"name": "city",
"app_id": null,
"value": 0.94089437
},
{
"id": "ai_dSCKh8xv",
"name": "fast",
"app_id": null,
"value": 0.9399334
},
{
"id": "ai_TZ3C79C6",
"name": "road",
"app_id": null,
"value": 0.93121606
},
{
"id": "ai_VSVscs9k",
"name": "terminal",
"app_id": null,
"value": 0.9230834
}
]
}
}
]
}
Via Bytes
Below is an example of how you would send the bytes of an image and receive back predictions from the general model.
app.models.predict(Clarifai.GENERAL_MODEL, {base64: "G7p3m95uAl..."}).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
from clarifai.rest import Image as ClImage
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('general-v1.3')
image = ClImage(file_obj=open('/home/user/image.jpeg', 'rb'))
model.predict([image])
client.getDefaultModels().generalModel().predict()
.withInputs(ClarifaiInput.forImage(new File("/home/user/image.jpeg")))
.executeSync();
using System.IO;
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Inputs;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.PublicModels.GeneralModel.Predict(
new ClarifaiFileImage(File.ReadAllBytes("/home/user/image.jpeg")))
.ExecuteAsync();
}
}
}
UIImage *image = [UIImage imageNamed:@"dress.jpg"];
ClarifaiImage *clarifaiImage = [[ClarifaiImage alloc] initWithImage:image];
[_app getModelByName:@"general-v1.3" completion:^(ClarifaiModel *model, NSError *error) {
[model predictOnImages:@[clarifaiImage]
completion:^(NSArray<ClarifaiSearchResult *> *outputs, NSError *error) {
NSLog(@"outputs: %@", outputs);
}];
}];
// Smaller files (195 KB or less)
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"data": {
"image": {
"base64": "'"$(base64 /home/user/image.jpeg)"'"
}
}
}
]
}'\
https://api.clarifai.com/v2/models/aaa03c23b3724a16a56b629203edc62c/outputs
// Larger Files (Greater than 195 KB)
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d @- https://api.clarifai.com/v2/models/aaa03c23b3724a16a56b629203edc62c/outputs << FILEIN
{
"inputs": [
{
"data": {
"image": {
"base64": "$(base64 /home/user/image.png)"
}
}
}
]
}
FILEIN- JS
- Py
- Java
- C#
- ObjC
- cURL
Response
{
"status": {
"code": 10000,
"description": "Ok"
},
"outputs": [
{
"id": "e1cf385843b94c6791bbd9f2654db5c0",
"status": {
"code": 10000,
"description": "Ok"
},
"created_at": "2016-11-22T16:59:23Z",
"model": {
"name": "general-v1.3",
"id": "aaa03c23b3724a16a56b629203edc62c",
"created_at": "2016-03-09T17:11:39Z",
"app_id": null,
"output_info": {
"message": "Show output_info with: GET /models/{model_id}/output_info",
"type": "concept"
},
"model_version": {
"id": "aa9ca48295b37401f8af92ad1af0d91d",
"created_at": "2016-07-13T01:19:12Z",
"status": {
"code": 21100,
"description": "Model trained successfully"
}
}
},
"input": {
"id": "e1cf385843b94c6791bbd9f2654db5c0",
"data": {
"image": {
"url": "https://s3.amazonaws.com/clarifai-api/img/prod/b749af061d564b829fb816215f6dc832/e11c81745d6d42a78ef712236023df1c.jpeg"
}
}
},
"data": {
"concepts": [
{
"id": "ai_l4WckcJN",
"name": "blur",
"app_id": null,
"value": 0.9973569
},
{
"id": "ai_786Zr311",
"name": "no person",
"app_id": null,
"value": 0.98865616
},
{
"id": "ai_JBPqff8z",
"name": "art",
"app_id": null,
"value": 0.986006
},
{
"id": "ai_5rD7vW4j",
"name": "wallpaper",
"app_id": null,
"value": 0.9722556
},
{
"id": "ai_sTjX6dqC",
"name": "abstract",
"app_id": null,
"value": 0.96476805
},
{
"id": "ai_Dm5GLXnB",
"name": "illustration",
"app_id": null,
"value": 0.922542
},
{
"id": "ai_5xjvC0Tj",
"name": "background",
"app_id": null,
"value": 0.8775655
},
{
"id": "ai_tBcWlsCp",
"name": "nature",
"app_id": null,
"value": 0.87474406
},
{
"id": "ai_rJGvwlP0",
"name": "insubstantial",
"app_id": null,
"value": 0.8196385
},
{
"id": "ai_2Bh4VMrb",
"name": "artistic",
"app_id": null,
"value": 0.8142488
},
{
"id": "ai_mKzmkKDG",
"name": "Christmas",
"app_id": null,
"value": 0.7996079
},
{
"id": "ai_RQccV41p",
"name": "woman",
"app_id": null,
"value": 0.7955615
},
{
"id": "ai_20SCBBZ0",
"name": "vector",
"app_id": null,
"value": 0.7775099
},
{
"id": "ai_4sJLn6nX",
"name": "dark",
"app_id": null,
"value": 0.7715479
},
{
"id": "ai_5Kp5FMJw",
"name": "still life",
"app_id": null,
"value": 0.7657637
},
{
"id": "ai_LM64MDHs",
"name": "shining",
"app_id": null,
"value": 0.7542407
},
{
"id": "ai_swtdphX8",
"name": "love",
"app_id": null,
"value": 0.74926054
},
{
"id": "ai_h45ZTxZl",
"name": "square",
"app_id": null,
"value": 0.7449074
},
{
"id": "ai_cMfj16kJ",
"name": "design",
"app_id": null,
"value": 0.73926914
},
{
"id": "ai_LxrzLJmf",
"name": "bright",
"app_id": null,
"value": 0.73790145
}
]
}
}
]
}
Videos
With a video input, the Predict API response will return a list of predicted concepts for every frame of a video. Video is processed at 1 frame per second. This means you will receive a list of concepts for every second of your video.
You can run Predict on your video using a select number of public models. The models that are currently supported are: Apparel, Food, General, NSFW, Travel, and Wedding. You make an API call by providing the {model-id} parameter and your data parameter is video instead of image.
Video Limits
The Predict API has limits to the length and size it can support. A video, uploaded through URL, can be anywhere up to 80MB in size or 10mins in length. When a video is sent through by bytes, the Predict API can support 10MB in size.
If your video exceeds the limits, please follow our tutorial on how to break up a large video into smaller components, and send those into the Video API. Otherwise, the processing will time out and you will receive an error response.
Via URL
Below is an example of how you would send video URLs and receive back predictions from the general model.
clarifai.models.predict(Clarifai.GENERAL_MODEL, 'https://samples.clarifai.com/beer.mp4', {video: true})
.then(log)
.catch(log);from clarifai.rest import ClarifaiApp
from clarifai.rest import Video as ClVideo
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('general-v1.3')
video = ClVideo(url='https://samples.clarifai.com/beer.mp4')
model.predict([video])ClarifaiResponse<List<ClarifaiOutput<Frame>>> frames = client.getDefaultModels().generalVideoModel().predict()
.withInputs(ClarifaiInput.forVideo("https://samples.clarifai.com/beer.mp4"))
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Inputs;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.PublicModels.GeneralVideoModel.Predict(
new ClarifaiURLVideo("https://samples.clarifai.com/beer.mp4"))
.ExecuteAsync();
}
}
}
Objective-C client details coming soon
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"data": {
"video": {
"url": "https://samples.clarifai.com/beer.mp4"
}
}
}
]
}'\
https://api.clarifai.com/v2/models/aaa03c23b3724a16a56b629203edc62c/outputs- JS
- Py
- Java
- C#
- ObjC
- cURL
Response{
"status": {
"code": 10000,
"description": "Ok"
},
"outputs": [
{
"id": "d8234da5d1f04ca8a2e13e34d51f9b85",
"status": {
"code": 10000,
"description": "Ok"
},
"created_at": "2017-06-28T14:58:41.835370141Z",
"model": {
"id": "aaa03c23b3724a16a56b629203edc62c",
"name": "general-v1.3",
"created_at": "2016-03-09T17:11:39.608845Z",
"app_id": "main",
"output_info": {
"message": "Show output_info with: GET /models/{model_id}/output_info",
"type": "concept",
"type_ext": "concept"
},
"model_version": {
"id": "aa9ca48295b37401f8af92ad1af0d91d",
"created_at": "2016-07-13T01:19:12.147644Z",
"status": {
"code": 21100,
"description": "Model trained successfully"
}
}
},
"input": {
"id": "f0fc1a005f124d389da4d80823a3125b",
"data": {
"video": {
"url": "https://samples.clarifai.com/beer.mp4"
}
}
},
"data": {
"frames": [
{
"frame_info": {
"index": 0,
"time": 0
},
"data": {
"concepts": [
{
"id": "ai_zJx6RbxW",
"name": "drink",
"value": 0.98658466,
"app_id": "main"
},
{
"id": "ai_mCpQg89c",
"name": "glass",
"value": 0.97975093,
"app_id": "main"
},
{
"id": "ai_drK6ClJR",
"name": "alcohol",
"value": 0.9783862,
"app_id": "main"
},
{
"id": "ai_8XGJjH7R",
"name": "foam",
"value": 0.97157896,
"app_id": "main"
},
{
"id": "ai_TBlp0Pt3",
"name": "beer",
"value": 0.969543,
"app_id": "main"
},
{
"id": "ai_SsmKLB4z",
"name": "bar",
"value": 0.96628696,
"app_id": "main"
},
{
"id": "ai_5VHsZr8N",
"name": "liquid",
"value": 0.95581007,
"app_id": "main"
},
{
"id": "ai_Lq00FggW",
"name": "desktop",
"value": 0.92861253,
"app_id": "main"
},
{
"id": "ai_7vR9zv7l",
"name": "bubble",
"value": 0.9082134,
"app_id": "main"
},
{
"id": "ai_B3MXt5Ng",
"name": "refreshment",
"value": 0.9020835,
"app_id": "main"
},
{
"id": "ai_7Xg5SQRW",
"name": "luxury",
"value": 0.8990605,
"app_id": "main"
},
{
"id": "ai_786Zr311",
"name": "no person",
"value": 0.89708906,
"app_id": "main"
},
{
"id": "ai_3R5pJ6hB",
"name": "lager",
"value": 0.8938055,
"app_id": "main"
},
{
"id": "ai_7qwGxLch",
"name": "gold",
"value": 0.8892093,
"app_id": "main"
},
{
"id": "ai_wmbvr5TG",
"name": "celebration",
"value": 0.88606626,
"app_id": "main"
},
{
"id": "ai_4lvjn8qv",
"name": "closeup",
"value": 0.881963,
"app_id": "main"
},
{
"id": "ai_pkvDRSJ1",
"name": "mug",
"value": 0.8674431,
"app_id": "main"
},
{
"id": "ai_12dz73B9",
"name": "bottle",
"value": 0.86288416,
"app_id": "main"
},
{
"id": "ai_zFnPQdgB",
"name": "wood",
"value": 0.86252767,
"app_id": "main"
},
{
"id": "ai_8LWlDfFD",
"name": "table",
"value": 0.86069393,
"app_id": "main"
}
]
}
},
{
"frame_info": {
"index": 1,
"time": 1000
},
"data": {
"concepts": [
{
"id": "ai_zJx6RbxW",
"name": "drink",
"value": 0.98658466,
"app_id": "main"
},
{
"id": "ai_mCpQg89c",
"name": "glass",
"value": 0.97975093,
"app_id": "main"
},
{
"id": "ai_drK6ClJR",
"name": "alcohol",
"value": 0.9783862,
"app_id": "main"
},
{
"id": "ai_8XGJjH7R",
"name": "foam",
"value": 0.97157896,
"app_id": "main"
},
{
"id": "ai_TBlp0Pt3",
"name": "beer",
"value": 0.969543,
"app_id": "main"
},
{
"id": "ai_SsmKLB4z",
"name": "bar",
"value": 0.96628696,
"app_id": "main"
},
{
"id": "ai_5VHsZr8N",
"name": "liquid",
"value": 0.95581007,
"app_id": "main"
},
{
"id": "ai_7vR9zv7l",
"name": "bubble",
"value": 0.9082134,
"app_id": "main"
},
{
"id": "ai_B3MXt5Ng",
"name": "refreshment",
"value": 0.9020835,
"app_id": "main"
},
{
"id": "ai_786Zr311",
"name": "no person",
"value": 0.89708906,
"app_id": "main"
},
{
"id": "ai_3R5pJ6hB",
"name": "lager",
"value": 0.8938055,
"app_id": "main"
},
{
"id": "ai_7qwGxLch",
"name": "gold",
"value": 0.8834515,
"app_id": "main"
},
{
"id": "ai_pkvDRSJ1",
"name": "mug",
"value": 0.8674431,
"app_id": "main"
},
{
"id": "ai_b01mhdxB",
"name": "party",
"value": 0.8603341,
"app_id": "main"
},
{
"id": "ai_XNmzgDnF",
"name": "pub",
"value": 0.85809004,
"app_id": "main"
},
{
"id": "ai_2gmKZLxp",
"name": "cold",
"value": 0.85319245,
"app_id": "main"
},
{
"id": "ai_Lq00FggW",
"name": "desktop",
"value": 0.8506696,
"app_id": "main"
},
{
"id": "ai_54zxXFGL",
"name": "full",
"value": 0.84634554,
"app_id": "main"
},
{
"id": "ai_zFnPQdgB",
"name": "wood",
"value": 0.8446485,
"app_id": "main"
},
{
"id": "ai_wmbvr5TG",
"name": "celebration",
"value": 0.8383831,
"app_id": "main"
}
]
}
},
{
"frame_info": {
"index": 2,
"time": 2000
},
"data": {
"concepts": [
{
"id": "ai_zJx6RbxW",
"name": "drink",
"value": 0.9856042,
"app_id": "main"
},
{
"id": "ai_mCpQg89c",
"name": "glass",
"value": 0.97975093,
"app_id": "main"
},
{
"id": "ai_8XGJjH7R",
"name": "foam",
"value": 0.9755833,
"app_id": "main"
},
{
"id": "ai_786Zr311",
"name": "no person",
"value": 0.9733174,
"app_id": "main"
},
{
"id": "ai_TBlp0Pt3",
"name": "beer",
"value": 0.969543,
"app_id": "main"
},
{
"id": "ai_SsmKLB4z",
"name": "bar",
"value": 0.94170487,
"app_id": "main"
},
{
"id": "ai_5VHsZr8N",
"name": "liquid",
"value": 0.92778283,
"app_id": "main"
},
{
"id": "ai_2gmKZLxp",
"name": "cold",
"value": 0.9227257,
"app_id": "main"
},
{
"id": "ai_3PlgVmlN",
"name": "food",
"value": 0.9179274,
"app_id": "main"
},
{
"id": "ai_drK6ClJR",
"name": "alcohol",
"value": 0.90887475,
"app_id": "main"
},
{
"id": "ai_B3MXt5Ng",
"name": "refreshment",
"value": 0.9045203,
"app_id": "main"
},
{
"id": "ai_3R5pJ6hB",
"name": "lager",
"value": 0.8938055,
"app_id": "main"
},
{
"id": "ai_WbwL0pPL",
"name": "breakfast",
"value": 0.87420183,
"app_id": "main"
},
{
"id": "ai_54zxXFGL",
"name": "full",
"value": 0.8699659,
"app_id": "main"
},
{
"id": "ai_pkvDRSJ1",
"name": "mug",
"value": 0.8674431,
"app_id": "main"
},
{
"id": "ai_XNmzgDnF",
"name": "pub",
"value": 0.85809004,
"app_id": "main"
},
{
"id": "ai_Lq00FggW",
"name": "desktop",
"value": 0.8506696,
"app_id": "main"
},
{
"id": "ai_zFnPQdgB",
"name": "wood",
"value": 0.8446485,
"app_id": "main"
},
{
"id": "ai_qNxqNBWN",
"name": "cream",
"value": 0.844169,
"app_id": "main"
},
{
"id": "ai_7D0mdp1W",
"name": "delicious",
"value": 0.8397074,
"app_id": "main"
}
]
}
},
{
"frame_info": {
"index": 3,
"time": 3000
},
"data": {
"concepts": [
{
"id": "ai_8XGJjH7R",
"name": "foam",
"value": 0.996614,
"app_id": "main"
},
{
"id": "ai_zJx6RbxW",
"name": "drink",
"value": 0.9794438,
"app_id": "main"
},
{
"id": "ai_786Zr311",
"name": "no person",
"value": 0.9733174,
"app_id": "main"
},
{
"id": "ai_TBlp0Pt3",
"name": "beer",
"value": 0.9645849,
"app_id": "main"
},
{
"id": "ai_mCpQg89c",
"name": "glass",
"value": 0.94761443,
"app_id": "main"
},
{
"id": "ai_pkvDRSJ1",
"name": "mug",
"value": 0.92864025,
"app_id": "main"
},
{
"id": "ai_2gmKZLxp",
"name": "cold",
"value": 0.9227257,
"app_id": "main"
},
{
"id": "ai_5VHsZr8N",
"name": "liquid",
"value": 0.91797745,
"app_id": "main"
},
{
"id": "ai_3PlgVmlN",
"name": "food",
"value": 0.9179274,
"app_id": "main"
},
{
"id": "ai_WbwL0pPL",
"name": "breakfast",
"value": 0.904904,
"app_id": "main"
},
{
"id": "ai_B3MXt5Ng",
"name": "refreshment",
"value": 0.9045203,
"app_id": "main"
},
{
"id": "ai_54zxXFGL",
"name": "full",
"value": 0.889248,
"app_id": "main"
},
{
"id": "ai_BrnHNkt0",
"name": "coffee",
"value": 0.8689867,
"app_id": "main"
},
{
"id": "ai_7D0mdp1W",
"name": "delicious",
"value": 0.86591685,
"app_id": "main"
},
{
"id": "ai_SsmKLB4z",
"name": "bar",
"value": 0.8546975,
"app_id": "main"
},
{
"id": "ai_mZ2tl6cW",
"name": "health",
"value": 0.8544879,
"app_id": "main"
},
{
"id": "ai_cHsR7RS8",
"name": "milk",
"value": 0.852397,
"app_id": "main"
},
{
"id": "ai_zFnPQdgB",
"name": "wood",
"value": 0.8446485,
"app_id": "main"
},
{
"id": "ai_qNxqNBWN",
"name": "cream",
"value": 0.844169,
"app_id": "main"
},
{
"id": "ai_8LWlDfFD",
"name": "table",
"value": 0.837438,
"app_id": "main"
}
]
}
},
{
"frame_info": {
"index": 4,
"time": 4000
},
"data": {
"concepts": [
{
"id": "ai_8XGJjH7R",
"name": "foam",
"value": 0.996614,
"app_id": "main"
},
{
"id": "ai_786Zr311",
"name": "no person",
"value": 0.9748836,
"app_id": "main"
},
{
"id": "ai_TBlp0Pt3",
"name": "beer",
"value": 0.9645849,
"app_id": "main"
},
{
"id": "ai_zJx6RbxW",
"name": "drink",
"value": 0.96217895,
"app_id": "main"
},
{
"id": "ai_3PlgVmlN",
"name": "food",
"value": 0.9179274,
"app_id": "main"
},
{
"id": "ai_WbwL0pPL",
"name": "breakfast",
"value": 0.904904,
"app_id": "main"
},
{
"id": "ai_B3MXt5Ng",
"name": "refreshment",
"value": 0.9045203,
"app_id": "main"
},
{
"id": "ai_2gmKZLxp",
"name": "cold",
"value": 0.9030821,
"app_id": "main"
},
{
"id": "ai_mCpQg89c",
"name": "glass",
"value": 0.8983356,
"app_id": "main"
},
{
"id": "ai_pkvDRSJ1",
"name": "mug",
"value": 0.894987,
"app_id": "main"
},
{
"id": "ai_7D0mdp1W",
"name": "delicious",
"value": 0.894392,
"app_id": "main"
},
{
"id": "ai_54zxXFGL",
"name": "full",
"value": 0.889248,
"app_id": "main"
},
{
"id": "ai_mZ2tl6cW",
"name": "health",
"value": 0.8860091,
"app_id": "main"
},
{
"id": "ai_4sJLn6nX",
"name": "dark",
"value": 0.8837219,
"app_id": "main"
},
{
"id": "ai_zFnPQdgB",
"name": "wood",
"value": 0.8712319,
"app_id": "main"
},
{
"id": "ai_BrnHNkt0",
"name": "coffee",
"value": 0.8695712,
"app_id": "main"
},
{
"id": "ai_8LWlDfFD",
"name": "table",
"value": 0.8664293,
"app_id": "main"
},
{
"id": "ai_SsmKLB4z",
"name": "bar",
"value": 0.8546975,
"app_id": "main"
},
{
"id": "ai_cHsR7RS8",
"name": "milk",
"value": 0.852397,
"app_id": "main"
},
{
"id": "ai_qNxqNBWN",
"name": "cream",
"value": 0.844169,
"app_id": "main"
}
]
}
},
{
"frame_info": {
"index": 5,
"time": 5000
},
"data": {
"concepts": [
{
"id": "ai_8XGJjH7R",
"name": "foam",
"value": 0.997158,
"app_id": "main"
},
{
"id": "ai_TBlp0Pt3",
"name": "beer",
"value": 0.97719705,
"app_id": "main"
},
{
"id": "ai_786Zr311",
"name": "no person",
"value": 0.9748836,
"app_id": "main"
},
{
"id": "ai_zJx6RbxW",
"name": "drink",
"value": 0.96217895,
"app_id": "main"
},
{
"id": "ai_pkvDRSJ1",
"name": "mug",
"value": 0.9210669,
"app_id": "main"
},
{
"id": "ai_3PlgVmlN",
"name": "food",
"value": 0.9124408,
"app_id": "main"
},
{
"id": "ai_54zxXFGL",
"name": "full",
"value": 0.90667903,
"app_id": "main"
},
{
"id": "ai_WbwL0pPL",
"name": "breakfast",
"value": 0.904904,
"app_id": "main"
},
{
"id": "ai_B3MXt5Ng",
"name": "refreshment",
"value": 0.9046865,
"app_id": "main"
},
{
"id": "ai_2gmKZLxp",
"name": "cold",
"value": 0.9030821,
"app_id": "main"
},
{
"id": "ai_BrnHNkt0",
"name": "coffee",
"value": 0.90262496,
"app_id": "main"
},
{
"id": "ai_mCpQg89c",
"name": "glass",
"value": 0.8983356,
"app_id": "main"
},
{
"id": "ai_4sJLn6nX",
"name": "dark",
"value": 0.8947689,
"app_id": "main"
},
{
"id": "ai_7D0mdp1W",
"name": "delicious",
"value": 0.894392,
"app_id": "main"
},
{
"id": "ai_SsmKLB4z",
"name": "bar",
"value": 0.8852426,
"app_id": "main"
},
{
"id": "ai_3R5pJ6hB",
"name": "lager",
"value": 0.8789175,
"app_id": "main"
},
{
"id": "ai_8LWlDfFD",
"name": "table",
"value": 0.8762902,
"app_id": "main"
},
{
"id": "ai_zFnPQdgB",
"name": "wood",
"value": 0.87279737,
"app_id": "main"
},
{
"id": "ai_mZ2tl6cW",
"name": "health",
"value": 0.8544879,
"app_id": "main"
},
{
"id": "ai_cHsR7RS8",
"name": "milk",
"value": 0.849733,
"app_id": "main"
}
]
}
},
{
"frame_info": {
"index": 6,
"time": 6000
},
"data": {
"concepts": [
{
"id": "ai_8XGJjH7R",
"name": "foam",
"value": 0.99790645,
"app_id": "main"
},
{
"id": "ai_TBlp0Pt3",
"name": "beer",
"value": 0.97817445,
"app_id": "main"
},
{
"id": "ai_786Zr311",
"name": "no person",
"value": 0.97463626,
"app_id": "main"
},
{
"id": "ai_zJx6RbxW",
"name": "drink",
"value": 0.9659773,
"app_id": "main"
},
{
"id": "ai_pkvDRSJ1",
"name": "mug",
"value": 0.9273318,
"app_id": "main"
},
{
"id": "ai_4sJLn6nX",
"name": "dark",
"value": 0.9219268,
"app_id": "main"
},
{
"id": "ai_B3MXt5Ng",
"name": "refreshment",
"value": 0.9185593,
"app_id": "main"
},
{
"id": "ai_2gmKZLxp",
"name": "cold",
"value": 0.91295856,
"app_id": "main"
},
{
"id": "ai_3PlgVmlN",
"name": "food",
"value": 0.9119204,
"app_id": "main"
},
{
"id": "ai_54zxXFGL",
"name": "full",
"value": 0.91089505,
"app_id": "main"
},
{
"id": "ai_SsmKLB4z",
"name": "bar",
"value": 0.9056676,
"app_id": "main"
},
{
"id": "ai_BrnHNkt0",
"name": "coffee",
"value": 0.90262496,
"app_id": "main"
},
{
"id": "ai_mCpQg89c",
"name": "glass",
"value": 0.89882934,
"app_id": "main"
},
{
"id": "ai_WbwL0pPL",
"name": "breakfast",
"value": 0.8932399,
"app_id": "main"
},
{
"id": "ai_7D0mdp1W",
"name": "delicious",
"value": 0.892028,
"app_id": "main"
},
{
"id": "ai_zFnPQdgB",
"name": "wood",
"value": 0.88797945,
"app_id": "main"
},
{
"id": "ai_3R5pJ6hB",
"name": "lager",
"value": 0.88745904,
"app_id": "main"
},
{
"id": "ai_8LWlDfFD",
"name": "table",
"value": 0.87949455,
"app_id": "main"
},
{
"id": "ai_MmRdqDFp",
"name": "soap",
"value": 0.87376094,
"app_id": "main"
},
{
"id": "ai_5VHsZr8N",
"name": "liquid",
"value": 0.8715329,
"app_id": "main"
}
]
}
},
{
"frame_info": {
"index": 7,
"time": 7000
},
"data": {
"concepts": [
{
"id": "ai_8XGJjH7R",
"name": "foam",
"value": 0.99790645,
"app_id": "main"
},
{
"id": "ai_TBlp0Pt3",
"name": "beer",
"value": 0.97817445,
"app_id": "main"
},
{
"id": "ai_786Zr311",
"name": "no person",
"value": 0.97463626,
"app_id": "main"
},
{
"id": "ai_zJx6RbxW",
"name": "drink",
"value": 0.9659773,
"app_id": "main"
},
{
"id": "ai_pkvDRSJ1",
"name": "mug",
"value": 0.9273318,
"app_id": "main"
},
{
"id": "ai_4sJLn6nX",
"name": "dark",
"value": 0.9219268,
"app_id": "main"
},
{
"id": "ai_B3MXt5Ng",
"name": "refreshment",
"value": 0.9185593,
"app_id": "main"
},
{
"id": "ai_2gmKZLxp",
"name": "cold",
"value": 0.91295856,
"app_id": "main"
},
{
"id": "ai_3PlgVmlN",
"name": "food",
"value": 0.9119204,
"app_id": "main"
},
{
"id": "ai_54zxXFGL",
"name": "full",
"value": 0.91089505,
"app_id": "main"
},
{
"id": "ai_SsmKLB4z",
"name": "bar",
"value": 0.9056676,
"app_id": "main"
},
{
"id": "ai_BrnHNkt0",
"name": "coffee",
"value": 0.90262496,
"app_id": "main"
},
{
"id": "ai_mCpQg89c",
"name": "glass",
"value": 0.89882934,
"app_id": "main"
},
{
"id": "ai_WbwL0pPL",
"name": "breakfast",
"value": 0.8932399,
"app_id": "main"
},
{
"id": "ai_7D0mdp1W",
"name": "delicious",
"value": 0.892028,
"app_id": "main"
},
{
"id": "ai_zFnPQdgB",
"name": "wood",
"value": 0.8913312,
"app_id": "main"
},
{
"id": "ai_3R5pJ6hB",
"name": "lager",
"value": 0.88745904,
"app_id": "main"
},
{
"id": "ai_8LWlDfFD",
"name": "table",
"value": 0.87949455,
"app_id": "main"
},
{
"id": "ai_MmRdqDFp",
"name": "soap",
"value": 0.87376094,
"app_id": "main"
},
{
"id": "ai_5VHsZr8N",
"name": "liquid",
"value": 0.8715329,
"app_id": "main"
}
]
}
},
{
"frame_info": {
"index": 8,
"time": 8000
},
"data": {
"concepts": [
{
"id": "ai_8XGJjH7R",
"name": "foam",
"value": 0.99790645,
"app_id": "main"
},
{
"id": "ai_TBlp0Pt3",
"name": "beer",
"value": 0.97817445,
"app_id": "main"
},
{
"id": "ai_786Zr311",
"name": "no person",
"value": 0.97463626,
"app_id": "main"
},
{
"id": "ai_zJx6RbxW",
"name": "drink",
"value": 0.9659773,
"app_id": "main"
},
{
"id": "ai_4sJLn6nX",
"name": "dark",
"value": 0.9219268,
"app_id": "main"
},
{
"id": "ai_pkvDRSJ1",
"name": "mug",
"value": 0.9210669,
"app_id": "main"
},
{
"id": "ai_B3MXt5Ng",
"name": "refreshment",
"value": 0.9185593,
"app_id": "main"
},
{
"id": "ai_2gmKZLxp",
"name": "cold",
"value": 0.91295856,
"app_id": "main"
},
{
"id": "ai_3PlgVmlN",
"name": "food",
"value": 0.9119204,
"app_id": "main"
},
{
"id": "ai_54zxXFGL",
"name": "full",
"value": 0.91089505,
"app_id": "main"
},
{
"id": "ai_SsmKLB4z",
"name": "bar",
"value": 0.9056676,
"app_id": "main"
},
{
"id": "ai_BrnHNkt0",
"name": "coffee",
"value": 0.90262496,
"app_id": "main"
},
{
"id": "ai_mCpQg89c",
"name": "glass",
"value": 0.89882934,
"app_id": "main"
},
{
"id": "ai_7D0mdp1W",
"name": "delicious",
"value": 0.894392,
"app_id": "main"
},
{
"id": "ai_WbwL0pPL",
"name": "breakfast",
"value": 0.8932399,
"app_id": "main"
},
{
"id": "ai_zFnPQdgB",
"name": "wood",
"value": 0.88797945,
"app_id": "main"
},
{
"id": "ai_3R5pJ6hB",
"name": "lager",
"value": 0.88745904,
"app_id": "main"
},
{
"id": "ai_8LWlDfFD",
"name": "table",
"value": 0.87949455,
"app_id": "main"
},
{
"id": "ai_MmRdqDFp",
"name": "soap",
"value": 0.87376094,
"app_id": "main"
},
{
"id": "ai_5VHsZr8N",
"name": "liquid",
"value": 0.8715329,
"app_id": "main"
}
]
}
}
]
}
}
]
}Via Bytes
Below is an example of how you would send the bytes of a video and receive back predictions from the general model.
clarifai.models.predict(Clarifai.GENERAL_MODEL, {base64: 'AAAAIGZ...'}, {video: true})
.then(log)
.catch(log);from clarifai.rest import ClarifaiApp
from clarifai.rest import Video as ClVideo
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('general-v1.3')
video = ClVideo(filename='/home/user/video.mp4')
model.predict([video])ClarifaiResponse<List<ClarifaiOutput<Frame>>> frames = client.getDefaultModels().generalVideoModel().predict()
.withInputs(ClarifaiInput.forVideo(new File("/home/user/video.mp4)")))
.executeSync();
using System.IO;
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Inputs;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.PublicModels.GeneralVideoModel.Predict(
new ClarifaiFileVideo(File.ReadAllBytes("/home/user/video.mp4")))
.ExecuteAsync();
}
}
}
Objective-C client details coming sooncurl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"data": {
"video": {
"base64": "'"$(base64 /home/user/video.mp4)"'"
}
}
}
]
}'\
https://api.clarifai.com/v2/models/aaa03c23b3724a16a56b629203edc62c/outputs- JS
- Py
- Java
- C#
- ObjC
- cURL
Response{
"status": {
"code": 10000,
"description": "Ok"
},
"outputs": [
{
"id": "f6f9e1b007d742fb9d777f35cf3bffd0",
"status": {
"code": 10000,
"description": "Ok"
},
"created_at": "2017-06-28T16:00:51.258194418Z",
"model": {
"id": "aaa03c23b3724a16a56b629203edc62c",
"name": "general-v1.3",
"created_at": "2016-03-09T17:11:39.608845Z",
"app_id": "main",
"output_info": {
"message": "Show output_info with: GET /models/{model_id}/output_info",
"type": "concept",
"type_ext": "concept"
},
"model_version": {
"id": "aa9ca48295b37401f8af92ad1af0d91d",
"created_at": "2016-07-13T01:19:12.147644Z",
"status": {
"code": 21100,
"description": "Model trained successfully"
}
}
},
"input": {
"id": "b17d29ad1b714869a8c729e510ab22d0",
"data": {
"video": {
"url": "https://s3.amazonaws.com/clarifai-api/vid/prod/ib81c84d5b2341858b86da18a2bd21d2/e86fbf516521425098081dd42e157a12",
"base64": "true"
}
}
},
"data": {
"frames": [
{
"frame_info": {
"index": 0,
"time": 0
},
"data": {
"concepts": [
{
"id": "ai_VTlCx2f2",
"name": "window",
"value": 0.99909437,
"app_id": "main"
},
{
"id": "ai_l8TKp2h5",
"name": "people",
"value": 0.99610686,
"app_id": "main"
},
{
"id": "ai_VPmHr5bm",
"name": "adult",
"value": 0.9958472,
"app_id": "main"
},
{
"id": "ai_SVshtN54",
"name": "one",
"value": 0.9937376,
"app_id": "main"
},
{
"id": "ai_TJ9wFfK5",
"name": "portrait",
"value": 0.9899301,
"app_id": "main"
},
{
"id": "ai_ZrPNDjxN",
"name": "daylight",
"value": 0.9885398,
"app_id": "main"
},
{
"id": "ai_dxSG2s86",
"name": "man",
"value": 0.9833108,
"app_id": "main"
},
{
"id": "ai_86sS08Pw",
"name": "wear",
"value": 0.9807093,
"app_id": "main"
},
{
"id": "ai_GxSDhQ34",
"name": "facial expression",
"value": 0.9769263,
"app_id": "main"
},
{
"id": "ai_Pf2b7clG",
"name": "indoors",
"value": 0.96838474,
"app_id": "main"
},
{
"id": "ai_VRmbGVWh",
"name": "travel",
"value": 0.96641624,
"app_id": "main"
},
{
"id": "ai_XNzGRk0F",
"name": "side view",
"value": 0.9603646,
"app_id": "main"
},
{
"id": "ai_Zmhsv0Ch",
"name": "outdoors",
"value": 0.9434113,
"app_id": "main"
},
{
"id": "ai_n9vjC1jB",
"name": "light",
"value": 0.94182396,
"app_id": "main"
},
{
"id": "ai_WcnFrjw1",
"name": "backlit",
"value": 0.9347838,
"app_id": "main"
},
{
"id": "ai_QKqjh1CM",
"name": "vehicle window",
"value": 0.92699903,
"app_id": "main"
},
{
"id": "ai_mlrv94tv",
"name": "reflection",
"value": 0.90993655,
"app_id": "main"
},
{
"id": "ai_41s912fX",
"name": "fair weather",
"value": 0.90100014,
"app_id": "main"
},
{
"id": "ai_rsX6XWc2",
"name": "building",
"value": 0.88111985,
"app_id": "main"
},
{
"id": "ai_L83krFdq",
"name": "veil",
"value": 0.8785704,
"app_id": "main"
}
]
}
}
]
}
}
]
}Search
Note: The Search API currently is only available for images.
A common case for using Clarifai is to get the concepts predicted in an image and then use those concepts to power search.
The Search API allows you to send images (url or bytes) to the service and have them indexed by 'general' model concepts and their visual representations. Once indexed, you can search for images by concept or by image.
Add Images to Search Index
To get started with search, you must first add images to the search index. You can add one or more images to the index at a time. You can supply an image either with a publicly accessible URL or by directly sending image bytes. You can send up to 128 images in one API call.
app.inputs.create([
{url: "https://samples.clarifai.com/metro-north.jpg"},
{url: "https://samples.clarifai.com/wedding.jpg"},
{base64: "G7p3m95uAl..."}
]).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
from clarifai.rest import Image as ClImage
app = ClarifaiApp(api_key='YOUR_API_KEY')
img1 = ClImage(url="https://samples.clarifai.com/metro-north.jpg")
img2 = ClImage(url="https://samples.clarifai.com/puppy.jpeg")
img3 = ClImage(file_obj=open('/home/user/image.jpeg', 'rb'))
app.inputs.bulk_create_images([img1, img2, img3])
client.addInputs()
.plus(
ClarifaiInput.forImage("https://samples.clarifai.com/metro-north.jpg"),
ClarifaiInput.forImage("https://samples.clarifai.com/wedding.jpg")
)
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Inputs;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.AddInputs(
new ClarifaiURLImage("https://samples.clarifai.com/metro-north.jpg"),
new ClarifaiURLImage("https://samples.clarifai.com/wedding.jpg")
).ExecuteAsync();
}
}
}
ClarifaiImage *image1 = [[ClarifaiImage alloc] initWithURL:@"https://samples.clarifai.com/metro-north.jpg"];
ClarifaiImage *image2 = [[ClarifaiImage alloc] initWithURL:@"https://samples.clarifai.com/wedding.jpg"];
[app addInputs:@[image1, image2] completion:^(NSArray<ClarifaiInput *> *inputs, NSError *error) {
NSLog(@"inputs: %@", inputs);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
}
},
{
"data": {
"image": {
"url": "https://samples.clarifai.com/wedding.jpg"
}
}
}
]
}'\
https://api.clarifai.com/v2/inputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Response
{
"status": {
"code": 10000,
"description": "Ok"
},
"inputs": [
{
"id": "edc70c917475499abdc7151f41d6cf3e",
"created_at": "2016-11-22T17:06:02Z",
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
},
"status": {
"code": 30001,
"description": "Download pending"
}
},
{
"id": "f96ca3bbf02041c59addcc13e3468b7d",
"created_at": "2016-11-22T17:06:02Z",
"data": {
"image": {
"url": "https://samples.clarifai.com/wedding.jpg"
}
},
"status": {
"code": 30001,
"description": "Download pending"
}
}
]
}
Search By Concept
Once your images are indexed, you can search for them by concept.
app.inputs.search({ concept: {name: 'people'} }).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
# search by public concept
app.inputs.search_by_predicted_concepts(concept='people')
# search by a list of concepts
app.inputs.search_by_predicted_concepts(concepts=['people'])
# search by concept id
app.inputs.search_by_predicted_concepts(concept_id='ai_dP13sXL4')
# search by a list of concept ids
app.inputs.search_by_predicted_concepts(concept_ids=['ai_dP13sXL4'])
client.searchInputs(SearchClause.matchConcept(Concept.forName("people")))
.getPage(1)
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Searches;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.SearchInputs(SearchBy.ConceptName("people"))
.Page(1)
.ExecuteAsync();
}
}
}
// First create a search term with a concept you want to search.
ClarifaiConcept *conceptFromGeneralModel = [[ClarifaiConcept alloc] initWithConceptName:@"people"];
ClarifaiSearchTerm *searchTerm = [ClarifaiSearchTerm searchByPredictedConcept:conceptFromGeneralModel];
[app search:@[searchTerm] page:@1 perPage:@20 completion:^(NSArray<ClarifaiSearchResult *> *results, NSError *error) {
// Print output of first search result.
NSLog(@"inputID: %@", results[0].inputID);
NSLog(@"URL: %@", results[0].mediaURL);
NSLog(@"probability of public concept: %@", results[0].score);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"query": {
"ands": [
{
"output": {
"data": {
"concepts": [
{
"name": "people"
}
]
}
}
}
]
}
}'\
https://api.clarifai.com/v2/searches
- JS
- Py
- Java
- C#
- ObjC
- cURL
Response
{
"status": {
"code": 10000,
"description": "Ok"
},
"hits": [
{
"score": 0.98155165,
"input": {
"id": "f96ca3bbf02041c59addcc13e3468b7d",
"created_at": "2016-11-22T17:06:02Z",
"data": {
"image": {
"url": "https://samples.clarifai.com/wedding.jpg"
}
},
"status": {
"code": 30000,
"description": "Download complete"
}
}
}
]
}
Search By Image
You can also search for images using another image. In this case, you provide an image (url or bytes) and the results will return all the images in your search index that are visually similar to the one provided.
app.inputs.search({ input: {url: 'https://samples.clarifai.com/puppy.jpeg'} }).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
# search by image url
app.inputs.search_by_image(url="https://samples.clarifai.com/metro-north.jpg")
# search by existing input id
input_id = "some_existing_input_id"
app.inputs.search_by_image(image_id=input_id)
# search by raw bytes
data = "image_raw_bytes"
app.inputs.search_by_image(imgbytes=data)
# search by base64 bytes
base64_data = "image_bytes_encoded_in_base64"
app.inputs.search_by_image(base64bytes=base64_data)
# search by local filename
filename="filename_on_local_disk.jpg"
app.inputs.search_by_image(filename=filename)
# search from fileio
fio = open("filename_on_local_disk.jpg", 'rb')
app.inputs.search_by_image(fileobj=fio)
client.searchInputs(SearchClause.matchImageVisually(ClarifaiImage.of("https://samples.clarifai.com/metro-north.jpg")))
.getPage(1)
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Searches;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.SearchInputs(SearchBy.ImageURL("https://samples.clarifai.com/metro-north.jpg"))
.Page(1)
.ExecuteAsync();
}
}
}
ClarifaiSearchTerm *searchTerm = [ClarifaiSearchTerm searchVisuallyWithImageURL:@"https://samples.clarifai.com/metro-north.jpg"];
[app search:@[searchTerm] page:@1 perPage:@20 completion:^(NSArray<ClarifaiSearchResult *> *results, NSError *error) {
// Print output of first search result.
NSLog(@"inputID: %@", results[0].inputID);
NSLog(@"URL: %@", results[0].mediaURL);
NSLog(@"probability of visual similarity: %@", results[0].score);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"query": {
"ands": [
{
"output": {
"input": {
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
}
}
}
}
]
}
}'\
https://api.clarifai.com/v2/searches
- JS
- Py
- Java
- C#
- ObjC
- cURL
Response
{
"status": {
"code": 10000,
"description": "Ok"
},
"hits": [
{
"score": 0.9999997,
"input": {
"id": "edc70c917475499abdc7151f41d6cf3e",
"created_at": "2016-11-22T17:06:02Z",
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
},
"status": {
"code": 30000,
"description": "Download complete"
}
}
},
{
"score": 0.3915897,
"input": {
"id": "f96ca3bbf02041c59addcc13e3468b7d",
"created_at": "2016-11-22T17:06:02Z",
"data": {
"image": {
"url": "https://samples.clarifai.com/wedding.jpg"
}
},
"status": {
"code": 30000,
"description": "Download complete"
}
}
}
]
}
Train
Clarifai provides many different models that 'see' the world differently. A model contains a group of concepts. A model will only see the concepts it contains.
There are times when you wish you had a model that sees the world the way you see it. The API allows you to do this. You can create your own model and train it with your own images and concepts. Once you train it to see how you would like it to see, you can then use that model to make predictions.
You do not need many images to get started. We recommend starting with 10 and adding more as needed. Before you train your first model you will have needed to create an application. From there you will be able to change your Base Workflow to optimize custom training using the knowledge base from select public models.
Add Images With Concepts
To get started training your own model, you must first add images that already contain the concepts you want your model to see.
app.inputs.create({
url: "https://samples.clarifai.com/puppy.jpeg",
concepts: [
{
id: "boscoe",
value: true
}
]
});
from clarifai.rest import ClarifaiApp
from clarifai.rest import Image as ClImage
app = ClarifaiApp(api_key='YOUR_API_KEY')
# add multiple images with concepts
img1 = ClImage(url="https://samples.clarifai.com/puppy.jpeg", concepts=['boscoe'], not_concepts=['our_wedding'])
img2 = ClImage(url="https://samples.clarifai.com/wedding.jpg", concepts=['our_wedding'], not_concepts=['cat','boscoe'])
app.inputs.bulk_create_images([img1, img2])
client.addInputs()
.plus(
ClarifaiInput.forImage("https://samples.clarifai.com/puppy.jpeg")
.withConcepts(Concept.forID("boscoe"))
)
.executeSync();
using System.Collections.Generic;
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Inputs;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.AddInputs(
new ClarifaiURLImage(
"https://samples.clarifai.com/puppy.jpeg",
positiveConcepts: new List<Concept> {new Concept(id: "boscoe")}))
.ExecuteAsync();
}
}
}
ClarifaiImage *image = [[ClarifaiImage alloc] initWithURL:@"https://samples.clarifai.com/puppy.jpeg" andConcepts:@"cute puppy"];
[_app addInputs:@[image] completion:^(NSArray<ClarifaiInput *> *inputs, NSError *error) {
NSLog(@"inputs: %@", inputs);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"data": {
"image": {
"url": "https://samples.clarifai.com/puppy.jpeg"
},
"concepts":[
{
"id": "boscoe",
"value": true
}
]
}
}
]
}'\
https://api.clarifai.com/v2/inputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Response
{
"status": {
"code": 10000,
"description": "Ok"
},
"inputs": [
{
"id": "e82fd13b11354d808cc48dc8f94ec3a9",
"created_at": "2016-11-22T17:16:00Z",
"data": {
"image": {
"url": "https://samples.clarifai.com/puppy.jpeg"
},
"concepts": [
{
"id": "boscoe",
"name": "boscoe",
"app_id": "f09abb8a57c041cbb94759ebb0cf1b0d",
"value": 1
}
]
},
"status": {
"code": 30000,
"description": "Download complete"
}
}
]
}
Create A Model
Once your images with concepts are added, you are now ready to create the model. You'll need a name for the model and you'll also need to provide it with the concepts you added above.
Take note of the model id that is returned in the response. You'll need that for the next two steps.
app.models.create(
"pets",
[
{ "id": "boscoe" }
]
).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.create('pets', concepts=['boscoe'])
client.createModel("pets")
.withOutputInfo(ConceptOutputInfo.forConcepts(
Concept.forID("boscoe")
))
.executeSync();
using System.Collections.Generic;
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.CreateModel(
"pets",
concepts: new List<Concept> {new Concept("boscoe")})
.ExecuteAsync();
}
}
}
[app createModel:@[concept] name:modelName conceptsMutuallyExclusive:NO closedEnvironment:NO
completion:^(ClarifaiModel *model, NSError *error) {
NSLog(@"model: %@", model);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"model": {
"id": "pets",
"output_info": {
"data": {
"concepts": [
{
"id": "boscoe"
}
]
},
"output_config": {
"concepts_mutually_exclusive": false,
"closed_environment":false
}
}
}
}'\
https://api.clarifai.com/v2/models
- JS
- Py
- Java
- C#
- ObjC
- cURL
Response
{
"status": {
"code": 10000,
"description": "Ok"
},
"model": {
"name": "pets",
"id": "a10f0cf48cf3426cbb8c4805e246c214",
"created_at": "2016-11-22T17:17:36Z",
"app_id": "f09abb8a57c041cbb94759ebb0cf1b0d",
"output_info": {
"message": "Show output_info with: GET /models/{model_id}/output_info",
"type": "concept",
"output_config": {
"concepts_mutually_exclusive": false,
"closed_environment": false
}
},
"model_version": {
"id": "e7bcd534b61b4874a3ab69fba974c012",
"created_at": "2016-11-22T17:17:36Z",
"status": {
"code": 21102,
"description": "Model not yet trained"
}
}
}
}
Train The Model
Now that you've added images with concepts, then created a model with those concepts, the next step is to train the model. When you train a model, you are telling the system to look at all the images with concepts you've provided and learn from them. This train operation is asynchronous. It may take a few seconds for your model to be fully trained and ready.
Keep note of the model_version id in the response. We'll need that for the next section when we predict with the model.
app.models.train("{model_id}").then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
// or if you have an instance of a model
model.train().then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('{model_id}')
model.train()
client.trainModel("{model_id}").executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.TrainModel<Concept>("{model_id}")
.ExecuteAsync();
}
}
}
ClarifaiImage *image = [[ClarifaiImage alloc] initWithURL:@"https://samples.clarifai.com/puppy.jpeg"]
[app getModel:@"{id}" completion:^(ClarifaiModel *model, NSError *error) {
[model train:^(ClarifaiModel *model, NSError *error) {
NSLog(@"model: %@", model);
}];
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
https://api.clarifai.com/v2/models/{model_id}/versions
- JS
- Py
- Java
- C#
- ObjC
- cURL
Response
{
"status": {
"code": 10000,
"description": "Ok"
},
"model": {
"name": "pets",
"id": "a10f0cf48cf3426cbb8c4805e246c214",
"created_at": "2016-11-22T17:17:36Z",
"app_id": "f09abb8a57c041cbb94759ebb0cf1b0d",
"output_info": {
"message": "Show output_info with: GET /models/{model_id}/output_info",
"type": "concept",
"output_config": {
"concepts_mutually_exclusive": false,
"closed_environment": false
}
},
"model_version": {
"id": "d1b38fd2251148d08675c5542ef00c7b",
"created_at": "2016-11-22T17:21:13Z",
"status": {
"code": 21103,
"description": "Custom model is currently in queue for training, waiting on inputs to process."
}
}
}
}
Predict With The Model
Now that we have a trained model. We can start making predictions with it. In our predict call we need to specify three items. The model id, model_version id and the input we want a prediction for.
Note: you can repeat the above steps as often as you like. By adding more images with concepts and training, you can get the model to predict exactly how you want it to.
let app = new Clarifai.App({apiKey: 'YOUR_API_KEY'});
app.models.predict({id:'MODEL_ID', version:'MODEL_VERSION_ID'}, "https://samples.clarifai.com/metro-north.jpg").then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('MODEL_ID')
model.model_version = 'MODEL_VERSION_ID'
image = ClImage(url='https://samples.clarifai.com/metro-north.jpg')
model.predict([image])
ModelVersion modelVersion = client.getModelVersionByID("MODEL_ID", "MODEL_VERSION_ID")
.executeSync()
.get();
client.predict("MODEL_ID")
.withVersion(modelVersion)
.withInputs(ClarifaiInput.forImage("https://samples.clarifai.com/metro-north.jpg"))
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Inputs;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
var response = await Client.Predict<Concept>(
"YOUR_MODEL_ID",
new ClarifaiURLImage("https://samples.clarifai.com/metro-north.jpg"),
modelVersionID: "MODEL_VERSION_ID")
.ExecuteAsync();
}
}
}
ClarifaiImage *image = [[ClarifaiImage alloc] initWithURL:@"https://samples.clarifai.com/puppy.jpeg"]
[app getModel:@"{id}" completion:^(ClarifaiModel *model, NSError *error) {
[model predictOnImages:@[image]
completion:^(NSArray<ClarifaiSearchResult *> *outputs, NSError *error) {
NSLog(@"outputs: %@", outputs);
}];
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
}
}
]
}'\
https://api.clarifai.com/v2/models/<model_id>/versions/<model_version_id>/outputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Response
{
"status": {
"code": 10000,
"description": "Ok"
},
"outputs": [
{
"id": "e8b6eb27de764f3fa8d4f7752a3a2dfc",
"status": {
"code": 10000,
"description": "Ok"
},
"created_at": "2016-11-22T17:22:23Z",
"model": {
"name": "pets",
"id": "a10f0cf48cf3426cbb8c4805e246c214",
"created_at": "2016-11-22T17:17:36Z",
"app_id": "f09abb8a57c041cbb94759ebb0cf1b0d",
"output_info": {
"message": "Show output_info with: GET /models/{model_id}/output_info",
"type": "concept",
"output_config": {
"concepts_mutually_exclusive": false,
"closed_environment": false
}
},
"model_version": {
"id": "d1b38fd2251148d08675c5542ef00c7b",
"created_at": "2016-11-22T17:21:13Z",
"status": {
"code": 21100,
"description": "Model trained successfully"
}
}
},
"input": {
"id": "e8b6eb27de764f3fa8d4f7752a3a2dfc",
"data": {
"image": {
"url": "https://samples.clarifai.com/puppy.jpeg"
}
}
},
"data": {
"concepts": [
{
"id": "boscoe",
"name": "boscoe",
"app_id": "f09abb8a57c041cbb94759ebb0cf1b0d",
"value": 0.98308545
}
]
}
}
]
}
Workflow
Workflows are a new entity added to Clarifai, which encompass one or more Public or Custom model(s). Every workflow is attached to one of your applications. Under each workflow, you will see a list of the public models and all custom models in that application when selecting models to add to your workflow. With Workflow Predict, you will be able to reduce the latency and in turn make your product more efficient.
Note: this won't have any impact on the price you are charged per call. You will still be charged for the same operation if it were separate calls to the API.
Workflow Setup
To set up a workflow, you will need to head over to the Applications page through your account. From there, you will need to select which application you want to create the workflow under.
Then under that application, you will see a section labeled "Workflows" and a button to "Create Workflow".
After that, the page will reveal a new workflow form to fill out. Fill out the Workflow ID field, this will be used to make the API call, so make sure to give it something URL friendly! Included there, you will also a list that consists of a model field and a version associated with it. For the public models, you will be mandated to use the latest version. For your custom models, you will be able to select the version of your model. To add another model, you will just click underneath your latest addition on the "Add Model". The max limit of models associated with any given workflow is 5 models. If you would like to remove a model, there is a large X that will allow you to remove a model. Once you have finished adding everything, press the "Save Workflow" button and that will save the state of your workflow. Now you are ready to predict using your brand new workflow. You can edit a given workflow at any time, in case you don't like it.
Workflow Predict
The Workflow Predict API allows you to predict using 1 or more model(s), regardless of them being public or custom, within a single API call! The max number of inputs processed at once with any given workflow is 32.
Now that you have that all set up, you will be able to predict under a workflow using the POST /v2/workflows/{workflow_id}/results endpoint. Your {workflow-id} currently is whatever you set as your ID. Then as far as your request body, nothing has changed with how you would normally do a predict. In the response body, you will see a results object and each object will be the response from the models in the same ordering from the workflow you set up.
You can also use the Explorer to see the results of your workflow's predictions on a given input.
app.workflow.predict('{workflow-id}', "https://samples.clarifai.com/metro-north.jpg").then(
function(response){
// Do something with response
},
function(err){
// There was an error
}
);from clarifai.rest import ClarifaiApp
from clarifai.rest import Workflow
from clarifai.rest import Image as ClImage
app = ClarifaiApp(api_key='YOUR_API_KEY')
workflow = Workflow(app.api, workflow_id="your-workflow-id")
image = ClImage(url='https://samples.clarifai.com/metro-north.jpg')
workflow.predict([image])
client.workflowPredict("{workflow-id}")
.withInputs(ClarifaiInput.forImage("https://samples.clarifai.com/metro-north.jpg"))
.executeSync();
using System.Collections.Generic;
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Inputs;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.WorkflowPredict(
"{workflow-id}",
new List<IClarifaiInput>
{
new ClarifaiURLImage("https://samples.clarifai.com/puppy.jpeg")
})
.ExecuteAsync();
}
}
}
// Coming Sooncurl -X POST \
-H 'authorization: Key YOUR_API_KEY' \
-H 'content-type: application/json' \
-d '{
"inputs": [
{
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
}
}
]
}'\
https://api.clarifai.com/v2/workflows/{workflow-id}/results- JS
- Py
- Java
- C#
- ObjC
- cURL
Response{
"status": {
"code": 10000,
"description": "Ok"
},
"workflow": {
"id": "my-workflow",
"app_id": "c54b7637df12407aa9c57dfd6d5c057f",
"created_at": "2017-07-10T01:45:05.672880Z"
},
"results": [
{
"status": {
"code": 10000,
"description": "Ok"
},
"input": {
"id": "c88aeed9d04c471cace6f8e4801f1a1c",
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
}
},
"outputs": [
{
"id": "feae971167a04d1bbebb7ea49d6ba0f7",
"status": {
"code": 10000,
"description": "Ok"
},
"created_at": "2017-07-10T12:01:44.929928529Z",
"model": {
"id": "d16f390eb32cad478c7ae150069bd2c6",
"name": "moderation",
"created_at": "2017-05-12T21:28:00.471607Z",
"app_id": "main",
"output_info": {
"message": "Show output_info with: GET /models/{model_id}/output_info",
"type": "concept",
"type_ext": "concept"
},
"model_version": {
"id": "b42ac907ac93483484483a0040a386be",
"created_at": "2017-05-12T21:28:00.471607Z",
"status": {
"code": 21100,
"description": "Model trained successfully"
}
}
},
"data": {
"concepts": [
{
"id": "ai_QD1zClSd",
"name": "safe",
"value": 0.99999714,
"app_id": "main"
},
{
"id": "ai_kBBGf7r8",
"name": "gore",
"value": 3.7771046e-05,
"app_id": "main"
},
{
"id": "ai_8QQwMjQR",
"name": "drug",
"value": 1.0449563e-05,
"app_id": "main"
},
{
"id": "ai_V76bvrtj",
"name": "explicit",
"value": 5.2887003e-06,
"app_id": "main"
},
{
"id": "ai_RtXh5qkR",
"name": "suggestive",
"value": 4.7939684e-06,
"app_id": "main"
}
]
}
},
{
"id": "f635b40cbeee47ddb7b348a981e14faf",
"status": {
"code": 10000,
"description": "Ok"
},
"created_at": "2017-07-10T12:01:44.929941126Z",
"model": {
"id": "aaa03c23b3724a16a56b629203edc62c",
"name": "general-v1.3",
"created_at": "2016-02-26T23:38:40.086101Z",
"app_id": "main",
"output_info": {
"message": "Show output_info with: GET /models/{model_id}/output_info",
"type": "concept",
"type_ext": "concept"
},
"model_version": {
"id": "aa9ca48295b37401f8af92ad1af0d91d",
"created_at": "2016-07-13T00:58:55.915745Z",
"status": {
"code": 21100,
"description": "Model trained successfully"
}
}
},
"data": {
"concepts": [
{
"id": "ai_HLmqFqBf",
"name": "train",
"value": 0.9989112,
"app_id": "main"
},
{
"id": "ai_fvlBqXZR",
"name": "railway",
"value": 0.9975532,
"app_id": "main"
},
{
"id": "ai_Xxjc3MhT",
"name": "transportation system",
"value": 0.9959158,
"app_id": "main"
},
{
"id": "ai_6kTjGfF6",
"name": "station",
"value": 0.992573,
"app_id": "main"
},
{
"id": "ai_RRXLczch",
"name": "locomotive",
"value": 0.992556,
"app_id": "main"
},
{
"id": "ai_VRmbGVWh",
"name": "travel",
"value": 0.98789215,
"app_id": "main"
},
{
"id": "ai_SHNDcmJ3",
"name": "subway system",
"value": 0.9816359,
"app_id": "main"
},
{
"id": "ai_jlb9q33b",
"name": "commuter",
"value": 0.9712483,
"app_id": "main"
},
{
"id": "ai_46lGZ4Gm",
"name": "railroad track",
"value": 0.9690325,
"app_id": "main"
},
{
"id": "ai_tr0MBp64",
"name": "traffic",
"value": 0.9687052,
"app_id": "main"
},
{
"id": "ai_l4WckcJN",
"name": "blur",
"value": 0.9667078,
"app_id": "main"
},
{
"id": "ai_2gkfMDsM",
"name": "platform",
"value": 0.9624243,
"app_id": "main"
},
{
"id": "ai_CpFBRWzD",
"name": "urban",
"value": 0.960752,
"app_id": "main"
},
{
"id": "ai_786Zr311",
"name": "no person",
"value": 0.95864904,
"app_id": "main"
},
{
"id": "ai_6lhccv44",
"name": "business",
"value": 0.95720303,
"app_id": "main"
},
{
"id": "ai_971KsJkn",
"name": "track",
"value": 0.9494642,
"app_id": "main"
},
{
"id": "ai_WBQfVV0p",
"name": "city",
"value": 0.94089437,
"app_id": "main"
},
{
"id": "ai_dSCKh8xv",
"name": "fast",
"value": 0.9399334,
"app_id": "main"
},
{
"id": "ai_TZ3C79C6",
"name": "road",
"value": 0.93121606,
"app_id": "main"
},
{
"id": "ai_VSVscs9k",
"name": "terminal",
"value": 0.9230834,
"app_id": "main"
}
]
}
}
]
}
]
}Evaluate
Now that you've successfully trained the model, you may want to test its performance before using it in the production environment. The Model Evaluation tool allows you to perform a cross validation on a specified model version. Once the evaluation is complete, you’ll be able to view various metrics that will inform the model’s performance.
This evaluate operation is asynchronous and is currently available on the Explorer only.
How It Works
Model Evaluation performs a K-split cross validation on data you used to train your custom model. 
In the cross validation process, it will:
- Set aside a random 1/K subset of the training data and designate as a test set,
- Train a new model with the remaining training data,
- Pass the test set data through this new model to make predictions,
- Compare the predictions against the test set’s actual labels, and
- Repeat steps 1) through 4) across K splits to average out the evaluation results
For a refresher on custom training, please refer to our training documentation.
Requirements
To run the evaluation on your custom model, it will need the meet the following criteria:
- A custom trained model model version with:
- At least 2 concepts
- At least 10 training inputs per concept (At least 50 inputs per concept is recommended)
Running Evaluation
You can run the evaluation on a specific model version of your custom model in the Explorer . Go to your Application, click on your model of interest, and select the Versions tab. Simply click on the Evaluate button for the specific model version.
The evaluation may take up to 30 minutes. Once it is complete, the Evaluate button will become View button. Click on the View button to see the evaluation results.
Note that the evaluation may result in an error if the model version doesn’t satisfy the requirements above.
For more information on how to interpret the evaluation results and to improve your model, check out the Evaluation corner under the “Advanced” section below.
Feedback
The feedback endpoint allows you to send feedback on your prediction and search responses. Currently, you can send feedback on 3 types of responses: 1. predictions without regions, 2. predictions with regions, and 3. searches. The request bodies are slightly different but all share the feedback_info component. Note that sending feedback will not result in immediate changes to your predictions or searches, but you may see accuracy improvements to them over time as we aggregate the feedback for evaluation and improvement of your models.
We highly encourage integrating feedback into your application as this dramatically improves accuracy of models over time. You or users of your application can correct errors, such as: if the system returned cat for an image of a dog, you can send feedback with add_tags="dog", remove_tags="cat". You can also use feedback to further enhance the models by suggesting a new concept or region, such as: if the model returned "dog", you can add a new concept called "laborador_retriever".
The feedback endpoint does not count towards your usage limits. Please use it as much as you can.
Feedback Info
The feedback info object allows for specific information to be recorded from your application and your end users. All fields are optional unless noted.
- •
event_type[required] - Use'annotation'for prediction feedback, and use'search_click'for search feedback. - •
output_id/search_id[required] - the id associated with the output or search received from a given API call. With this, the API can traceback the response. - •
end_user_id- the id associated with your end user. If you want to be able to understand which user was producing this feedback, you can associate it with this field. - •
session_id- the id associated with your user's interface. This is allows you to understand what the user was doing during that time and allows you an avenue to investigate more.
Prediction Feedback
To send feedback on predictions, you can use POST /v2/models/{model_id}/feedback.
Please use this request to send feedback on the model prediction responses without any region data. This includes any feedback on concept models such as General, Food, and Travel models.
In each request body, you can send
- • concept
id: either the correct concept name that you defined or the id that Clarifai defined in the predict response - • concept
value:trueif the concept is present in the input, orfalseif it is not present in the input
This allows for our model to re-evaluate the concept and improve the accuracy of our predictions.
Note that the event_type should be set to annotation.
clarifai.models.feedback(Clarifai.GENERAL_MODEL, 'https://samples.clarifai.com/dog.tiff', {
id: '{input_id}',
data: {
concepts: [
{'id': 'ai_8S2Vq3cR', 'value': true },
{'id': 'ai_mFqxrph2', 'value': false }
]
},
info: {
'eventType': 'annotation',
'outputId': '{output_id}',
'endUserId': '{end_user_id}',
'sessionId': '{session_id}'
}
})
.then(log)
.catch(log);from clarifai.rest import ClarifaiApp, FeedbackInfo
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('general-v1.3')
model.send_concept_feedback(
input_id='{input_id}',
url='https://samples.clarifai.com/dog.tiff',
concepts=['ai_8S2Vq3cR'],
not_concepts=['ai_mFqxrph2'],
feedback_info=FeedbackInfo(event_type='annotation',
output_id='{output_id}',
end_user_id='{end_user_id}',
session_id='{session_id}'))ModelFeedbackRequest request =
client.modelFeedback(client.getDefaultModels().generalModel().id())
.withInputId("{input_id}")
.withImageUrl("https://samples.clarifai.com/dog.tiff")
.withConcepts(
ConceptFeedback.forIdAndValue("ai_8S2Vq3cR", true),
ConceptFeedback.forIdAndValue("ai_mFqxrph2", false)
)
.withEventType("annotation")
.withOutputId("{output_id}")
.withEndUserId("{end_user_id}")
.withSessionId("{session_id}");
using System.Collections.Generic;
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Feedbacks;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.ModelFeedback(
client.PublicModels.GeneralModel.ModelID,
imageUrl: "https://samples.clarifai.com/dog.tiff",
inputID: "inputID",
outputID: "{output_id}",
endUserID: "{end_user_id}",
sessionID: "{session_id}",
concepts: new List<ConceptFeedback>
{
new ConceptFeedback("ai_8S2Vq3cR", true),
new ConceptFeedback("ai_mFqxrph2", false)
}
)
.ExecuteAsync();
}
}
}
// Coming Sooncurl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"input": {
"id": "{input_id}",
"data": {
"image": {
"url": "https://samples.clarifai.com/dog.tiff"
},
"concepts":[
{"id": "ai_8S2Vq3cR", "value": true },
{"id": "ai_mFqxrph2", "value": false }
]
},
"feedback_info":{
"event_type": "annotation",
"output_id": "{output_id}",
"end_user_id": "{end_user_id}",
"session_id": "{session_id}"
}
}
}'\
https://api.clarifai.com/v2/models/aaa03c23b3724a16a56b629203edc62c/feedback- JS
- Py
- Java
- C#
- ObjC
- cURL
Response{
"status": {
"code": 10000,
"description": "Ok"
}
}Prediction with Regions Feedback
To send feedback on predictions, you can use POST /v2/models/{model_id}/feedback.
Please use this request to send feedback on the model prediction responses with region data. This includes any feedback on detection models such as Face Detection, Demographic, Celebrity, and Logo models. These models return a list of region objects, each of which contains local coordinates and detected concepts.
In the request body for the feedback for predictions with regions, you can send each region object with:
- • correct
bounding_boxcoordinates of the region - •
feedbackfor the originally predicted region:accurateif it's correct,misplacedif the coordinates should be modified,not_detectedif the region was not originally detected by the model, orfalse_positiveif the region should not have been detected by the model - • concept
id: either the correct concept name that you defined or the id that Clarifai defined in the predict response - • concept
value:trueif the concept is present in the input, orfalseif it is not present in the input
Note that the event_type should be set to annotation.
clarifai.models.feedback(Clarifai.LOGO_MODEL, 'https://clarifai.com/developer/static/images/model-samples/logo-002.jpg', {
id: '{input_id}',
data: {
'regions': [
{
'region_info': {
'bounding_box': {
'top_row': 0.3,
'left_col': 0.2,
'bottom_row': 0.7,
'right_col': 0.8
},
'feedback': 'accurate'
},
'data': {
'concepts': [
{"id": "ai_TG4GHlnf", "value": true }
]
}
}
]
},
info: {
'eventType': 'annotation',
'outputId': '{output_id}',
'endUserId': '{end_user_id}',
'sessionId': '{session_id}'
}
})
.then(log)
.catch(log);from clarifai.rest import ClarifaiApp, Region, RegionInfo, BoundingBox, Concept, FeedbackInfo
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('logo')
model.send_region_feedback(
input_id='{input_id}',
url='https://clarifai.com/developer/static/images/model-samples/logo-002.jpg',
regions=[Region(
region_info=RegionInfo(
bbox=BoundingBox(top_row=0.3, left_col=0.2, bottom_row=0.7, right_col=0.8),
feedback_type='accurate'),
concepts=[Concept(concept_id='ai_TG4GHlnf', value=True)])],
feedback_info=FeedbackInfo(event_type='annotation',
output_id='{output_id}',
end_user_id='{end_user_id}',
session_id='{session_id}'))ModelFeedbackRequest request =
client.modelFeedback(client.getDefaultModels().logoModel().id())
.withInputId("{input_id}")
.withImageUrl("https://clarifai.com/developer/static/images/model-samples/logo-002.jpg")
.withRegions(
RegionFeedback.make(
Crop.create().top(0.3f).left(0.2f).bottom(0.7f).right(0.8f),
Feedback.ACCURATE
)
.withConceptFeedbacks(ConceptFeedback.forIdAndValue("ai_TG4GHlnf", true)))
.withEventType("annotation")
.withOutputId("{output_id}")
.withEndUserId("{end_user_id}")
.withSessionId("{session_id}");
using System.Collections.Generic;
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs;
using Clarifai.DTOs.Feedbacks;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.ModelFeedback(
client.PublicModels.GeneralModel.ModelID,
imageUrl: "https://samples.clarifai.com/dog.tiff",
inputID: "inputID",
outputID: "{output_id}",
endUserID: "{end_user_id}",
sessionID: "{session_id}",
regions: new List<RegionFeedback>
{
new RegionFeedback(
new Crop(top: 0.3m, left: 0.2m, bottom: 0.7m, right: 0.8m),
Feedback.Accurate,
new List<ConceptFeedback>
{
new ConceptFeedback("ai_8S2Vq3cR", true),
new ConceptFeedback("ai_mFqxrph2", false)
})
})
.ExecuteAsync();
}
}
}
// Coming Sooncurl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"input": {
"id": "{input_id}",
"data": {
"image": {
"url": "https://clarifai.com/developer/static/images/model-samples/logo-002.jpg"
},
"regions":
[
{
"region_info": {
"bounding_box": {
"top_row": 0.3,
"left_col": 0.2,
"bottom_row": 0.7,
"right_col": 0.8
},
"feedback": "accurate"
},
"data": {
"concepts":[
{"id": "ai_TG4GHlnf", "value": true }
]
}
}
]
},
"feedback_info":{
"event_type": "annotation",
"output_id": "{output_id}",
"end_user_id": "{end_user_id}",
"session_id": "{session_id}",
}
}
}'\
https://api.clarifai.com/v2/models/c443119bf2ed4da98487520d01a0b1e3/feedback- JS
- Py
- Java
- C#
- ObjC
- cURL
Response{
"status": {
"code": 10000,
"description": "Ok"
}
}Search Feedback
To send feedback on searches, you can use POST /v2/searches/feedback.
This request is meant to collect the correctly searched inputs, which is usually done by capturing your end user's clicks on the given search results. Your feedback will help us improve our search algorithm.
In the request body for search feedback, you can specify:
- • input id of a correct image ('hit') from the search results
- •
search_idfrom the search API response
Note that the event_type should be set to search_click.
// Coming Soon# Coming SoonSearchesFeedbackRequest request = client.searchesFeedback()
.withId("{input_id}")
.withEndUserId("{end_user_id}")
.withSessionId("{session_id}")
.withEventType("search_click")
.withSearchId("{search_id}");
using System.Threading.Tasks;
using Clarifai.API;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.SearchesFeedback(
inputID: "{input_id}",
endUserID: "{end_user_id}",
sessionID: "{session_id}",
searchID: "{search_id}")
.ExecuteAsync();
}
}
}
// Coming Sooncurl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"input": {
"id": "{input_id}",
"feedback_info": {
"event_type": "search_click",
"search_id": "{output_id}",
"end_user_id": "{end_user_id}",
"session_id": "{session_id}"
}
}
}'\
https://api.clarifai.com/v2/searches/feedback- JS
- Py
- Java
- C#
- ObjC
- cURL
Response{
"status": {
"code": 10000,
"description": "Ok"
}
}Public Models
Clarifai provides a variety of pre-trained models that you can use to make predictions. These models were developed in-house and have been thoroughly tested. Before you train your own model, we suggest trying these out to see if they fit your needs.
You can browse all of our models here.
Applications
API calls are tied to an account and application. Any model you create or search indexes you add images to, will be contained within an application.
You can create as many applications as you want and can edit or delete them as you see fit. Each application has a unique API Key. These are used for authentication. You can learn more about authentication above.
Create an Application
To create an application, head on over to the applications page and press the 'Create a New Application' button.
At a minimum, you'll need to provide an application name and a public model to choose as your Base Workflow. You may also set the default language. If you plan on using a language other than English, you must use the General model. You can learn more about models and languages in the public model guide above.
Base Workflow
When you create an application, you will need to select a Base Workflow, along with App Name and Default Language. You can choose from a select list of public models as your Base Workflow.
The Base Workflow you choose will optimize custom trained models to use the knowledge base from the selected public model. For example, if you're training a custom model around food, you could choose the Food model to gain optimized results. We recommend choosing the General model if you're not sure which public model would best suit your inputs.
Check out our model gallery for a description of the select public models that can be used as a base workflow. Also note that you cannot change the Base Workflow of an app once it's created.
Edit an Application
If at any point you'd like to change the application name, add/remove API Keys, and modify Workflow management for Predict, you may do so by visiting the application page and changing the values.
Delete an Application
If you'd like to delete an application, you may do so at any time by visiting the application page and pressing the 'Delete application' button. You'll be asked to confirm your change. Please note that once you delete an application, we cannot recover it. You will also lose all images, concepts and models associated with that application. Proceed with caution.
Languages
The Clarifai API supports many languages in addition to English. When making a predict api request, you can pass in the language you would like the concepts returned in.
Supported Languages
| Language | Code |
|---|---|
| Arabic (ar) | ar |
| Bengali (bn) | bn |
| Danish (da) | da |
| German (de) | de |
| English (en) | en |
| Spanish (es) | es |
| Finnish (fi) | fi |
| French (fr) | fr |
| Hindi (hi) | hi |
| Hungarian (hu) | hu |
| Italian (it) | it |
| Japanese (ja) | ja |
| Korean (ko) | ko |
| Dutch (nl) | nl |
| Norwegian (no) | no |
| Punjabi (pa) | pa |
| Polish (pl) | pl |
| Portuguese (pt) | pt |
| Russian (ru) | ru |
| Swedish (sv) | sv |
| Turkish (tr) | tr |
| Chinese Simplified (zh) | zh |
| Chinese Traditional (zh-TW) | zh-TW |
Supported Models
The only public model which supports languages other than English is the General model. If you make a predict request using a language other than English on a public model other than General, you will receive an error.
Default Language
When you create a new Application, you must specify a default language. This will be the default language concepts are returned in when you do not explicitly set a language in an API request. You cannot change the default language. You can however change languages per request.
Example Predict API Request
You can predict concepts in a language other then the Application's default, by explicitly passing in the language. Here is how you predict concepts in Chinese:
app.models.predict(Clarifai.GENERAL_MODEL, "https://samples.clarifai.com/metro-north.jpg", {language: 'zh'}).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
m = app.models.get('general-v1.3')
# predict labels in simplified Chinese
m.predict_by_url('https://samples.clarifai.com/metro-north.jpg', lang='zh')
# predict labels in Japanese
m.predict_by_url('https://samples.clarifai.com/metro-north.jpg', lang='ja')
client.predict(client.getDefaultModels().generalModel().id())
.withInputs(ClarifaiInput.forImage("https://samples.clarifai.com/metro-north.jpg"))
.withLanguage("zh")
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Inputs;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.Predict<Concept>(
client.PublicModels.GeneralModel.ModelID,
input: new ClarifaiURLImage("https://samples.clarifai.com/metro-north.jpg"),
language: "zh")
.ExecuteAsync();
}
}
}
// first get the general model.
[app getModelByName:@"general-v1.3" completion:^(ClarifaiModel *model, NSError *error) {
// create input to predict on.
ClarifaiImage *input = [[ClarifaiImage alloc] initWithURL:@"https://samples.clarifai.com/metro-north.jpg"];
// predict with the general model in Chinese.
[model predictOnImages:@[input] withLanguage:@"zh" completion:^(NSArray<ClarifaiOutput *> *outputs, NSError *error) {
for (ClarifaiConcept *concept in outputs[0].concepts) {
NSLog(@"tag: %@", concept.conceptName);
NSLog(@"probability: %f", concept.score);
}
}];
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
}
}
],
"model":{
"output_info":{
"output_config":{
"language":"zh"
}
}
}
}'\
https://api.clarifai.com/v2/models/aaa03c23b3724a16a56b629203edc62c/outputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Response{
"status": {
"code": 10000,
"description": "Ok"
},
"outputs": [
{
"id": "b9f3c12f1534440fa984dc463e491780",
"status": {
"code": 10000,
"description": "Ok"
},
"created_at": "2017-01-31T20:59:27Z",
"model": {
"name": "general-v1.3",
"id": "aaa03c23b3724a16a56b629203edc62c",
"created_at": "2016-03-09T17:11:39Z",
"app_id": null,
"output_info": {
"message": "Show output_info with: GET /models/{model_id}/output_info",
"type": "concept"
},
"model_version": {
"id": "aa9ca48295b37401f8af92ad1af0d91d",
"created_at": "2016-07-13T01:19:12Z",
"status": {
"code": 21100,
"description": "Model trained successfully"
}
}
},
"input": {
"id": "b9f3c12f1534440fa984dc463e491780",
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
}
},
"data": {
"concepts": [
{
"id": "ai_HLmqFqBf",
"name": "铁路列车",
"app_id": null,
"value": 0.9989112
},
{
"id": "ai_fvlBqXZR",
"name": "铁路",
"app_id": null,
"value": 0.9975532
},
{
"id": "ai_Xxjc3MhT",
"name": "运输系统",
"app_id": null,
"value": 0.9959158
},
{
"id": "ai_6kTjGfF6",
"name": "站",
"app_id": null,
"value": 0.992573
},
{
"id": "ai_RRXLczch",
"name": "火车",
"app_id": null,
"value": 0.992556
},
{
"id": "ai_VRmbGVWh",
"name": "旅游",
"app_id": null,
"value": 0.98789215
},
{
"id": "ai_SHNDcmJ3",
"name": "地铁",
"app_id": null,
"value": 0.9816359
},
{
"id": "ai_jlb9q33b",
"name": "通勤",
"app_id": null,
"value": 0.9712483
},
{
"id": "ai_46lGZ4Gm",
"name": "铁路",
"app_id": null,
"value": 0.9690325
},
{
"id": "ai_tr0MBp64",
"name": "交通",
"app_id": null,
"value": 0.9687052
},
{
"id": "ai_l4WckcJN",
"name": "模煳",
"app_id": null,
"value": 0.9667078
},
{
"id": "ai_2gkfMDsM",
"name": "平台",
"app_id": null,
"value": 0.9624243
},
{
"id": "ai_CpFBRWzD",
"name": "城市的",
"app_id": null,
"value": 0.960752
},
{
"id": "ai_786Zr311",
"name": "沒有人",
"app_id": null,
"value": 0.95864904
},
{
"id": "ai_6lhccv44",
"name": "商业",
"app_id": null,
"value": 0.95720303
},
{
"id": "ai_971KsJkn",
"name": "跑道",
"app_id": null,
"value": 0.9494642
},
{
"id": "ai_WBQfVV0p",
"name": "城市",
"app_id": null,
"value": 0.94089437
},
{
"id": "ai_dSCKh8xv",
"name": "快速的",
"app_id": null,
"value": 0.9399334
},
{
"id": "ai_TZ3C79C6",
"name": "马路",
"app_id": null,
"value": 0.93121606
},
{
"id": "ai_VSVscs9k",
"name": "终点站",
"app_id": null,
"value": 0.9230834
}
]
}
}
]
}Example Search By Tag API Request
You can search for concepts in other languages even if the default language of your application is English. When you add inputs to your application, concepts are predicted for every language. Here is an example of searching for '人' which is simplified Chinese for 'people'.
app.inputs.search({
concept: {
name: '人'
},
language: 'ja'
}).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
# search '人' in simplified Chinese
app.inputs.search_by_predicted_concepts(u'人', lang='zh')
client.searchInputs(
SearchClause.matchImageURL(ClarifaiImage.of("https://samples.clarifai.com/metro-north.jpg")))
.withLanguage("zh")
.getPage(1)
.executeSync();
using System.Collections.Generic;
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Searches;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.SearchInputs(
new List<SearchBy>
{
SearchBy.ImageURL("https://samples.clarifai.com/metro-north.jpg")
},
language: "zh")
.Page(1)
.ExecuteAsync();
}
}
}
// create search term with concept you want to search predicted inputs with.
ClarifaiConcept *concept1 = [[ClarifaiConcept alloc] initWithConceptName:@"人"];
ClarifaiSearchTerm *searchTerm = [[ClarifaiSearchTerm alloc] initWithSearchItem:concept1 isInput:NO];
// search will find inputs predicted to be associated with the given concept.
[_app search:@[searchTerm] page:@1 perPage:@20 language:@"zh" completion:^(NSArray<ClarifaiSearchResult *> *results, NSError *error) {
for (ClarifaiSearchResult *result in results) {
NSLog(@"image url: %@", result.mediaURL);
NSLog(@"probability: %f", [result.score floatValue]);
}
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"query": {
"ands": [
{
"output": {
"data": {
"concepts": [
{
"name": "人"
}
]
}
}
}
],
"language": "zh"
}
}'\
https://api.clarifai.com/v2/searches
- JS
- Py
- Java
- C#
- ObjC
- cURL
Example Search Concepts API Request
You can also search for concepts in a different language:
app.concepts.search('人*', 'zh').then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
app.concepts.search(u'人*', lang='zh')
client.searchConcepts("人*")
.withLanguage("zh")
.getPage(1)
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.SearchConcepts(
"人*",
language: "zh")
.Page(1)
.ExecuteAsync();
}
}
}
// Search for all concept names in chinese, beginning with "人".
[_app searchForConceptsByName:@"人*" andLanguage:@"zh" completion:^(NSArray<ClarifaiConcept *> *concepts, NSError *error) {
for (ClarifaiConcept *concept in concepts) {
NSLog(@"tag name: %@", concept.conceptName);
}
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"concept_query": {
"name":"人*",
"language": "zh"
}
}'\
https://api.clarifai.com/v2/concepts/searches
- JS
- Py
- Java
- C#
- ObjC
- cURL
Response{
"status": {
"code": 10000,
"description": "Ok"
},
"concepts": [
{
"id": "ai_l8TKp2h5",
"name": "人",
"created_at": "2016-03-17T11:43:01Z",
"updated_at": "2016-03-17T11:43:01Z",
"app_id": null,
"language": "zh"
},
{
"id": "ai_ZKJ48TFz",
"name": "人",
"created_at": "2016-03-17T11:43:01Z",
"updated_at": "2016-03-17T11:43:01Z",
"app_id": null,
"language": "zh"
},
{
"id": "ai_GlPlRlTZ",
"name": "人为破坏",
"created_at": "2016-03-17T11:43:01Z",
"updated_at": "2016-03-17T11:43:01Z",
"app_id": null,
"language": "zh"
},
{
"id": "ai_8ZsdCrVZ",
"name": "人体模型",
"created_at": "2016-03-17T11:43:01Z",
"updated_at": "2016-03-17T11:43:01Z",
"app_id": null,
"language": "zh"
},
{
"id": "ai_K1KL0zgk",
"name": "人力的",
"created_at": "2016-03-17T11:43:01Z",
"updated_at": "2016-03-17T11:43:01Z",
"app_id": null,
"language": "zh"
},
{
"id": "ai_Tm9d2BZ2",
"name": "人口",
"created_at": "2016-03-17T11:43:01Z",
"updated_at": "2016-03-17T11:43:01Z",
"app_id": null,
"language": "zh"
},
{
"id": "ai_NLF8h1fJ",
"name": "人口",
"created_at": "2016-03-17T11:43:01Z",
"updated_at": "2016-03-17T11:43:01Z",
"app_id": null,
"language": "zh"
},
{
"id": "ai_8bHdFtsg",
"name": "人口",
"created_at": "2016-03-17T11:43:01Z",
"updated_at": "2016-03-17T11:43:01Z",
"app_id": null,
"language": "zh"
},
{
"id": "ai_vLnr3Mcj",
"name": "人孔",
"created_at": "2016-03-17T11:43:01Z",
"updated_at": "2016-03-17T11:43:01Z",
"app_id": null,
"language": "zh"
},
{
"id": "ai_HRt4nfvL",
"name": "人工智能",
"created_at": "2016-03-17T11:43:01Z",
"updated_at": "2016-03-17T11:43:01Z",
"app_id": null,
"language": "zh"
},
{
"id": "ai_Qc3mqxTJ",
"name": "人才",
"created_at": "2016-03-17T11:43:01Z",
"updated_at": "2016-03-17T11:43:01Z",
"app_id": null,
"language": "zh"
},
{
"id": "ai_VFKQ0qD6",
"name": "人物",
"created_at": "2016-03-17T11:43:01Z",
"updated_at": "2016-03-17T11:43:01Z",
"app_id": null,
"language": "zh"
},
{
"id": "ai_Wz8JXXMB",
"name": "人类免疫缺陷病毒",
"created_at": "2016-03-17T11:43:01Z",
"updated_at": "2016-03-17T11:43:01Z",
"app_id": null,
"language": "zh"
},
{
"id": "ai_bzp3Lg81",
"name": "人类的",
"created_at": "2016-03-17T11:43:01Z",
"updated_at": "2016-03-17T11:43:01Z",
"app_id": null,
"language": "zh"
},
{
"id": "ai_dJ15S9s6",
"name": "人群",
"created_at": "2016-03-17T11:43:01Z",
"updated_at": "2016-03-17T11:43:01Z",
"app_id": null,
"language": "zh"
},
{
"id": "ai_MNCVrmml",
"name": "人行天桥",
"created_at": "2016-03-17T11:43:01Z",
"updated_at": "2016-03-17T11:43:01Z",
"app_id": null,
"language": "zh"
},
{
"id": "ai_CChWH41S",
"name": "人行横道",
"created_at": "2016-03-17T11:43:01Z",
"updated_at": "2016-03-17T11:43:01Z",
"app_id": null,
"language": "zh"
},
{
"id": "ai_4lbXrFgT",
"name": "人造",
"created_at": "2016-03-17T11:43:01Z",
"updated_at": "2016-03-17T11:43:01Z",
"app_id": null,
"language": "zh"
},
{
"id": "ai_277LRf4d",
"name": "人造卫星",
"created_at": "2016-03-17T11:43:01Z",
"updated_at": "2016-03-17T11:43:01Z",
"app_id": null,
"language": "zh"
},
{
"id": "ai_H3RDmvSn",
"name": "人造奶油",
"created_at": "2016-03-17T11:43:01Z",
"updated_at": "2016-03-17T11:43:01Z",
"app_id": null,
"language": "zh"
}
]
}Mobile SDK
Clarifai’s Mobile SDK enables machine learning directly on your device, bypassing the traditional requirement of internet connectivity and extensive computing power. It makes it easy to use image and video recognition on device and in real-time.
The Mobile SDK is currently available in Limited Preview. You can get more information about the Limited Preview program here.
Getting Started On Mobile
Since the Mobile SDK is in a Limited Preview program, you need to apply to get access. In order to apply you would require a Clarifai account. Please create one here before proceeding with this guide.
Once you are accepted into the program, you will receive email instructions on how to install the SDK and how to download your first public model on device.
Currently the mobile SDK supports both Objective-C and Swift.
Inputs On Device
The SDK is built around a simple idea. You give inputs (images) to the library and it returns predictions (concepts). You need to add inputs to make predictions on it, and/or to train custom models using the inputs as training data.
The sections below will showcase how to add Inputs on your device and tag them with concepts in order to train a custom model.
Add Device Inputs
All inputs are created from a DataAsset object in the Mobile SDK. A Data Asset is a container for the asset in question, plus metadata related to it. You can create a DataAsset initialized with an Image on Device or from a URL as shown in the example below.
// Only for the Mobile SDK
// Only for the Mobile SDK
// Only for the Mobile SDK
// Initialize CAIImage object from an image URL
NSURL *imageURL = [NSURL urlWithString:@"<your image url>"];
CAIImage *image = [[CAIImage alloc] initWithURL:imageURL];
// Initialize CAIImage object with an image on device
UIImage *deviceImage = [UIImage ...];
CAIImage *localImage = [[CAIImage alloc] initWithImage:deviceImage];
// A Data Asset is a container for the asset in question, plus metadata
// related to it
CAIDataAsset *dataAsset = [[CAIDataAsset alloc] initWithImage:image];
// An input object contains the data asset, temporal information, and
// is a fundamental component to be used by models to train on or
// predict
CAIInput *input = [[CAIInput alloc] initWithDataAsset:dataAsset];
// Only for the Mobile SDK
- JS
- Py
- Java
- ObjC
- cURL
Add Concepts to an Input
You can also add concepts to your inputs if you want to use them for custom training instead of just predictions.
// Only for the Mobile SDK# Only for the Mobile SDK// Only for the Mobile SDK// A concept is a descriptive attribute regarding a data asset. If used
// for training you may want to provide a score.
CAIConcept *concept = [[CAIConcept alloc] initWithId:@"<unique id>"
name:@"<label describing the concept>"];
UIImage *deviceImage = [UIImage ...];
CAIImage *localImage = [[CAIImage alloc] initWithImage:deviceImage];
CAIDataAsset *dataAsset = [[CAIDataAsset alloc] initWithImage:localImage];
[dataAsset addConcepts:@[concept]];
// Only for the Mobile SDK
- JS
- Py
- Java
- ObjC
- cURL
Prediction On Device
Just as with our API, you can use the predict functionality on device with any of our Public Models or any Custom Models that you have on your device.
Predictions generate outputs. An output has a similar structure to an input. It contains a data asset and concepts. The concepts associated with an output contain the predictions and their respective score (degree of confidence.)
// Only for the Mobile SDK
// Only for the Mobile SDK
// Only for the Mobile SDK
// See how to create an input from the examples above
CAIInput *input = [[CAIInput alloc] initWithDataAsset:dataAsset];
// Use the model you want to predict on. This can be a custom trained or one of our public models.
// The model in the sample code below is our General Model.
CAIModel *generalModel = [Clarifai sharedInstance].generalModel;
[generalModel predict:@[input] completionHandler:^(NSArray<CAIOutput *> * _Nullable outputs, NSError * _Nullable error) {
// Iterate through outputs to learn about what has been predicted
for (CAIOutput *output in outputs) {
}
}];
// Only for the Mobile SDK
- JS
- Py
- Java
- ObjC
- cURL
Models On Device
In order for predictions to happen on device, you must have models loaded on the device. For the Mobile SDK, we have Public Models and Custom Models available on device, similar to our web API.
On Device Public Models
Our team has developed a variety of pre-trained public models which our users can make predictions against. As of now, we have enabled our most comprehensive model, the General Model, to be loaded on device. As the product evolves and we start to understand how you're using the SDK, we will be making more of our Public Models available. The sample code below shows how to load the General Model and make a prediction against it.
// Only for the Mobile SDK
// Only for the Mobile SDK
// Only for the Mobile SDK
//Get General Model
CAIModel *generalModel = [Clarifai sharedInstance].generalModel;
// Only for the Mobile SDK
- JS
- Py
- Java
- ObjC
- cURL
Custom Models
There are times when you wish you had a model that sees the world the way you see it. The SDK allows you to do this on device. You can create your own model and train it with your own images and concepts. Once you train it to see how you would like it to see, you can then use that model to make predictions.
Just to quickly refresh, a model contains a group of concepts and it will only see the concepts it contains; you use inputs as the training data and label them with concepts. You do not need many inputs to get started. We recommend starting with 10 and adding more as needed.
For the SDK, we have enabled two types of Custom Models: Local Models and Cloud Models.
Local Models
Local Models are created and trained on the mobile device itself, which can then be used for prediction. All of the data used to train a Local Model stays on device - nothing is sent to the cloud. Local models are great for privacy, but have limits around sharing, and scaling these models to other devices.
Create a Local Model
// Only for the Mobile SDK
// Only for the Mobile SDK
// Only for the Mobile SDK
// Create a custom trained model on device
CAIModel *customModel = [[CAIModel alloc] initWithId:@"<unique id>" name:@"<model name>"];
// Only for the Mobile SDK
- JS
- Py
- Java
- ObjC
- cURL
Train a Local Model
// Only for the Mobile SDK
// Only for the Mobile SDK
// Only for the Mobile SDK
// Images you want to train on
UIImage *localImage = [UIImage ...];
CAIImage *image1 = [[CAIImage alloc] initWithImage:localImage];
UIImage *anotherImage = [UIImage ...];
CAIImage *image2 = [[CAIImage alloc] initWithImage:anotherImage];
// Create concept(s) with scores. 1 for THING, 0 for NOT-THING
CAIConcept *trainingConcept = [[CAIConcept alloc] initWithId:@"<unique id>"
name:@"<Nice name>"
score:1.0];
NSArray<CAIConcept *> *trainingConcepts = @[trainingConcept];
// Create Data Assets and Tag them with Concept(s)
CAIDataAsset *dataAsset = [[CAIDataAsset alloc] initWithImage:image1];
[dataAsset addConcepts:trainingConcepts];
CAIDataAsset *dataAsset2 = [[CAIDataAsset alloc] initWithImage:image2];
[dataAsset2 addConcepts:trainingConcepts];
// Create Inputs
CAIInput *input = [[CAIInput alloc] initWithDataAsset:dataAsset];
CAIInput *input2 = [[CAIInput alloc] initWithDataAsset:dataAsset2];
NSArray<CAIInput *> *trainingInputs = @[input, input2];
[[Clarifai sharedInstance] saveEntities:trainingInputs];
// Create custom trained model on device
CAIModel *customModel = [[CAIModel alloc] initWithId:@"<unique id>" name:@"<model name>"];
// Train the custom model
[customModel trainWithConcepts:trainingConcepts inputs:trainingInputs completionHandler:^(NSError * _Nullable error) {
if (error) {
NSLog(@"Custom model not trained");
return;
}
// The model is ready to be used to make predictions
NSLog(@"Custom model trained");
}];
// Only for the Mobile SDK
- JS
- Py
- Java
- ObjC
- cURL
Cloud Models
Cloud Models are models that are created and trained in Clarifai’s cloud environment and can be viewed in Explorer. You are able to use models that you have created in the past with Clarifai, or can create a brand new one for the SDK. Cloud models are great for deploying the same model on many devices where you would like prediction consistency.
Loading Models from the Cloud
// Only for the Mobile SDK
// Only for the Mobile SDK
// Only for the Mobile SDK
// loads all models created locally and from the cloud.
[[Clarifai sharedInstance] loadEntityType:CAIEntityTypeModel range:NSMakeRange(1, 20) completionHandler:^(NSArray<CAIModel *> * _Nullable models, NSError * _Nullable error) {
// If you have trained models on the cloud, they will show up in the returned list of models.
__block CAIModel *remoteModel = nil;
[models enumerateObjectsUsingBlock:^(CAIModel * _Nonnull model, NSUInteger idx, BOOL * _Nonnull stop) {
if ([model.modelId isEqualToString:@"your-remote-model's-id"]) {
remoteModel = model;
*stop = YES;
}
}];
// Models loaded from the cloud can be used for predicting offline, but re-training will occur on the cloud.
[remoteModel predict:@[input] completionHandler:^(NSArray * _Nullable outputs, NSError * _Nullable error) {
// Iterate through outputs to learn about what has been predicted
for (CAIOutput *output in outputs) {
}
}];
}];
// Only for the Mobile SDK
- JS
- Py
- Java
- ObjC
- cURL
Inputs
The API is built around a simple idea. You send inputs (images) to the service and it returns predictions. In addition to receiving predictions on inputs, you can also 'save' inputs and their predictions to later search against. You can also 'save' inputs with concepts to later train your own model.
Add Inputs
You can add inputs one by one or in bulk. If you do send bulk, you are limited to sending 128 inputs at a time.
Images can either be publicly accessible URLs or file bytes. If you are sending file bytes, you must use base64 encoding.
You are encouraged to send inputs with your own id. This will help you later match the input to your own database. If you do not send an id, one will be created for you.
Add an input using a publicly accessible URL
app.inputs.create({
url: "https://samples.clarifai.com/metro-north.jpg"
}).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
app.inputs.create_image_from_url("https://samples.clarifai.com/metro-north.jpg")
client.addInputs()
.plus(ClarifaiInput.forImage("https://samples.clarifai.com/metro-north.jpg"))
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Inputs;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.AddInputs(
new ClarifaiURLImage("https://samples.clarifai.com/metro-north.jpg"))
.ExecuteAsync();
}
}
}
ClarifaiImage *image = [[ClarifaiImage alloc] initWithURL:@"https://samples.clarifai.com/metro-north.jpg"];
[app addInputs:@[image] completion:^(NSArray<ClarifaiInput *> *inputs, NSError *error) {
NSLog(@"inputs: %@", inputs);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
}
}
]
}'\
https://api.clarifai.com/v2/inputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Add an input using bytes
The data must be base64 encoded. When you add a base64 image to our servers, a copy will be stored and hosted on our servers. If you already have an image hosting service we recommend using it and adding images via the url parameter.
app.inputs.create({
base64: "Zvfauhti4D..."
}).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
# add from filename
app.inputs.create_image_from_filename(filename)
# add from base64 bytes
app.inputs.create_image_from_base64(base64_bytes)
client.addInputs()
.plus(ClarifaiInput.forImage(new File("image.png")))
.executeSync();
using System.IO;
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Inputs;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.AddInputs(
new ClarifaiFileImage(File.ReadAllBytes("image.png")))
.ExecuteAsync();
}
}
}
ClarifaiImage *imageFromImage = [[ClarifaiImage alloc] initWithImage:@"dress.jpg"];
[app addInputs:@[imageFromImage] completion:^(NSArray<ClarifaiInput *> *inputs, NSError *error) {
NSLog(@"inputs: %@", inputs);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"data": {
"image": {
"base64": '"`base64 /home/user/image.jpeg`"'"
}
}
}
]
}'\
https://api.clarifai.com/v2/inputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Add multiple inputs with ids
app.inputs.create([
{
url: "https://samples.clarifai.com/metro-north.jpg",
id: 'train1'
},
{
url: "https://samples.clarifai.com/puppy.jpeg",
id: 'puppy1'
}
]).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
from clarifai.rest import Image as ClImage
app = ClarifaiApp(api_key='YOUR_API_KEY')
img1 = ClImage(url="https://samples.clarifai.com/metro-north.jpg", image_id="train1")
img2 = ClImage(url="https://samples.clarifai.com/puppy.jpeg", image_id="puppy1")
app.inputs.bulk_create_images([img1, img2])
client.addInputs()
.plus(
ClarifaiInput.forImage("https://samples.clarifai.com/metro-north.jpg")
.withConcepts(Concept.forID("id1")),
ClarifaiInput.forImage("https://samples.clarifai.com/wedding.jpg")
.withConcepts(Concept.forID("id2"))
)
.executeSync();
using System.Collections.Generic;
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Inputs;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.AddInputs(
new ClarifaiURLImage(
"https://samples.clarifai.com/metro-north.jpg",
positiveConcepts: new List<Concept> {new Concept("id1")}),
new ClarifaiURLImage(
"https://samples.clarifai.com/wedding.jpg",
positiveConcepts: new List<Concept> {new Concept("id2")}))
.ExecuteAsync();
}
}
}
ClarifaiImage *train = [[ClarifaiImage alloc] initWithURL:@"https://samples.clarifai.com/metro-north.jpg"];
train.inputID = @"train";
ClarifaiImage *puppy = [[ClarifaiImage alloc] initWithURL:@"https://samples.clarifai.com/puppy.jpeg"];
puppy.inputID = @"puppy";
[app addInputs:@[train, puppy] completion:^(NSArray<ClarifaiInput *> *inputs, NSError *error) {
NSLog(@"inputs: %@", inputs);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
},
"id": "{id1}"
},
{
"data": {
"image": {
"url": "https://samples.clarifai.com/puppy.jpeg"
}
},
"id": "{id2}"
}
]
}'\
https://api.clarifai.com/v2/inputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Add Inputs With Concepts
If you would like to add an input with concepts, you can do so like this. Concepts play an important role in creating your own models using your own concepts. You can learn more about creating your own models above. Concepts also help you search for inputs. You can learn more about search here.
When you add a concept to an input, you need to indicate whether the concept is present in the image or if it is not present.
You can add inputs with concepts as either a URL or bytes.
app.inputs.create({
url: "https://samples.clarifai.com/puppy.jpeg",
concepts: [
{
id: "boscoe",
value: true
}
]
}).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
from clarifai.rest import Image as ClImage
app = ClarifaiApp(api_key='YOUR_API_KEY')
# add by url
app.inputs.create_image_from_url("https://samples.clarifai.com/puppy.jpeg", concepts=['boscoe'])
# add by base64 bytes
app.inputs.create_image_from_base64(base64_bytes, concepts=['boscoe'])
# add by raw bytes
app.inputs.create_image_from_bytes(raw_bytes, concepts=['boscoe'])
# add by local file
app.inputs.create_image_from_filename(local_filename, concepts=['boscoe'])
# add multiple with concepts
img1 = ClImage(url="https://samples.clarifai.com/puppy.jpeg", concepts=['boscoe'], not_concepts=['our_wedding'])
img2 = ClImage(url="https://samples.clarifai.com/wedding.jpg", concepts=['our_wedding'], not_concepts=['cat','boscoe'])
app.inputs.bulk_create_images([img1, img2])
client.addInputs()
.plus(ClarifaiInput.forImage("https://samples.clarifai.com/puppy.jpeg")
.withConcepts(
// To mark a concept as being absent, chain `.withValue(false)`
Concept.forID("boscoe")
)
)
.executeSync();
using System.Collections.Generic;
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Inputs;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.AddInputs(
new ClarifaiURLImage(
"https://samples.clarifai.com/puppy.jpeg",
// To mark a concept as being absent, use negativeConcepts
positiveConcepts: new List<Concept> {new Concept("boscoe")}))
.ExecuteAsync();
}
}
}
ClarifaiImage *puppy = [[ClarifaiImage alloc] initWithURL:@"https://samples.clarifai.com/puppy.jpeg"
andConcepts:@[@"cute puppy"]];
[app addInputs:@[puppy] completion:^(NSArray<ClarifaiInput *> *inputs, NSError *error) {
NSLog(@"inputs: %@", inputs);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"data": {
"image": {
"url": "https://samples.clarifai.com/puppy.jpeg"
},
"concepts":[
{
"id": "boscoe",
"value": true
}
]
}
}
]
}'\
https://api.clarifai.com/v2/inputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Add Inputs With Custom Metadata
In addition to adding an input with concepts, you can also add an input with custom metadata. This metadata will then be searchable. Metadata can be any arbitrary JSON.
app.inputs.create({
url: "https://samples.clarifai.com/puppy.jpeg",
metadata: {id: 'id001', type: 'plants', size: 100}
}).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
from clarifai.rest import Image as ClImage
app = ClarifaiApp(api_key='YOUR_API_KEY')
# metadata must be defined as JSON object
metadata = {'id':'id001', 'type':'plants', 'size':100}
# adding metadata along with url, filename, etc
app.inputs.create_image_from_url(url="https://samples.clarifai.com/puppy.jpeg", metadata=metadata)
app.inputs.create_image_from_filename(filename="aa.jpg", metadata=metadata)
# define an image with metadata for bulk import
img = Image(url="", metadata=metadata)
app.inputs.bulk_create_images([img])
final JsonObject metadata = new JsonObject();
metadata.addProperty("isPuppy", true);
client.addInputs()
.plus(
ClarifaiInput.forImage("https://samples.clarifai.com/puppy.jpeg")
.withMetadata(metadata)
).executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Inputs;
using Newtonsoft.Json.Linq;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
var metadata = new JObject(
new JProperty("key1", "val1"),
new JProperty("key2", "val2"));
await client.AddInputs(
new ClarifaiURLImage(
"https://samples.clarifai.com/puppy.jpeg",
metadata: metadata))
.ExecuteAsync();
}
}
}
ClarifaiImage *puppy = [[ClarifaiImage alloc] initWithURL:@"https://samples.clarifai.com/puppy.jpeg"
andConcepts:@[@"cute puppy"]];
puppy.metadata = @{@"my_key": @[@"my",@"values"], @"cuteness": @"extra-cute"};
[app addInputs:@[puppy] completion:^(NSArray<ClarifaiInput *> *inputs, NSError *error) {
NSLog(@"inputs: %@", inputs);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"data": {
"image": {
"url": "https://samples.clarifai.com/puppy.jpeg",
"allow_duplicate_url": true
},
"metadata": {
"key": "value",
"list":[1,2,3]
}
}
}
]
}'\
https://api.clarifai.com/v2/inputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Add Input With A Crop
When adding an input, you can specify crop points. The API will crop the image and use the resulting image. Crop points are given as percentages from the top left point in the order of top, left, bottom and right.
As an example, if you provide a crop as 0.2, 0.4, 0.3, 0.6 that means the cropped image will have a top edge that starts 20% down from the original top edge, a left edge that starts 40% from the original left edge, a bottom edge that starts 30% from the original top edge and a right edge that starts 60% from the original left edge.
app.inputs.create(
{
"url": "https://samples.clarifai.com/metro-north.jpg",
"crop": [0.2, 0.4, 0.3, 0.6]
}
).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
# add url with crop
app.inputs.create_image_from_url(url="https://samples.clarifai.com/metro-north.jpg", crop=[0.2, 0.4, 0.3, 0.6])
client.addInputs()
.plus(
ClarifaiInput.forInputValue(
ClarifaiURLImage.of("https://samples.clarifai.com/metro-north.jpg")
.withCrop(Crop.create()
.top(0.2F)
.left(0.4F)
.bottom(0.3F)
.right(0.6F)
)
)
)
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs;
using Clarifai.DTOs.Inputs;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.AddInputs(
new ClarifaiURLImage(
"https://samples.clarifai.com/metro-north.jpg",
crop: new Crop(top: 0.2M, left: 0.4M, bottom: 0.3M, right: 0.6M)))
.ExecuteAsync();
}
}
}
ClarifaiCrop *crop = [[ClarifaiCrop alloc] initWithTop:0.2 left:0.3 bottom:0.7 right:0.8];
ClarifaiImage *puppy = [[ClarifaiImage alloc] initWithURL:@"https://samples.clarifai.com/puppy.jpeg" andCrop:crop];
[app addInputs:@[puppy] completion:^(NSArray<ClarifaiInput *> *inputs, NSError *error) {
NSLog(@"inputs: %@", inputs);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg",
"crop": [0.2, 0.4, 0.3, 0.6]
}
}
}
]
}'\
https://api.clarifai.com/v2/inputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Get Inputs
You can list all the inputs (images) you have previously added either for search or train.
If you added inputs with concepts, they will be returned in the response as well.
This request is paginated.
app.inputs.list({page: 1, perPage: 20}).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
# this is a generator
app.inputs.get_all()
# get a page of inputs
app.inputs.get_by_page(page=1, per_page=20)
client.getInputs() // optionally takes a perPage parameter
.getPage(1)
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.GetInputs()
.Page(1)
.ExecuteAsync();
}
}
}
[app getInputsOnPage:1 pageSize:20 completion:^(NSArray<ClarifaiInput *> *inputs, NSError *error) {
NSLog(@"inputs: %@", inputs);
}];
curl -X GET \
-H "Authorization: Key YOUR_API_KEY" \
https://api.clarifai.com/v2/inputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Get Input By Id
If you'd like to get a specific input by id, you can do that as well.
app.inputs.get({id}).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
image = app.inputs.get(input_id)
client.getInputByID("{id}").executeSync();
using System.Threading.Tasks;
using Clarifai.API;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.GetInput("{id}")
.ExecuteAsync();
}
}
}
[_app getInput:input_id completion:^(ClarifaiInput *input, NSError *error) {
NSLog(@"input": %@, input);
}];
curl -X GET \
-H "Authorization: Key YOUR_API_KEY" \
https://api.clarifai.com/v2/inputs/{id}
- JS
- Py
- Java
- C#
- ObjC
- cURL
Get Inputs Status
If you add inputs in bulk, they will process in the background. You can get the status of all your inputs (processed, to_process and errors) like this:
app.inputs.getStatus().then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
app.inputs.check_status()
client.getInputsStatus().executeSync();
using System.Threading.Tasks;
using Clarifai.API;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.GetInputsStatus()
.ExecuteAsync();
}
}
}
[app getInputsStatus:^(int numProcessed, int numToProcess, int errors, NSError *error) {
NSLog(@"number of inputs processed: %d", numProcessed);
}];
curl -X GET \
-H "Authorization: Key YOUR_API_KEY" \
https://api.clarifai.com/v2/inputs/status
- JS
- Py
- Java
- C#
- ObjC
- cURL
Update Input With Concepts
To update an input with a new concept, or to change a concept value from true/false, you can do that:
app.inputs.mergeConcepts([
{
id: "{id}",
concepts: [
{
id: "tree"
},
{
id: "water",
value: false
}
]
},
])
// or if you have an input instance
app.inputs.get({id}).then(
function(input) {
input.mergeConcepts([
{
id: "tree",
value: true
},
{
id: "water",
value: false
}
])
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
app.inputs.merge_concepts('{id}', concepts=['tree'], not_concepts=['water'])
client.mergeConceptsForInput("{input_id}")
.plus(
Concept.forID("tree"),
Concept.forID("water").withValue(false)
)
.executeSync();
using System.Collections.Generic;
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.API.Requests.Models;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.ModifyInput(
"{input_id}",
ModifyAction.Merge,
positiveConcepts: new List<Concept> {new Concept("tree")},
negativeConcepts: new List<Concept> {new Concept("water")})
.ExecuteAsync();
}
}
}
ClarifaiConcept *concept = [[ClarifaiConcept alloc] initWithConceptName:@"cute cat"];
[_app addConcepts:@[concept] forInputWithID:@"{id}" completion:^(ClarifaiInput *input, NSError *error) {
NSLog(@"input: %@", input);
}];
curl -X PATCH \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"id": "{id}",
"data": {
"concepts": [
{
"id": "tree",
"value": true
},
{
"id": "water",
"value": false
}
]
}
}
],
"action":"merge"
}'\
https://api.clarifai.com/v2/inputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Delete Concepts From An Input
To remove concepts that were already added to an input, you can do this:
app.inputs.deleteConcepts([
{
id: "{id}",
concepts: [
{
id: "tree"
},
{
id: "water",
value: false
}
]
},
])
// or if you have an input instance
app.inputs.get({id}).then(
function(input) {
input.deleteConcepts([
{
id: "tree",
value: true
},
{
id: "water",
value: false
}
])
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
app.inputs.delete_concepts({id}, concepts=['tree', 'water'])
client.removeConceptsForInput("{input_id}")
.plus(
Concept.forID("tree"),
Concept.forID("water")
)
.executeSync();
using System.Collections.Generic;
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.API.Requests.Models;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.ModifyInput(
"{input_id}",
ModifyAction.Remove,
positiveConcepts: new List<Concept> {new Concept("tree")},
negativeConcepts: new List<Concept> {new Concept("water")})
.ExecuteAsync();
}
}
}
ClarifaiConcept *concept = [[ClarifaiConcept alloc] initWithConceptName:@"cute cat"];
[app deleteConcepts:@[concept] forInputWithID:{id} completion:^(ClarifaiInput *input, NSError *error) {
NSLog(@"input: %@", input);
}];
curl -X PATCH \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"id":"",
"data": {
"concepts":[
{"id":"mattid2", "value":true},
{"id":"ferrari", "value":false}
]
}
}
],
"action":"remove"
}'\
https://api.clarifai.com/v2/inputs/
- JS
- Py
- Java
- C#
- ObjC
- cURL
Bulk Update Inputs With Concepts
You can update an existing input using its Id. This is useful if you'd like to add concepts to an input after its already been added.
app.inputs.mergeConcepts([
{
id: "{id1}",
concepts: [
{
id: "tree",
value: true
},
{
id: "water",
value: false
}
]
},
{
id: "{id2}",
concepts: [
{
id: "animal",
value: true
},
{
id: "fruit",
value: false
}
]
}
]).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
input_ids = ["{id1}", "{id2}"]
concept_pairs = [
[('tree', True), ('water', False)],
[('animal', True), ('fruit', False)],
]
app.inputs.bulk_merge_concepts(input_ids, concept_pairs)
// Coming soon
// Coming soon
ClarifaiConcept *newConcept = [[ClarifaiConcept alloc] initWithConceptID:@"tree"];
[_app getInput:@"{input_id}" completion:^(ClarifaiInput *input, NSError *error) {
// Add tree concept to each current input's concept list.
NSMutableArray *newConceptList = [NSMutableArray arrayWithArray:input.concepts];
[newConceptList addObject:newConcept];
input.concepts = newConceptList;
// Merge the new list for one or more inputs.
[_app mergeConceptsForInputs:@[input] completion:^(NSArray<ClarifaiInput *> *inputs, NSError *error) {
NSLog(@"updated inputs: %@", inputs);
}];
}];
curl -X PATCH \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"id": "{id1}",
"data": {
"concepts": [
{
"id": "tree",
"value": true
},
{
"id": "water",
"value": false
}
]
}
},
{
"id": "{id2}",
"data": {
"concepts": [
{
"id": "tree",
"value": true
},
{
"id": "water",
"value": false
}
]
}
}
],
"action":"merge"
}'\
https://api.clarifai.com/v2/inputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Bulk Delete Concepts From A List Of Inputs
You can bulk delete multiple concepts from a list of inputs:
app.inputs.deleteConcepts([
{
id: "{id1}",
concepts: [
{ id: "tree" },
{ id: "water" }
]
},
{
id: "{id2}",
concepts: [
{ id: "animal" },
{ id: "fruit" }
]
}
]).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
input_ids = ["{id1}", "{id2}"]
concept_pairs = [
['tree', 'water'],
['animal', 'fruit']
]
app.inputs.bulk_delete_concepts(input_ids, concept_pairs)
// Coming soon
// Coming soon
ClarifaiConcept *concept = [[ClarifaiConcept alloc] initWithConceptName:@"cute cat"];
[app deleteConcepts:@[concept] forInputWithID:input_id completion:^(ClarifaiInput *input, NSError *error) {
NSLog(@"input: %@", input);
}];
curl -X PATCH \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"id": "{id1}",
"data": {
"concepts":[
{
"id": "mattid2"
},
{
"id": "ferrari"
}
]
}
},
{
"id": "{id2}",
"data": {
"concepts":[
{
"id": "mattid2"
},
{
"id": "ferrari"
}
]
}
}
],
"action":"remove"
}'\
https://api.clarifai.com/v2/inputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Delete Input By Id
You can delete a single input by id:
app.inputs.delete({id}).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
app.inputs.delete("{id}")
client.deleteInput("{id}")
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.DeleteInputs("{id}")
.ExecuteAsync();
}
}
}
[_app deleteInputsByIDList:@[{id1}] completion:^(NSError *error) {
NSLog(@"input has been deleted");
}];
curl -X DELETE \
-H "Authorization: Key YOUR_API_KEY" \
https://api.clarifai.com/v2/inputs/{id}
- JS
- Py
- Java
- C#
- ObjC
- cURL
Delete A List Of Inputs
You can also delete multiple inputs in one API call. This will happen asynchronously.
app.inputs.delete([{id1}, {id2}]).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
app.delete(["{id1}", "{id2}"])
client.deleteInputsBatch()
.plus("{id1}", "{id2}");
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.DeleteInputs("{id1}", "{id2}")
.ExecuteAsync();
}
}
}
[_app deleteInputsByIDList:@[{id1}, {id2}] completion:^(NSError *error) {
NSLog(@"inputs have been deleted");
}];
curl -X DELETE \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"ids":["{id1}","{id2}"]
}'\
https://api.clarifai.com/v2/inputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Delete All Inputs
If you would like to delete all inputs from an application, you can do that as well. This will happen asynchronously.
app.inputs.delete().then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
app.inputs.delete_all()
client.deleteAllInputs().executeSync();
using System.Threading.Tasks;
using Clarifai.API;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.DeleteAllInputs()
.ExecuteAsync();
}
}
}
[app deleteAllInputs:^(ClarifaiInput *input, NSError *error) {
NSLog(@"all inputs have been deleted");
}];
curl -X DELETE \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"delete_all":true
}'\
https://api.clarifai.com/v2/inputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Models
There are many methods to work with models.
Create Model
You can create your own model and train it with your own images and concepts. Once you train it to see how you would like it to see, you can then use that model to make predictions.
When you create a model you give it a name and an id. If you don't supply an id, one will be created for you.
app.models.create("petsID").then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
app.models.create('petsID')
client.createModel("petsID").executeSync();
using System.Threading.Tasks;
using Clarifai.API;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.CreateModel("petsID")
.ExecuteAsync();
}
}
}
[_app createModel:nil name:@"petsModel" modelID:@"petsID" conceptsMutuallyExclusive:NO closedEnvironment:NO completion:^(ClarifaiModel *model, NSError *error) {
NSLog(@"model: %@", model);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"model": {
"id": "petsID"
}
}'\
https://api.clarifai.com/v2/models
- JS
- Py
- Java
- C#
- ObjC
- cURL
Create Model With Concepts
You can also create a model and initialize it with the concepts it will contain. You can always add and remove concepts later.
app.models.create(
"petsID",
[
{ "id": "boscoe" }
]
).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.create('petsID', concepts=['boscoe'])
client.createModel("petsID")
.withOutputInfo(ConceptOutputInfo.forConcepts(
Concept.forID("boscoe")
))
.executeSync();
using System.Collections.Generic;
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.CreateModel(
"petsID",
concepts: new List<Concept> {new Concept("boscoe")})
.ExecuteAsync();
}
}
}
[_app createModel:@[@"cat", @"dog"] name:@"petsModel" modelID:@"petsID" conceptsMutuallyExclusive:NO closedEnvironment:NO completion:^(ClarifaiModel *model, NSError *error) {
NSLog(@"model: %@", model);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"model": {
"id": "petsID",
"output_info": {
"data": {
"concepts": [
{
"id": "boscoe"
}
]
},
"output_config": {
"concepts_mutually_exclusive": false,
"closed_environment":false
}
}
}
}'\
https://api.clarifai.com/v2/models
- JS
- Py
- Java
- C#
- ObjC
- cURL
Add Concepts To A Model
You can add concepts to a model at any point. As you add concepts to inputs, you may want to add them to your model.
app.models.initModel({model_id}).then(function(model) {
updateModel,
function(err) {
// there was an error
}
});
function updateModel(model) {
model.mergeConcepts({"id": "boscoe"}).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
}
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('{model_id}')
model.add_concepts(['boscoe'])
client.modifyModel("")
.withConcepts(Action.MERGE, Concept.forID("dogs"))
.executeSync();
// Or, if you have a ConceptModel object, you can do it in an OO fashion
final ConceptModel model = client.getModelByID("{model_id}").executeSync().get().asConceptModel();
model.modify()
.withConcepts(Action.MERGE, Concept.forID("dogs"))
.executeSync();
using System.Collections.Generic;
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.API.Requests.Models;
using Clarifai.DTOs.Models;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.ModifyModel(
"petsID",
ModifyAction.Merge,
concepts: new List<Concept> {new Concept("dogs")})
.ExecuteAsync();
// Or, if you have a ConceptModel object, you can do it in an OO fashion
ConceptModel model = (ConceptModel) (
await client.GetModel<Concept>("petsID")
.ExecuteAsync()).Get();
await model.Modify(
ModifyAction.Merge,
concepts: new List<Concept> {new Concept("dogs")})
.ExecuteAsync();
}
}
}
ClarifaiConcept *concept = [[ClarifaiConcept alloc] initWithConceptName:@"dress"];
[app addConcepts:@[concept] toModelWithID:@"{model_id}" completion:^(ClarifaiModel *model, NSError *error) {
NSLog(@"model: %@", model);
}];
curl -X PATCH \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"models": [
{
"id": "{model_id}",
"output_info": {
"data": {
"concepts": [
{
"id": "dogs"
}
]
}
}
}
],
"action": "merge"
}'\
https://api.clarifai.com/v2/models/
- JS
- Py
- Java
- C#
- ObjC
- cURL
Remove Concepts From A Model
Conversely, if you'd like to remove concepts from a model, you can also do that.
app.models.initModel({model_id}).then(function(model) {
updateModel,
function(err) {
// there was an error
}
});
function updateModel(model) {
model.deleteConcepts({"id": "boscoe"}).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
}
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('{model_id}')
model.delete_concepts(['boscoe'])
client.modifyModel("")
.withConcepts(Action.REMOVE, Concept.forID("dogs"))
.executeSync();
// Or, if you have a ConceptModel object, you can do it in an OO fashion
final ConceptModel model = client.getModelByID("").executeSync().get().asConceptModel();
model.modify()
.withConcepts(Action.REMOVE, Concept.forID("dogs"))
.executeSync();
using System.Collections.Generic;
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.API.Requests.Models;
using Clarifai.DTOs.Models;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.ModifyModel(
"petsID",
ModifyAction.Remove,
concepts: new List<Concept> {new Concept("dogs")})
.ExecuteAsync();
// Or, if you have a ConceptModel object, you can do it in an OO fashion
ConceptModel model = (ConceptModel) (
await client.GetModel<Concept>("petsID")
.ExecuteAsync())
.Get();
await model.Modify(
ModifyAction.Remove,
concepts: new List<Concept> {new Concept("dogs")})
.ExecuteAsync();
}
}
}
ClarifaiConcept *concept = [[ClarifaiConcept alloc] initWithConceptName:@"dress"];
[app deleteConcepts:@[concept] fromModelWithID:@"{model_id}" completion:^(ClarifaiModel *model, NSError *error) {
NSLog(@"model: %@", model);
}];
curl -X PATCH \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"models": [
{
"id": "{model_id}",
"output_info": {
"data": {
"concepts": [
{
"id": "dogs"
}
]
}
}
}
],
"action": "remove"
}'\
https://api.clarifai.com/v2/models/
- JS
- Py
- Java
- C#
- ObjC
- cURL
Update Concept Name
The code below showcases how to update a concept's name given its id.
** Coming Soonfrom clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
app.concepts.update(concept_id='concept_id', concept_name='new_concept_name')
** Coming Soon
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.ModifyConcepts(
new Concept("{concept-id}", name: "{new-concept-name}"))
.ExecuteAsync();
}
}
}
** Coming Soon
curl -X PATCH \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"concepts": [
{
"id": "{concept_id}",
"name": "{new_concept_name}"
}
],
"action": "overwrite"
}'\
https://api.clarifai.com/v2/concepts
- JS
- Py
- Java
- C#
- ObjC
- cURL
Update Model Name and Configuration
Here we will change the model name to 'newname' and the model's configuration to have concepts_mutually_exclusive=true and closed_environment=true.
app.models.initModel({model_id}).then(
updateModel,
function(err) {
// there was an error
}
);
function updateModel(model) {
model.update({
name: 'newname',
conceptsMutuallyExclusive: true,
closedEnvironment: true,
concepts: ['birds', 'hurd']
}).then(
}
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('{model_id}')
# only update the name
model.update(model_name="newname")
# update the model attributes
model.update(concepts_mutually_exclusive=True, closed_environment=True)
# update more together
model.update(model_name="newname",
concepts_mutually_exclusive=True, closed_environment=True)
# update attributes together with concepts
model.update(model_name="newname",
concepts_mutually_exclusive=True,
concepts=["birds", "hurd"])
client.modifyModel("")
.withName("newname")
.withConceptsMutuallyExclusive(true)
.withClosedEnvironment(true)
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.ModifyModel(
"someModel",
name: "{newName}",
areConceptsMutuallyExclusive: true,
isEnvironmentClosed: false)
.ExecuteAsync();
}
}
}
[_app updateModel:@"{model_id}" name:@"newName" conceptsMutuallyExclusive:NO closedEnvironment:NO completion:^(ClarifaiModel *model, NSError *error) {
NSLog(@"model: %@", model);
}];
curl -X PATCH \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"models": [
{
"id": "",
"name": "newname",
"output_info": {
"output_config": {
"concepts_mutually_exclusive": true,
"closed_environment": true
}
}
}
],
"action": "overwrite"
}'\
https://api.clarifai.com/v2/models/
- JS
- Py
- Java
- C#
- ObjC
- cURL
Get Models
To get a list of all models including models you've created as well as public models:
app.models.list().then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
# this is a generator
app.models.get_all()
client.getModels().getPage(1).executeSync();
using System.Threading.Tasks;
using Clarifai.API;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.GetModels()
.Page(1)
.ExecuteAsync();
}
}
}
[_app getModels:1 resultsPerPage:30 completion:^(NSArray<ClarifaiModel *> *models, NSError *error) {
NSLog(@"models: %@", models);
}];
curl -X GET \
-H "Authorization: Key YOUR_API_KEY" \
https://api.clarifai.com/v2/models
- JS
- Py
- Java
- C#
- ObjC
- cURL
Get Model By Id
All models have unique Ids. You can get a specific model by its id:
app.models.get({model_id}).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
# get model by id
model = app.models.get(model_id')
# get model by name
model = app.models.get('my_model1')
client.getModelByID("{model_id}").executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
// Change Concept to whatever the model's type is.
await client.GetModel<Concept>("{model_id}")
.ExecuteAsync();
}
}
}
[_app getModel:@"model_id" completion:^(ClarifaiModel *model, NSError *error) {
NSLog(@"model: %@", model);
}];
curl -X GET \
-H "Authorization: Key YOUR_API_KEY" \
https://api.clarifai.com/v2/models/{model_id}
- JS
- Py
- Java
- C#
- ObjC
- cURL
Get Model Output Info By Id
The output info of a model lists what concepts it contains.
app.models.initModel({model_id}).then(
getModelOutputInfo,
handleError
);
function getModelOutputInfo(model) {
model.getOutputInfo().then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
}
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('my_model1')
model.get_info(verbose=True)
client.getModelByID("{model_id}").executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Models;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
var r = await client.GetModel<Concept>("{model-id}")
.ExecuteAsync();
var model = (ConceptModel) r.Get();
var outputInfo = model.OutputInfo;
}
}
}
[_app getModelByID:@"{model_id}" completion:^(ClarifaiModel *model, NSError *error) {
NSLog(@"model: %@", model);
}];
curl -X GET \
-H "Authorization: Key YOUR_API_KEY" \
https://api.clarifai.com/v2/models/{model_id}/output_info
- JS
- Py
- Java
- C#
- ObjC
- cURL
List Model Versions
Every time you train a model, it creates a new version. You can list all the versions created.
app.models.initModel('{id}').then(
function(model) {
model.getVersions().then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('{id}')
model.list_versions()
client.getModelVersions("{model_id}").getPage(1).executeSync();
using System.Threading.Tasks;
using Clarifai.API;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.GetModelVersions("{model_id}")
.ExecuteAsync();
}
}
}
[app listVersionsForModel:@"{model_id}" page:1 resultsPerPage:30 completion:^(NSArray<ClarifaiModelVersion *> *versions, NSError *error) {
NSLog(@"versions: %@", versions);
}];
curl -X GET \
-H "Authorization: Key YOUR_API_KEY" \
https://api.clarifai.com/v2/models/{model_id}/versions
- JS
- Py
- Java
- C#
- ObjC
- cURL
Get Model Version By Id
To get a specific model version, you must provide the model_id as well as the version_id. You can inspect the model version status to determine if your model is trained or still training.
app.models.initModel('{id}').then(
function(model) {
model.getVersion('{version_id}').then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('{id}')
model.get_version('{version_id}')
client.getModelVersionByID("{model_id}", "{version_id}").executeSync();
// Or in a more object-oriented manner:
client.getModelByID("{model_id}")
.executeSync().get() // Returns Model object
.getVersionByID("{version_id}").executeSync();
using System.Threading.Tasks;
using Clarifai.API;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.GetModelVersion("{model_id}", "{version_id}")
.ExecuteAsync();
}
}
}
[app getVersionForModel:@"{model_id}" versionID:{version_id} completion:^(ClarifaiModelVersion *version, NSError *error) {
NSLog(@"version: %@", version);
}];
curl -X GET \
-H "Authorization: Key YOUR_API_KEY" \
https://api.clarifai.com/v2/models/{model_id}/versions/{version_id}
- JS
- Py
- Java
- C#
- ObjC
- cURL
Get Model Training Inputs
You can list all the inputs that were used to train the model.
app.models.initModel('{id}').then(
function(model) {
model.getInputs().then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('{id}')
model.get_inputs()
client.getModelInputs("{model_id}").getPage(1).executeSync();
using System.Threading.Tasks;
using Clarifai.API;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.GetModelInputs("{model_id}")
.Page(1)
.ExecuteAsync();
}
}
}
[app listTrainingInputsForModel:@"{model_id}" page:1 resultsPerPage:30 completion:^(NSArray<ClarifaiInput *> *inputs, NSError *error) {
NSLog(@"inputs: %@", inputs);
}];
curl -X GET \
-H "Authorization: Key YOUR_API_KEY" \
https://api.clarifai.com/v2/models/{model_id}/inputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Get Model Training Inputs By Version
You can also list all the inputs that were used to train a specific model version.
app.models.initModel({id: '{model_id}', version: '{version_id}'}).then(
function(model) {
model.getInputs().then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('{id}')
model.get_inputs('{version_id}')
client.getModelInputs("{model_id}")
.fromSpecificModelVersion("{version_id}")
.getPage(1)
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.GetModelInputs("{model_id}", "{version_id}")
.Page(1)
.ExecuteAsync();
}
}
}
[_app listTrainingInputsForModel:@"{model_id}" page:1 resultsPerPage:30 completion:^(NSArray<ClarifaiInput *> *inputs, NSError *error) {
NSLog(@"inputs: %@", inputs);
}];
curl -X GET \
-H "Authorization: Key YOUR_API_KEY" \
https://api.clarifai.com/v2/models/{model_id}/versions/{version_id}/inputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Delete A Model
You can delete a model using the model_id.
app.models.delete('{id}').then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
app.models.delete('{id}')
client.deleteModel("{model_id}").executeSync();
using System.Threading.Tasks;
using Clarifai.API;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.DeleteModel("{model_id}")
.ExecuteAsync();
}
}
}
[app deleteModel:@"{model_id}" completion:^(NSError *error) {
NSLog(@"model is deleted");
}];
curl -X DELETE \
-H "Authorization: Key YOUR_API_KEY" \
https://api.clarifai.com/v2/models/{model_id}
- JS
- Py
- Java
- C#
- ObjC
- cURL
Delete A Model Version
You can also delete a specific version of a model with the model_id and version_id.
app.models.delete('{model_id}', '{version_id}').then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
app.models.delete('{id}', '{version_id}')
# or
model = app.models.get('{id}')
model.delete_version('{version_id}')
client.deleteModelVersion("{model_id}", "{version_id}").executeSync();
// Or in a more object-oriented manner:
client.getModelByID("{model_id}")
.executeSync().get() // Returns Model object
.deleteVersion("{version_id}")
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.DeleteModelVersion("{model_id}", "{version_id}")
.ExecuteAsync();
// Or in a more object-oriented manner:
var model = (await client.GetModel<Concept>("{model_id}")
.ExecuteAsync()).Get(); // Returns a Model<Concept> object.
await model.DeleteModelVersion("{version_id}").ExecuteAsync();
}
}
}
[app deleteVersionForModel:{model_id} versionID:{version_id} completion:^(NSError *error) {
NSLog(@"model version deleted");
}];
curl -X DELETE \
-H "Authorization: Key YOUR_API_KEY" \
https://api.clarifai.com/v2/models/{model_id}/versions/{version_id}
- JS
- Py
- Java
- C#
- ObjC
- cURL
Delete All Models
If you would like to delete all models associated with an application, you can also do that. Please proceed with caution as these cannot be recovered.
app.models.delete().then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
app.models.delete_all()
client.deleteAllModels().executeSync();
using System.Threading.Tasks;
using Clarifai.API;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.DeleteAllModels()
.ExecuteAsync();
}
}
}
[_app deleteAllModels:^(NSError *error) {
NSLog(@"delete all models");
}];
curl -X DELETE \
-H "Authorization: Key YOUR_API_KEY" \
https://api.clarifai.com/v2/models/
- JS
- Py
- Java
- C#
- ObjC
- cURL
Train A Model
When you train a model, you are telling the system to look at all the images with concepts you've provided and learn from them. This train operation is asynchronous. It may take a few seconds for your model to be fully trained and ready.
Note: you can repeat this operation as often as you like. By adding more images with concepts and training, you can get the model to predict exactly how you want it to.
app.models.train("{model_id}").then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
// or if you have an instance of a model
model.train().then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('{model_id}')
model.train()
client.trainModel("{model_id}").executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.TrainModel<Concept>("{model_id}")
.ExecuteAsync();
}
}
}
ClarifaiImage *image = [[ClarifaiImage alloc] initWithURL:@"https://samples.clarifai.com/puppy.jpeg"]
[app getModel:@"{id}" completion:^(ClarifaiModel *model, NSError *error) {
[model train:^(ClarifaiModel *model, NSError *error) {
NSLog(@"model: %@", model);
}];
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
https://api.clarifai.com/v2/models/{model_id}/versions
- JS
- Py
- Java
- C#
- ObjC
- cURL
Predict With A Model
Once you have trained a model you are ready to use your new model to get predictions. The predictions returned will only contain the concepts that you told it to see.
app.models.predict("{model_id}", ["https://samples.clarifai.com/puppy.jpeg"]).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
// or if you have an instance of a model
model.predict("https://samples.clarifai.com/puppy.jpeg").then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp, Image as ClImage
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('{model_id}')
image = ClImage(url='https://samples.clarifai.com/puppy.jpeg')
model.predict([image])
client.predict("{model_id}")
.withInputs(
ClarifaiInput.forImage("https://samples.clarifai.com/puppy.jpeg")
)
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Inputs;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.Predict<Concept>(
"{model_id}",
new ClarifaiURLImage("https://samples.clarifai.com/puppy.jpeg"))
.ExecuteAsync();
}
}
}
ClarifaiImage *image = [[ClarifaiImage alloc] initWithURL:@"https://samples.clarifai.com/puppy.jpeg"]
[app getModel:@"{model_id}" completion:^(ClarifaiModel *model, NSError *error) {
[model predictOnImages:@[image]
completion:^(NSArray<ClarifaiSearchResult *> *outputs, NSError *error) {
NSLog(@"outputs: %@", outputs);
}];
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"data": {
"image": {
"url": "https://samples.clarifai.com/puppy.jpeg"
}
}
}
]
}'\
https://api.clarifai.com/v2/models/{model_id}/outputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Search Models By Name And Type
You can search all your models by name and type of model.
app.models.search('general-v1.3', 'concept').then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
# search model name
app.models.search('general-v1.3')
# search model name and type
app.models.search(model_name='general-v1.3', model_type='concept')
client.findModel()
.withModelType(ModelType.CONCEPT)
.withName("general-v1.3")
.getPage(1)
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Models;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.SearchModels(
modelType: ModelType.Concept,
name: "general-v1.3")
.ExecuteAsync();
}
}
}
[app searchForModelByName:@"general-v1.3" modelType:ClarifaiModelTypeConcept completion:^(NSArray<ClarifaiModel *> *models, NSError *error) {
NSLog(@"models: %@", models);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"model_query": {
"name": "general-v1.3",
"type": "concept"
}
}'\
https://api.clarifai.com/v2/models/searches
- JS
- Py
- Java
- C#
- ObjC
- cURL
Evaluation
Interpreting Results
Once the Model Evaluation is complete, you’ll be able to view the evaluation results in the Explorer here
We suggest that you start by looking at the Concept by Concept Probabilities Results and the Summary Table to get a sense of the overall model prediction performance and identify the high-performing and low-performing concepts. Afterwards, hone in on the Selection Details of the False Positives and False Negatives to identify any biases, if any, in how the model is predicting, and to correct any inputs that are mislabeled
Generally, you’ll be looking at results that represent either a) the average across K splits, or b) the test set of a single split, which is about 1/K of your original training set. Note that a single split will be capped at 1,000 inputs.
Model Analysis
Model Accuracy Score
Model Accuracy Score is the highest level metric for your model’s prediction performance. It is defined as the macro average of the areas under the receiver operating characteristic curve for every concept. This metric does not depend on the Prediction Threshold. This metric is an average across K splits.
A score of 1 represents a perfect model; a score of .5 represents a worthless model. As a general rule of thumb, a score above .9 is considered good.
Note that we discourage users from making a final assessment of the model accuracy based on the Model Accuracy Score only.
Concept Analysis
Prediction Threshold
Probability threshold determines the model’s predictions. The default threshold is .5. The input is predicted as (i.e. “counts” as) as a concept, such as “dog”, only if the prediction probability for “dog” is higher than the set threshold, for example, 0.5. You can adjust the threshold depending on how ‘strict’ you want your classification to be.
All prediction binary metrics, such as True Positives, False Negatives, and False Positives, and Total Predicted, Recall Rate, Precision Rate, depend on this threshold.
Evaluation Summary
This table summarizes the numerical evaluation results for every concept. For every concept, it calculates:
K-Split Average
- ROC AUC (Concept Accuracy Score): concept’s prediction performance score, defined the area under the receiver operating characteristic curve. A score of 1 represents a perfect model; a score of .5 represents a worthless model. As a general rule of thumb, a score above .9 is considered good. Note: ROC AUC is not dependent on the prediction threshold.
1-Split
- Total Labeled: Total number of inputs that were originally labeled as the concept in the test set. Total Labeled is the sum of True Positives (correct) and False Negatives (incorrect). Note: Total Labeled is not dependent on the prediction threshold.
- Total Predicted: Total number of inputs that were predicted as the concept in the test set. This means these inputs were predicted as a concept with probability greater than the prediction threshold value. Total Predicted is the sum of True Positives (correct) and False Positives (incorrect).
- True Positives (TP): Number of inputs that were correctly predicted as the concept they were actually labeled. Also known as “hits”. (E.g. These are the images that were labeled as “dog” and were predicted as “dog”)
- False Negatives (FN): Number of inputs that were not predicted as the concept they were actually labeled. Also known as “misses”. (E.g. These are the images that were labeled as “dog” but were not predicted as “dog”)
- False Positives (FP): Number of inputs that were predicted as the concept, but they were not labeled as the concept. Also known as “false alarms”. (E.g. These are the images that were predicted as “dog” but were not labeled as “dog”)
- Recall Rate: proportion of the images labeled as the concept that were predicted as the concept. It is calculated as True Positives divided by Total Labeled. Also known as “sensitivity” or “true positive rate”.
- Precision Rate: proportion of the images predicted as a concept that had been actually labeled as the concept. It is calculated as True Positives divided by Total Predicted. Also known as “positive predictive value”.
Concept by Concept Results (Advanced)
This section has a concept by concept matrix. Note that this is not a confusion matrix; we recommend that you read this through before interpreting the data.
In general, the matrix is meant to be read by fixing each row. Each row represents a subset of the test set that was actually labeled as a particular concept. For this subset, each cell across the row represents either
- The number of inputs that were predicted as a concept (i.e. “counts”), or
- The average prediction probability for each concept, noted by the column name, for all inputs in this subset, across all K splits (i.e. “probabilities”).
Concepts that co-occur, or are similar, may form a visual cluster on the matrix. On the other hand, exclusive or dissimilar concepts should not form a cluster.
Counts (1-Split) 
Each row represents the subset of the test set that were actually labeled as a concept, e.g. “dog”. As you go across the row, each cell shows the number of times those images were predicted as each concept, noted by the column name.
The diagonal cells represent True Positives, i.e. correctly predicted inputs. You’d want this number to be as close to the Total Labeled as possible.
Depending on how your model was trained, the off-diagonal cells could include both correct and incorrect predictions. In a non-mutually exclusive concepts environment, you can label an image with more than 1 concept. For example, an image is labeled as both “dog” and “cat”, this image would be counted in both “dog” row and “cat” row. If the model correctly predicts this image to be both “dog” and “cat”, then this input will be counted in both on and off-diagonal cells.
Few things to note:
- Remember that the prediction counts depend on the threshold. This means the images are counted toward a predicted concept only if the prediction probability for this concept is higher than the threshold.
- This means the sum of the row may be less or greater than the # of total labeled inputs that were labeled as the concept.
- You can click on each cell to view the actual prediction results for every input that was counted in this cell.
- This represents the test set data of a single split.
Probabilities (K-Split Average) 
Each row represents the subset of the test set that were actually labeled as a concept, e.g. “dog”. As you go across the row, each cell shows the average prediction probability for each concept, noted by the column name, for all inputs in this subset. In short, the cell shows the average prediction probability for a concept given the images labeled as a concept.
Few things to note:
- This matrix does not depend on the Prediction Threshold.
- You can click on each cell to view the actual prediction results for every input that were used to calculate this cell.
- This represents the average across all K splits.
Input Analysis
Selection Details
This Selection Details table shows the input-level details of the selection you made on the Summary Table or Concept by Concept Results. It shows the image input and prediction probabilities for a specific concept.
Note: the prediction probabilities on this table may seem different from your actual model’s probabilities. The reason is that all the evaluation results are based on the new model that was built for evaluation purposes during the cross validation process.
Expanded Selection Details
You can click on the expand button to view the prediction probabilities for every concept for each image. The blue dot denotes the concept(s) the input was actually labeled, i.e. true positives. The red dot denotes the concept(s) that the input was not labeled.
Improving your Model
The evaluation metrics are meant to help you diagnose the quality of your model. Your model may belong to one or more of many categories, including, but not limited to:
- Good model with all great concepts.
- OK model with a few bad concepts.
- Bad model: with all bad concepts.
- Biased model: the model is consistently picking up certain visual cues other than what you’d like to pick up. This is sometimes obvious to pick up in the Explorer here .
- Model with Variance: there is no consistency in the way the model is predicting on inputs.
Possible Areas of Improvement
The performance of your model depends on the performance of each concept, which is trained on a set of inputs. We’d recommend that you look at both inputs and concepts when diagnosing areas of improvement.
Inputs
- Diversity: try to include all perspectives of the concept, e.g. include all angles of a “dog”, if you’re building a “dog” concept.
- Strong positives: Images that are the true representation of your concept.
- Training data should be representative of the real world data -- avoid making models where the data is too ‘easy’, i.e. unrealistic set of data.
- Number: minimum 50 inputs per concept; more inputs the better.
- File dimensions: minimum 512px x 512px.
Concepts
- Concepts: avoid concepts that do not rely on visual cues within the image. Also, current custom training does not perform well on training to identify faces.
- Labels: check to see if any inputs are labeled with wrong concepts.
Tips
When improving your model, there is no one-size-fits-all answer. Here are some tips to keep in mind:
- Although we use ROC AUC as a general top-level ‘score’ for both concept and model, we do not recommend that you rely on 1 metric only to draw your final conclusion on your model performance.
- Refer to both Concepts by Concepts Results as well as Selection Details to get a better grasp of your model.
- When interpreting the evaluation results, keep in mind the nature of your model. Specifically, pay attention to whether or not you have labeled the inputs with more than 1 concept (i.e. non-mutually exclusive concepts environment), vs. only 1 concept per image.
- Remember, the rule of diminishing returns may also apply to training models. After a certain point, the changes may not make a big difference in the model quality.
Predictions
You can set additional parameters to gain flexibility in the predict operation.
Select Concepts
By putting this additional parameter on your predict calls, you can receive predict value(s) for only the concepts that you want to. You can specify particular concepts by either their id and/or their name. The concept names and ids are case sensitive, and so, these must be exact matches.
To retrieve an entire list of concepts from a given model use the GET /v2/models/{model_id}/output_info endpoint. Check out the Advanced Models section for how to use with any of the API clients!
If you submit a request with not an exact match of the concept id or name, you will receive an invalid model argument error. However, if one or more matches while one or more do not, the API will respond with a Mixed Success.
clarifai.models.predict(Clarifai.GENERAL_MODEL, 'https://samples.clarifai.com/metro-north.jpg', {
selectConcepts: [
{name: 'train'},
{id: 'ai_6kTjGfF6'}
]
})
from clarifai.rest import ClarifaiApp, Concept
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('general-v1.3')
select_concept_list = [Concept(concept_name='train'), Concept(concept_id='ai_6kTjGfF6')]
model.predict_by_url(url='https://samples.clarifai.com/metro-north.jpg', select_concepts=select_concept_list)client.predict(client.getDefaultModels().generalModel().id())
.withInputs(ClarifaiInput.forImage("https://samples.clarifai.com/metro-north.jpg"))
.selectConcepts(Concept.forID("dog"), Concept.forID("cat"))
.executeSync();
using System.Collections.Generic;
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Inputs;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.Predict<Concept>(
client.PublicModels.GeneralModel.ModelID,
new ClarifaiURLImage("https://samples.clarifai.com/metro-north.jpg"),
selectConcepts: new List<Concept>
{
new Concept(id: "", name: "dog"),
new Concept(id: "", name: "cat")
})
.ExecuteAsync();
}
}
}
// Coming Sooncurl -X POST \
-H 'authorization: Key YOUR_API_KEY' \
-H 'content -type: application/json' \
-d '{
"inputs": [
{
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
}
}
],
"model": {
"output_info": {
"output_config": {
"select_concepts": [
{"name": "train"},
{"id": "ai_6kTjGfF6"}
]
}
}
}
}'\
https://api.clarifai.com/v2/models/aaa03c23b3724a16a56b629203edc62c/outputs- JS
- Py
- Java
- C#
- ObjC
- cURL
Response{
"status": {
"code": 10000,
"description": "Ok"
},
"outputs": [
{
"id": "c8abf5cbe52746efa9df8a2319d49d0a",
"status": {
"code": 10000,
"description": "Ok"
},
"created_at": "2017-06-27T13:31:57.493797045Z",
"model": {
"id": "aaa03c23b3724a16a56b629203edc62c",
"name": "general-v1.3",
"created_at": "2016-03-09T17:11:39.608845Z",
"app_id": "main",
"output_info": {
"message": "Show output_info with: GET /models/{model_id}/output_info",
"type": "concept",
"type_ext": "concept"
},
"model_version": {
"id": "aa9ca48295b37401f8af92ad1af0d91d",
"created_at": "2016-07-13T01:19:12.147644Z",
"status": {
"code": 21100,
"description": "Model trained successfully"
}
}
},
"input": {
"id": "c613b3254da34382b2fca65365da7c49",
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
}
},
"data": {
"concepts": [
{
"id": "ai_HLmqFqBf",
"name": "train",
"value": 0.9989112,
"app_id": "main"
},
{
"id": "ai_6kTjGfF6",
"name": "station",
"value": 0.992573,
"app_id": "main"
}
]
}
}
]
}Maximum Concepts
Setting the max concepts parameter will customize how many concepts and their corresponding probability scores the predict endpoint will return. If not specified, the predict endpoint will return the top 20 concepts. You can currently set the max concepts parameter to any number in the range: [1-200]. If your use case requires more concepts, please contact Support.
clarifai.models.predict(Clarifai.GENERAL_MODEL, 'https://samples.clarifai.com/metro-north.jpg', { maxConcepts: 3 })
.then(log)
.catch(log);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('general-v1.3')
model.predict_by_url(url='https://samples.clarifai.com/metro-north.jpg', max_concepts=3)
client.predict(client.getDefaultModels().generalModel().id())
.withInputs(ClarifaiInput.forImage("https://samples.clarifai.com/metro-north.jpg"))
.withMaxConcepts(3)
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Inputs;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.Predict<Concept>(
client.PublicModels.GeneralModel.ModelID,
new ClarifaiURLImage("https://samples.clarifai.com/metro-north.jpg"),
maxConcepts: 3)
.ExecuteAsync();
}
}
}
//Coming soon
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
}
}
],
"model":{
"output_info":{
"output_config":{
"max_concepts": 3
}
}
}
}'\
https://api.clarifai.com/v2/models/aaa03c23b3724a16a56b629203edc62c/outputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Response
{
"status": {
"code": 10000,
"description": "Ok"
},
"outputs": [
{
"id": "c8c400234b0d47df9084857df0d69efb",
"status": {
"code": 10000,
"description": "Ok"
},
"created_at": "2017-06-15T16:09:48.984389535Z",
"model": {
"id": "aaa03c23b3724a16a56b629203edc62c",
"name": "general-v1.3",
"created_at": "2016-02-26T23:38:40.086101Z",
"app_id": "main",
"output_info": {
"message": "Show output_info with: GET /models/{model_id}/output_info",
"type": "concept",
"type_ext": "concept"
},
"model_version": {
"id": "aa9ca48295b37401f8af92ad1af0d91d",
"created_at": "2016-07-13T00:58:55.915745Z",
"status": {
"code": 21100,
"description": "Model trained successfully"
}
}
},
"input": {
"id": "fd99d9e345f3495a8bd2802151d09efa",
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
}
},
"data": {
"concepts": [
{
"id": "ai_HLmqFqBf",
"name": "train",
"value": 0.9989112,
"app_id": "main"
},
{
"id": "ai_fvlBqXZR",
"name": "railway",
"value": 0.9975532,
"app_id": "main"
},
{
"id": "ai_Xxjc3MhT",
"name": "transportation system",
"value": 0.9959158,
"app_id": "main"
}
]
}
}
]
}
Minimum Prediction Value
This parameter lets you set a minimum probability threshold for the outputs you want to view for the Predict operation. For example if you want to see all concepts with a probability score of .90 or higher, this parameter will allow you to accomplish that. Also note that if you don't specify the number of max concepts, you will only see the top 20. If your result can contain more values you will have to increase the number of maximum concepts as well.
clarifai.models.predict(Clarifai.GENERAL_MODEL, 'https://samples.clarifai.com/metro-north.jpg', { minValue: 0.97 })
.then(log)
.catch(log);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_CLARIFAI_KEY')
model = app.models.get('general-v1.3')
model.predict_by_url(url='https://samples.clarifai.com/metro-north.jpg', min_value=0.97)
client.predict(client.getDefaultModels().generalModel().id())
.withInputs(ClarifaiInput.forImage("https://samples.clarifai.com/metro-north.jpg"))
.withMinValue(0.9)
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Inputs;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.Predict<Concept>(
client.PublicModels.GeneralModel.ModelID,
new ClarifaiURLImage("https://samples.clarifai.com/metro-north.jpg"),
minValue: 0.9M)
.ExecuteAsync();
}
}
}
//Coming soon
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
}
}
],
"model":{
"output_info":{
"output_config":{
"min_value": 0.97
}
}
}
}'\
https://api.clarifai.com/v2/models/aaa03c23b3724a16a56b629203edc62c/outputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Response
{
"status": {
"code": 10000,
"description": "Ok"
},
"outputs": [
{
"id": "b2027bccf4964d03b062ce653cff85b6",
"status": {
"code": 10000,
"description": "Ok"
},
"created_at": "2017-06-15T20:22:05.841603659Z",
"model": {
"id": "aaa03c23b3724a16a56b629203edc62c",
"name": "general-v1.3",
"created_at": "2016-02-26T23:38:40.086101Z",
"app_id": "main",
"output_info": {
"message": "Show output_info with: GET /models/{model_id}/output_info",
"type": "concept",
"type_ext": "concept"
},
"model_version": {
"id": "aa9ca48295b37401f8af92ad1af0d91d",
"created_at": "2016-07-13T00:58:55.915745Z",
"status": {
"code": 21100,
"description": "Model trained successfully"
}
}
},
"input": {
"id": "f7640568d37f47fbba9d6fdc892ec64d",
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
}
},
"data": {
"concepts": [
{
"id": "ai_HLmqFqBf",
"name": "train",
"value": 0.9989112,
"app_id": "main"
},
{
"id": "ai_fvlBqXZR",
"name": "railway",
"value": 0.9975532,
"app_id": "main"
},
{
"id": "ai_Xxjc3MhT",
"name": "transportation system",
"value": 0.9959158,
"app_id": "main"
},
{
"id": "ai_6kTjGfF6",
"name": "station",
"value": 0.992573,
"app_id": "main"
},
{
"id": "ai_RRXLczch",
"name": "locomotive",
"value": 0.992556,
"app_id": "main"
},
{
"id": "ai_VRmbGVWh",
"name": "travel",
"value": 0.98789215,
"app_id": "main"
},
{
"id": "ai_SHNDcmJ3",
"name": "subway system",
"value": 0.9816359,
"app_id": "main"
},
{
"id": "ai_jlb9q33b",
"name": "commuter",
"value": 0.9712483,
"app_id": "main"
}
]
}
}
]
}
By Model Version ID
Every time you train a custom model, it creates a new model version. By specifying version id in your predict call, you can continue to predict on a previous version, for consistent prediction results. Clarifai also updates our pre-built models on a regular basis.
If you are looking for consistent results from your predict calls, use version id. If the model version id is not specified, predict will default to the most current model.
let app = new Clarifai.App({apiKey: 'YOUR_API_KEY'});
app.models.predict({id:'MODEL_ID', version:'MODEL_VERSION_ID'}, "https://samples.clarifai.com/metro-north.jpg").then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.models.get('MODEL_ID')
model.get_version('MODEL_VERSION_ID')
image = ClImage(url='https://samples.clarifai.com/metro-north.jpg')
model.predict([image])
ModelVersion modelVersion = client.getModelVersionByID("MODEL_ID", "MODEL_VERSION_ID")
.executeSync()
.get();
client.predict("MODEL_ID")
.withVersion(modelVersion)
.withInputs(ClarifaiInput.forImage("https://samples.clarifai.com/metro-north.jpg"))
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Inputs;
using Clarifai.DTOs.Predictions;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
var response = await Client.Predict<Concept>(
"YOUR_MODEL_ID",
new ClarifaiURLImage("https://samples.clarifai.com/metro-north.jpg"),
modelVersionID: "MODEL_VERSION_ID")
.ExecuteAsync();
}
}
}
//Coming soon
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
}
}
]
}'\
https://api.clarifai.com/v2/models/<model_id>/versions/<model_version_id>/outputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Searches
By Public Concepts
When you add an input, it automatically gets predictions from the general model. You can search for those predictions.
app.inputs.search([
{
concept: {
name: 'cat'
}
},
{
concept: {
name: 'dog'
}
}
]).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_CLARIFAI_KEY')
# search by single concept name
app.inputs.search_by_predicted_concepts(concept='cat')
# search by single concept id
app.inputs.search_by_predicted_concepts(concept_id='ai_mFqxrph2')
# search by multiple concepts with name
app.inputs.search_by_predicted_concepts(concepts=['cat', 'cute'])
# search by multiple concepts with ids
app.inputs.search_by_predicted_concepts(concept_ids=['ai_mFqxrph2', 'ai_4CRlSvbV'])
# search by multiple concepts with not logic
app.inputs.search_by_predicted_concepts(concepts=['cat', 'dog'], values=[True, False])
// Search concept by name
client.searchInputs(SearchClause.matchConcept(Concept.forName("cat")))
.getPage(1)
.executeSync();
// Search concept by ID
client.searchInputs(SearchClause.matchConcept(Concept.forID("ai_mFqxrph2")))
.getPage(1)
.executeSync();
// Search multiple concepts
client.searchInputs(SearchClause.matchConcept(Concept.forID("cat")))
.and(SearchClause.matchConcept(Concept.forID("cute")))
.getPage(1)
.executeSync();
// Search NOT by concept
client.searchInputs(SearchClause.matchConcept(Concept.forID("cat").withValue(false)))
.getPage(1)
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Searches;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
// Search concept by name
await client.SearchInputs(SearchBy.ConceptName("cat"))
.Page(1)
.ExecuteAsync();
// Search concept by ID
await client.SearchInputs(SearchBy.ConceptID("cat"))
.Page(1)
.ExecuteAsync();
// Search multiple concepts
await client.SearchInputs(
SearchBy.ConceptID("cat"),
SearchBy.ConceptID("cute"))
.Page(1)
.ExecuteAsync();
}
}
}
// First create a search term with a concept you want to search.
ClarifaiConcept *conceptFromGeneralModel = [[ClarifaiConcept alloc] initWithConceptName:@"fast"];
ClarifaiSearchTerm *searchTerm = [ClarifaiSearchTerm searchByPredictedConcept:conceptFromGeneralModel];
[app search:@[searchTerm] page:@1 perPage:@20 completion:^(NSArray<ClarifaiSearchResult *> *results, NSError *error) {
// Print output of first search result.
NSLog(@"inputID: %@", results[0].inputID);
NSLog(@"URL: %@", results[0].mediaURL);
NSLog(@"probability of input matching search query: %@", results[0].score);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"query": {
"ands": [
{
"output": {
"data": {
"concepts": [
{
"name":"dog"
}
]
}
}
}
]
}
}'\
https://api.clarifai.com/v2/searches
- JS
- Py
- Java
- C#
- ObjC
- cURL
By Custom Concepts
After you have added inputs with concepts, you can search by those concepts.
app.inputs.search([
{
concept: {
type: 'input',
name: 'cat'
}
},
{
concept: {
type: 'input',
name: 'dog'
}
}
]).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_CLARIFAI_KEY')
# search by single concept name
app.inputs.search_by_annotated_concepts(concept='cat')
# search by single concept id
app.inputs.search_by_annotated_concepts(concept_id='ai_mFqxrph2')
# search by multiple concepts with name
app.inputs.search_by_annotated_concepts(concepts=['cat', 'cute'])
# search by multiple concepts with ids
app.inputs.search_by_annotated_concepts(concept_ids=['ai_mFqxrph2', 'ai_4CRlSvbV'])
# search by multiple concepts with not logic
app.inputs.search_by_annotated_concepts(concepts=['cat', 'dog'], values=[True, False])
// Search concept by name
client.searchInputs(SearchClause.matchUserTaggedConcept(Concept.forName("cat")))
.getPage(1)
.executeSync();
// Search concept by ID
client.searchInputs(SearchClause.matchUserTaggedConcept(Concept.forID("ai_mFqxrph2")))
.getPage(1)
.executeSync();
// Search multiple concepts
client.searchInputs(SearchClause.matchUserTaggedConcept(Concept.forID("cat")))
.and(SearchClause.matchUserTaggedConcept(Concept.forID("cute")))
.getPage(1)
.executeSync();
// Search NOT by concept
client.searchInputs(SearchClause.matchUserTaggedConcept(Concept.forID("cat").withValue(false)))
.getPage(1)
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Searches;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
// Search concept by name
await client.SearchInputs(SearchBy.UserTaggedConceptName("cat")
)
.Page(1)
.ExecuteAsync();
// Search concept by ID
await client.SearchInputs(SearchBy.UserTaggedConceptID("ai_mFqxrph2")
)
.Page(1)
.ExecuteAsync();
// Search multiple concepts
await client.SearchInputs(
SearchBy.UserTaggedConceptID("cat"),
SearchBy.UserTaggedConceptID("cute"))
.Page(1)
.ExecuteAsync();
}
}
}
// If you have previously added inputs tagged with "dog", you can search for them by the same tag.
ClarifaiConcept *concept = [[ClarifaiConcept alloc] initWithConceptName:@"dog"];
ClarifaiSearchTerm *term = [ClarifaiSearchTerm searchInputsByConcept:concept];
[app search:@[term] page:@1 perPage:@20 completion:^(NSArray<ClarifaiSearchResult *> *results, NSError *error) {
// Print output of first search result.
NSLog(@"inputID: %@", results[0].inputID);
NSLog(@"URL: %@", results[0].mediaURL);
NSLog(@"probability of input matching search query: %@", results[0].score);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"query": {
"ands": [
{
"input": {
"data": {
"concepts": [
{
"name":"dog"
}
]
}
}
}
]
}
}'\
https://api.clarifai.com/v2/searches
- JS
- Py
- Java
- C#
- ObjC
- cURL
By Image
You can use images to search through your collection. The API will return ranked results based on how similar the results are to the image you provided in your query.
app.inputs.search(
{
input: {
url: 'https://samples.clarifai.com/puppy.jpeg'
}
}
).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_CLARIFAI_KEY')
# search by image url
app.inputs.search_by_image(url="https://samples.clarifai.com/metro-north.jpg")
# search by existing input id
input_id = "some_existing_input_id"
app.inputs.search_by_image(image_id=input_id)
# search by raw bytes
data = "image_raw_bytes"
app.inputs.search_by_image(imgbytes=data)
# search by base64 bytes
base64_data = "image_bytes_encoded_in_base64"
app.inputs.search_by_image(base64bytes=base64_data)
# search by local filename
filename="filename_on_local_disk.jpg"
app.inputs.search_by_image(filename=filename)
# search from fileio
fio = open("filename_on_local_disk.jpg", 'rb')
app.inputs.search_by_image(fileobj=fio)
// Search by image URL (String or java.net.URL)
client.searchInputs(SearchClause.matchImageVisually(ClarifaiImage.of("https://samples.clarifai.com/metro-north.jpg")))
.getPage(1)
.executeSync();
// Search by local image (java.io.File or byte[])
client.searchInputs(SearchClause.matchImageVisually(ClarifaiImage.of(new File("image.png"))))
.getPage(1)
.executeSync();
using System.IO;
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Searches;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
// Search by image URL
await client.SearchInputs(
SearchBy.ImageVisually("https://samples.clarifai.com/metro-north.jpg"))
.Page(1)
.ExecuteAsync();
// Search by local image
await client.SearchInputs(
SearchBy.ImageVisually(File.ReadAllBytes("image.png")))
.Page(1)
.ExecuteAsync();
}
}
}
ClarifaiSearchTerm *searchTerm = [ClarifaiSearchTerm searchVisuallyWithImageURL:@"https://samples.clarifai.com/metro-north.jpg"];
[app search:@[searchTerm] page:@1 perPage:@20 completion:^(NSArray<ClarifaiSearchResult *> *results, NSError *error) {
// Print output of first search result.
NSLog(@"inputID: %@", results[0].inputID);
NSLog(@"URL: %@", results[0].mediaURL);
NSLog(@"probability of input matching search query: %@", results[0].score);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"query": {
"ands": [
{
"output":{
"input":{
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
}
}
}
}
]
}
}'\
https://api.clarifai.com/v2/searches
- JS
- Py
- Java
- C#
- ObjC
- cURL
By Image With Crop
You can search using a crop of an image through your collection. The API will still return ranked results based upon on how similar the results are to the crop of the image you provide in your query.
app.inputs.search(
{
input: {
url: 'https://samples.clarifai.com/puppy.jpeg',
crop: [0.1, 0.1, 0.9, 0.9]
}
}
).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
search = app.inputs.search_by_image(url='https://samples.clarifai.com/puppy.jpeg', crop=[0.1, 0.1, 0.9, 0.9])
print searchclient.searchInputs(SearchClause.matchImageVisually(ClarifaiImage.of("https://samples.clarifai.com/puppy.jpeg")
.withCrop(Crop.create()
.top(0.1F)
.left(0.1F)
.bottom(0.9F)
.right(0.9F)
)
))
.getPage(1)
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs;
using Clarifai.DTOs.Searches;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.SearchInputs(SearchBy.ImageVisually(
"https://samples.clarifai.com/metro-north.jpg",
crop: new Crop(0.1M, 0.1M, 0.9M, 0.9M)))
.Page(1)
.ExecuteAsync();
}
}
}
// Coming Sooncurl -X POST \
-H 'authorization: Key YOUR_API_KEY' \
-H 'content-type: application/json' \
-d '{
"query": {
"ands": [
{
"output": {
"input": {
"data": {
"image": {
"url": "https://samples.clarifai.com/puppy.jpeg",
"crop": [0.1,0.1,0.9,0.9]
}
}
}
}
}
]
}
}'\
https://api.clarifai.com/v2/searches- JS
- Py
- Java
- C#
- ObjC
- cURL
By Public and Custom Concepts
You can combine a search to find inputs that have concepts you have supplied as well as predictions from your model.
app.inputs.search([
// this is the public concept
{
concept: {
name: 'cat'
}
},
// this is the user-supplied concept
{
concept: {
type: 'input',
name: 'dog'
}
}
]).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp, InputSearchTerm, OutputSearchTerm, SearchQueryBuilder
app = ClarifaiApp(api_key='YOUR_API_KEY')
term1 = InputSearchTerm(concept='cat')
term2 = OutputSearchTerm(concept='dog', value=False)
query = SearchQueryBuilder()
query.add_term(term1)
query.add_term(term2)
app.inputs.search(query)
client.searchInputs()
// Matches images we tagged as "cat", and that the API tagged as not having "dog"
.ands(
SearchClause.matchUserTaggedConcept(Concept.forName("cat")),
SearchClause.matchConcept(Concept.forName("dog").withValue(false))
)
.getPage(1)
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Searches;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.SearchInputs(
SearchBy.UserTaggedConceptName("cat"),
SearchBy.ConceptID("dog"))
.Page(1)
.ExecuteAsync();
}
}
}
ClarifaiConcept *conceptFromGeneralModel = [[ClarifaiConcept alloc] initWithConceptName:@"fast"];
ClarifaiConcept *conceptFromTrainedCustomModel = [[ClarifaiConcept alloc] initWithConceptName:@"dog"];
ClarifaiSearchTerm *term1 = [ClarifaiSearchTerm searchByPredictedConcept:conceptFromGeneralModel];
ClarifaiSearchTerm *term2 = [ClarifaiSearchTerm searchByPredictedConcept:conceptFromTrainedCustomModel];
[_app search:@[term1, term2] page:@1 perPage:@20 completion:^(NSArray<ClarifaiSearchResult *> *results, NSError *error) {
// Print output of first search result.
NSLog(@"inputID: %@", results[0].inputID);
NSLog(@"URL: %@", results[0].mediaURL);
NSLog(@"probability of input matching search query: %@", results[0].score);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"query": {
"ands": [
{
"output": {
"data": {
"concepts": [
{
"name": "fast"
}
]
}
}
},
{
"input": {
"data": {
"concepts": [
{
"name": "ferrari23",
"value": true
}
]
}
}
}
]
}
}'\
https://api.clarifai.com/v2/searches
- JS
- Py
- Java
- C#
- ObjC
- cURL
By Custom Metadata
After you have added inputs with custom metadata, you can search by that metadata.
Below is an example of searching over custom metadata. You can exact match any key: value pair no matter how nested it is. For example, if the metadata on an input is:
{
"keyname": "value1",
"somelist": [1,2,3],
"somenesting": {
"keyname2":"value2",
"list2":[4,5]
}
}Then the following searches will find this:
{
"keyname": "value1"
}{
"somelist": [1,2,3]
}{
"somelist": [1,2]
}{
"somenesting": {"keyname2":"value2"}
}{
"somenesting": {"list2":[5]}
}How to perform searches:
// Search with only metadata
app.inputs.search({
input: {
metadata: {
key: 'value'
}
}
}).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
// Search with nested metadata
app.inputs.search({
input: {
metadata: {
parent: {
key: 'value'
}
}
}
}).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
// Search with metadata and concepts or input source
app.inputs.search([
{
input: { metadata: { key: 'value' } }
},
{
concept: { name: 'cat' }
},
{
concept: { type: 'output', name: 'group', value: false }
}
]).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp, InputSearchTerm, OutputSearchTerm, SearchQueryBuilder
app = ClarifaiApp(api_key='YOUR_API_KEY')
# search with simple metadata only
app.inputs.search_by_metadata(metadata={'name':'bla'})
# search with nested metadata only
app.inputs.search_by_metadata(metadata={'my_class1': { 'name' : 'bla' }})
# search with metadata combined with others
query = SearchQueryBuilder()
query.add_term(InputSearchTerm(concept='cat'))
query.add_term(InputSearchTerm(metadata={'name':'value'}))
query.add_term(OutputSearchTerm(concept='group', value=False))
app.inputs.search(query)
JsonObject metadata = new JsonObject();
metadata.addProperty("isPuppy", true);
List<SearchHit> hits = client
.searchInputs(SearchClause.matchMetadata(metadata))
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Searches;
using Newtonsoft.Json.Linq;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
var metadata = new JObject();
metadata.Add("isPuppy", true);
await client.SearchInputs(
SearchBy.Metadata(metadata))
.Page(1)
.ExecuteAsync();
}
}
}
// Search by metadata only.
[_app searchByMetadata:@{@"my_key": @[@"my", @"values"]} page:@1 perPage:@20 completion:^(NSArray<ClarifaiSearchResult *> *results, NSError *error) {
// Print output of first search result.
NSLog(@"inputID: %@", results[0].inputID);
NSLog(@"URL: %@", results[0].mediaURL);
NSLog(@"probability of input matching search query: %@", results[0].score);
}];
// Search metadata in conjunction with other ClarifaiSearchTerms. For example, the
// following will search for inputs with predicted tag "fast" and matching metadata.
ClarifaiConcept *conceptFromGeneralModel = [[ClarifaiConcept alloc] initWithConceptName:@"fast"];
ClarifaiSearchTerm *searchTerm1 = [ClarifaiSearchTerm searchByPredictedConcept:conceptFromGeneralModel];
ClarifaiSearchTerm *searchTerm2 = [ClarifaiSearchTerm searchInputsWithMetadata:@{@"my_key": @[@"my", @"values"]}];
[app search:@[searchTerm1, searchTerm2] page:@1 perPage:@20 completion:^(NSArray<ClarifaiSearchResult *> *results, NSError *error) {
// Print output of first search result.
NSLog(@"inputID: %@", results[0].inputID);
NSLog(@"URL: %@", results[0].mediaURL);
NSLog(@"probability of input matching search query: %@", results[0].score);
}];
curl -X POST \
-H "Authorization: Key {api-key}" \
-H "Content-Type: application/json" \
-d '
{
"query": {
"ands": [
{
"input":{
"data": {
"metadata": {
"key": "value"
}
}
}
}
]
}
}'\
https://api.clarifai.com/v2/searches
- JS
- Py
- Java
- C#
- ObjC
- cURL
By Url
You can also search for an input by URL.
app.inputs.search(
{
input: {
type: 'input',
url: 'https://samples.clarifai.com/puppy.jpeg'
}
}
).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
meta = {"url":"https://samples.clarifai.com/metro-north.jpg"}
app.inputs.search_by_metadata(meta)
// Lookup images with this URL
client.searchInputs(SearchClause.matchImageURL(ClarifaiImage.of("https://samples.clarifai.com/puppy.jpeg")))
.getPage(1)
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Searches;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.SearchInputs(
SearchBy.ImageURL("https://samples.clarifai.com/metro-north.jpg"))
.Page(1)
.ExecuteAsync();
}
}
}
// Lookup images with this URL
ClarifaiSearchTerm *term = [ClarifaiSearchTerm searchInputsWithImageURL:@"https://samples.clarifai.com/metro-north.jpg"];
[app search:@[term] page:@1 perPage:@20 completion:^(NSArray<ClarifaiSearchResult *> *results, NSError *error) {
// Print output of first search result.
NSLog(@"inputID: %@", results[0].inputID);
NSLog(@"URL: %@", results[0].mediaURL);
NSLog(@"probability of input matching search query: %@", results[0].score);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"query": {
"ands": [
{
"input":{
"data": {
"image": {
"url": "https://samples.clarifai.com/metro-north.jpg"
}
}
}
}
]
}
}'\
https://api.clarifai.com/v2/searches
- JS
- Py
- Java
- C#
- ObjC
- cURL
By Geo Location
Search by geo location allows you to restrict your search results to a bounding box based on longitude and latitude points. There are two ways you can provide longitude/latitude points. You can provide one point and a radius or you can provide two points.
It is important to note that a search by geo location acts as a filter and returns results ranked by any other provided search criteria, whether that is a visual search, concept search or something else. If no other criteria is provided, results will return in the order the inputs were created, NOT by their distance to center of the search area.
If you are providing one point and a radius, the radius can be in "mile", "kilometer", "degree", or "radian", marked by keywords withinMiles, withinKilometers, withinDegrees, withinRadians.
If you are providing two points, a box will be drawn from the uppermost point to the lowermost point and the leftmost point to the rightmost point.
Before you perform a search by geo location, make sure you have added inputs with longitude and latitude points.
Add inputs with longitiude and latitude points
Provide a geo point to an input. The geo point is a JSON object consisting of a longitude and a latitude in GPS coordinate system (SRID 4326). There can be at most one single geo point associated with each input.
app.inputs.create({
url: "https://samples.clarifai.com/puppy.jpeg",
geo: { longitude: 116.2317, latitude: 39.5427},
}).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);from clarifai.rest import ClarifaiApp, Geo, GeoPoint
app = ClarifaiApp(api_key='YOUR_API_KEY')
geo_p1 = Geo(geo_point=GeoPoint(116.2317,39.5427))
app.inputs.create_image_from_url(url="https://samples.clarifai.com/puppy.jpeg", geo=geo_p1)client.addInputs().plus(ClarifaiInput.forImage("https://samples.clarifai.com/puppy.jpeg")
.withGeo(PointF.at(116.2317F, 39.5427F))).executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs;
using Clarifai.DTOs.Inputs;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.AddInputs(
new ClarifaiURLImage(
"https://samples.clarifai.com/puppy.jpeg",
geo: new GeoPoint(116.2317M, 39.5427M)))
.ExecuteAsync();
}
}
}
ClarifaiImage *image = [[ClarifaiImage alloc] initWithURL:@"https://samples.clarifai.com/metro-north.jpg"];
image.location = [[ClarifaiLocation alloc] initWithLatitude:116.2317 longitude:39.5427];
[_app addInputs:@[image] completion:^(NSArray<ClarifaiInput *> *inputs, NSError *error) {
NSLog(@"%@",inputs);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"inputs": [
{
"data": {
"image": {
"url": "https://samples.clarifai.com/dog.tiff",
"allow_duplicate_url": true
},
"geo": {
"geo_point": {
"longitude": -30,
"latitude": 40
}
}
}
}
]
}'\
https://api.clarifai.com/v2/inputs
- JS
- Py
- Java
- C#
- ObjC
- cURL
Perform a search with one geo point and radius in kilometers
app.inputs.search({
input: {
geo: {
longitude: 116.2317,
latitude: 39.5427,
type: 'withinKilometers',
value: 1
}
}
}).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);from clarifai.rest import ClarifaiApp, GeoPoint, GeoLimit
app = ClarifaiApp(api_key='YOUR_API_KEY')
geo_p = GeoPoint(116.2317, 39.5427)
geo_l = GeoLimit(limit_type='kilometer', limit_range=1)
imgs = app.inputs.search_by_geo(geo_point=geo_p, geo_limit=geo_l)client.searchInputs(SearchClause.matchGeo(PointF.at(59F, 29.75F), Radius.of(500, Radius.Unit.KILOMETER)))
.getPage(1)
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs;
using Clarifai.DTOs.Searches;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.SearchInputs(
SearchBy.Geo(
new GeoPoint(59M, 29.75M),
new GeoRadius(500, GeoRadius.RadiusUnit.WithinKilometers)))
.Page(1)
.ExecuteAsync();
}
}
}
ClarifaiLocation *loc = [[ClarifaiLocation alloc] initWithLatitude:116.2317 longitude:39.5427];
ClarifaiGeo *geoFilterKilos = [[ClarifaiGeo alloc] initWithLocation:loc radius:50.0 andRadiusUnit:ClarifaiRadiusUnitKilometers];
ClarifaiSearchTerm *term = [ClarifaiSearchTerm searchInputsWithGeoFilter:geoFilterKilos];
[_app search:@[term] page:@1 perPage:@20 completion:^(NSArray<ClarifaiSearchResult *> *results, NSError *error) {
NSLog(@"inputID: %@", results[0].inputID);
NSLog(@"URL: %@", results[0].mediaURL);
NSLog(@"probability of predicted concept: %@", results[0].score);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"query": {
"ands": [
{
"input": {
"data": {
"geo": {
"geo_point": {
"longitude": 59,
"latitude": 29.75
},
"geo_limit": {
"type": "withinKilometers",
"value": 1
}
}
}
}
}
]
}
}'\
https://api.clarifai.com/v2/searches
- JS
- Py
- Java
- C#
- ObjC
- cURL
Perform a search with two geo points
app.inputs.search({
input: {
geo: [{
latitude: 116.2316,
longitude: 39.5426
}, {
latitude: 116.2318,
longitude: 39.5428
}]
}
}).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);from clarifai.rest import ClarifaiApp, GeoBox, GeoPoint
app = ClarifaiApp(api_key='YOUR_API_KEY')
p1 = GeoPoint(116.2316, 39.5426)
p2 = GeoPoint(116.2318, 39.5428)
box1 = GeoBox(point1=p1, point2=p2)
imgs = app.inputs.search_by_geo(geo_box=box1)client.searchInputs(SearchClause.matchGeo(PointF.at(3F, 0F), PointF.at(70, 30F)))
.getPage(1)
.executeSync()
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs;
using Clarifai.DTOs.Searches;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.SearchInputs(
SearchBy.Geo(
new GeoPoint(3M, 0M),
new GeoPoint(70M, 30M)))
.Page(1)
.ExecuteAsync();
}
}
}
ClarifaiLocation *startLoc = [[ClarifaiLocation alloc] initWithLatitude:50 longitude:58];
ClarifaiLocation *endLoc = [[ClarifaiLocation alloc] initWithLatitude:32 longitude:-30];
ClarifaiGeo *geoBox = [[ClarifaiGeo alloc] initWithGeoBoxFromStartLocation:startLoc toEndLocation:endLoc];
[_app search:@[term] page:@1 perPage:@20 completion:^(NSArray<ClarifaiSearchResult *> *results, NSError *error) {
NSLog(@"inputID: %@", results[0].inputID);
NSLog(@"URL: %@", results[0].mediaURL);
NSLog(@"probability of predicted concept: %@", results[0].score);
}];
curl -X POST \
-H "Authorization: Key YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '
{
"query": {
"ands": [
{
"input": {
"data": {
"geo": {
"geo_box": [
{
"geo_point": {
"latitude": 35,
"longitude": -30
}
},
{
"geo_point": {
"latitude": 50,
"longitude": -35
}
}
]
}
}
}
}
]
}
}'\
https://api.clarifai.com/v2/searches
- JS
- Py
- Java
- C#
- ObjC
- cURL
Search ANDing
You can also combine searches using AND.
app.inputs.search([
{ input: { url: 'https://samples.clarifai.com/puppy.jpeg' } },
{ concept: { name: 'cat', type: 'input' } },
{ concept: { name: 'dog' } }
]).then(
function(response) {
// do something with response
},
function(err) {
// there was an error
}
);
from clarifai.rest import ClarifaiApp, InputSearchTerm, OutputSearchTerm, SearchQueryBuilder
app = ClarifaiApp(api_key='YOUR_API_KEY')
term1 = InputSearchTerm(concept='cat')
term2 = OutputSearchTerm(concept='dog', value=False)
term3 = OutputSearchTerm(url="https://samples.clarifai.com/metro-north.jpg")
query = SearchQueryBuilder()
query.add_term(term1)
query.add_term(term2)
query.add_term(term3)
app.inputs.search(query)
client.searchInputs()
.ands(
SearchClause.matchUserTaggedConcept(Concept.forName("cat")),
SearchClause.matchConcept(Concept.forName("dog").withValue(false)),
SearchClause.matchImageVisually(ClarifaiImage.of("https://samples.clarifai.com/metro-north.jpg"))
)
.getPage(1)
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
using Clarifai.DTOs.Searches;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.SearchInputs(
SearchBy.UserTaggedConceptName("cat"),
SearchBy.ConceptName("dog"),
SearchBy.ImageURL("https://samples.clarifai.com/metro-north.jpg"))
.Page(1)
.ExecuteAsync();
}
}
}
//Search for inputs that are predicted as "fast" and visually similar to the given image.
ClarifaiConcept *conceptFromGeneralModel = [[ClarifaiConcept alloc] initWithConceptName:@"fast"];
ClarifaiSearchTerm *term1 = [ClarifaiSearchTerm searchByPredictedConcept:conceptFromGeneralModel];
ClarifaiSearchTerm *term2 = [ClarifaiSearchTerm searchVisuallyWithImageURL:@"https://samples.clarifai.com/metro-north.jpg"];
[_app search:@[term1, term2] page:@1 perPage:@20 completion:^(NSArray<ClarifaiSearchResult *> *results, NSError *error) {
// Print output of first search result.
NSLog(@"inputID: %@", results[0].inputID);
NSLog(@"URL: %@", results[0].mediaURL);
NSLog(@"probability of input matching search query: %@", results[0].score);
}];
curl -X POST \
-H "Authorization: Key {api-key}" \
-H "Content-Type: application/json" \
-d '
{
"query": {
"ands": [
{
"output": {
"input":{
"data": {
"image": {
"url": "http://i.imgur.com/HEoT5xR.png"
}
}
}
}
},
{
"output": {
"data": {
"concepts": [
{"name":"fast", "value":true}
]
}
}
}
]
}
}'\
https://api.clarifai.com/v2/searches
- JS
- Py
- Java
- C#
- ObjC
- cURL
Pagination
Many API calls are paginated. You can provide page and per_page params to the API. In the example below we are getting all inputs and specifying to start at page 2 and get back 20 results per page.
app.inputs.list({page: 2, perPage: 20});
from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
app.inputs.get_by_page(page=2, per_page=20)
client.getInputs()
.perPage(20) // OPTIONAL, to specify how many results should be on one page
.getPage(2)
.executeSync();
using System.Threading.Tasks;
using Clarifai.API;
namespace YourNamespace
{
public class YourClassName
{
public static async Task Main()
{
var client = new ClarifaiClient("YOUR_API_KEY");
await client.GetInputs()
.PerPage(20) // OPTIONAL, to specify how many results should be on one page
.Page(2)
.ExecuteAsync();
}
}
}
[app getInputsOnPage:2 pageSize:20 completion:^(NSArray<ClarifaiInput *> *inputs, NSError *error) {
NSLog(@"inputs: %@", inputs);
}];
curl -X GET \
-H "Authorization: Key YOUR_API_KEY" \
https://api.clarifai.com/v2/inputs?page=2&per_page=20
- JS
- Py
- Java
- C#
- ObjC
- cURL
Patching
We designed PATCH to work over multiple resources at the same time (bulk) and be flexible enough for all your needs to minimize round trips to the server. Therefore it might seem a little different to any PATCH you've seen before, but it's not complicated. All three actions that are supported do overwrite by default, but have special behaviour for lists of objects (for example lists of concepts).
Merge
merge action will overwrite a key:value with key:new_value or append to an existing list of values, merging dictionaries that match by a corresponding id field.
In the following examples A is being patched into B to create the Result:
*Merges different key:values*
A = `{"a":[1,2,3]}`
B = `{"blah":true}`
Result = `{"blah":true, "a":[1,2,3]}`
*For id lists, merge will append*
A = `{"a":[{"id": 1}]}`
B = `{"a":[{"id": 2}]}`
Result = `{"a":[{"id": 2}, {"id":1}]}`
*Simple merge of key:values and within a list*
A = `{"a":[{"id": "1", "other":true}], "blah":1}`
B = `{"a":[{"id": "2"},{"id":"1", "other":false}]}`
Result = `{"a":[{"id": "2"},{"id": "1"}], "blah":1}`
*Different types should overwrite fine*
A = `{"a":[{"id": "1"}], "blah":1}`
B = `{"a":[{"id": "2"}], "blah":"string"}`
Result = `{"a":[{"id": "2"},{"id": "1"}], "blah":1}`
*Deep merge, notice the "id":"1" matches, so those dicts are merged in the list*
A = `{"a":[{"id": "1","hey":true}], "blah":1}`
B = `{"a":[{"id": "1","foo":"bar","hey":false},{"id":"2"}], "blah":"string"}`
Result = `{"a":[{"hey":true,"id": "1","foo":"bar"},{"id":"2"}], "blah":1}`
*For non-id lists, merge will append*
A = `{"a":[{"blah": "1"}], "blah":1}`
B = `{"a":[{"blah": "2"}], "blah":"string"}`
Result = `{"a":[{"blah": "2"}, {"blah":"1"}], "blah":1}`
*For non-id lists, merge will append*
A = `{"a":[{"blah": "1"}], "blah":1, "dict":{"a":1,"b":2}}`
B = `{"a":[{"blah": "2"}], "blah":"string"}`
Result = `{"a":[{"blah": "2"}, {"blah":"1"}], "blah":1, "dict":{"a":1,"b":2}}`
*Simple overwrite root element*
A = `{"key1":true}`
B = `{"key1":{"key2":"value2", "key3":"value3"}}`
Result = `{"key1":true}`
*Overwrite a sub element*
A = `{"key1":{"key2":true}}`
B = `{"key1":{"key2":"value2", "key3":"value3"}}`
Result = `{"key1":{"key2":true, "key3":"value3"}}`
*Merge a sub element*
A = `{"key1":{"key2":{"key4":"value4"}}}`
B = `{"key1":{"key2":"value2", "key3":"value3"}}`
Result = `{"key1":{"key2":{"key4":"value4"}, "key3":"value3"}}`
*Merge multiple trees*
A = `{"key1":{"key2":{"key9":"value9"}, "key3":{"key4":"value4", "key10":[1,2,3]}}, "key6":{"key11":"value11"}}`
B = `{"key1":{"key2":"value2", "key3":{"key4":{"key5":"value5"}}}, "key6":{"key7":{"key8":"value8"}}}`
Result = `{"key1":{"key2":{"key9":"value9"}, "key3":{"key4":"value4", "key10":[1,2,3]}}, "key6":{"key7":{"key8":"value8"}, "key11":"value11"}}`
*Merge {} element will replace*
A = `{"key1":{"key2":{}}}`
B = `{"key1":{"key2":"value2", "key3":"value3"}}`
Result = `{"key1":{"key2":{}, "key3":"value3"}}`
*Merge a null element does nothing*
A = `{"key1":{"key2":null}}`
B = `{"key1":{"key2":"value2", "key3":"value3"}}`
Result = `{"key1":{"key2":"value2", "key3":"value3"}}`
*Merge a blank list [] will replace root element*
A = `{"key1":[]}`
B = `{"key1":{"key2":"value2", "key3":"value3"}}`
Result = `{"key1":[]}`
*Merge a blank list [] will replace single element*
A = `{"key1":{"key2":[]}}`
B = `{"key1":{"key2":"value2", "key3":"value3"}}`
Result = `{"key1":{"key2":[], "key3":"value3"}}`
*Merge a blank list [] will remove nested objects*
A = `{"key1":{"key2":[{"key3":"value3"}]}}`
B = `{"key1":{"key2":{"key3":"value3"}}}`
Result = `{"key1":{"key2":[{"key3":"value3"}]}}`
*Merge an existing list with some other struct*
A = `{"key1":{"key2":{"key3":[{"key4":"value4"}]}}}`
B = `{"key1":{"key2":[]}}`
Result = `{"key1":{"key2":{"key3":[{"key4":"value4"}]}}}`Remove
remove action will overwrite a key:value with key:new_value or delete anything in a list that matches the provided values' ids.
In the following examples A is being patched into B to create the Result:
*Remove from list*
A = `{"a":[{"id": "1"}], "blah":1}`
B = `{"a":[{"id": "2"},{"id": "3"}, {"id":"1"}], "blah":"string"}`
Result = `{"a":[{"id": "2"},{"id":"3"}], "blah":1}`
*For non-id lists, remove will append*
A = `{"a":[{"blah": "1"}], "blah":1}`
B = `{"a":[{"blah": "2"}], "blah":"string"}`
Result = `{"a":[{"blah": "2"}, {"blah":"1"}], "blah":1}`
*Empty out a nested dictionary*
A = `{"key1":{"key2":true}}`
B = `{"key1":{"key2":"value2"}}`
Result = `{"key1":{}}`
*Remove the root element, should be empty*
A = `{"key1":true}`
B = `{"key1":{"key2":"value2", "key3":"value3"}}`
Result = `{}`
*Remove a sub element*
A = `{"key1":{"key2":true}}`
B = `{"key1":{"key2":"value2", "key3":"value3"}}`
Result = `{"key1":{"key3":"value3"}}`
*Remove a multiple sub elements*
A = `{"key1":{"key2":{"key3":true}, "key4":true}}`
B = `{"key1":{"key2":{"key3":{"key5":"value5"}}, "key4":{"key6":{"key7":"value7"}}}}`
Result = `{"key1":{"key2":{}}}`
*Remove one of the root elements if there are more than one*
A = `{"key1":true}`
B = `{"key1":{"key2":"value2", "key3":"value3"}, "key4":["a", "b", "c"]}`
Result = `{"key4":["a", "b", "c"]}`
*Remove with false should over write*
A = `{"key1":{"key2":false, "key3":true}, "key4":false}`
B = `{"key1":{"key2":"value2", "key3":"value3"}, "key4":[{"key5":"value5", "key6":"value6"}, {"key7": "value7"}]}`
Result = `{"key1":{"key2":false}, "key4":false}`
*Only objects with id's can be put into lists*
A = `{"key1":[{"key2":true}]}`
B = `{"key1":[{"key2":"value2"}, {"key3":"value3"}]}`
Result = `{}`
*Elements with {} should do nothing*
A = `{"key1":{}}`
B = `{"key1":{"key2":"value2", "key3":"value3"}}`
Result = `{"key1":{"key2":"value2", "key3":"value3"}}`
*Elements with nil should do nothing*
A = `{"key1":{"key2":null}}`
B = `{"key1":{"key2":"value2", "key3":"value3"}}`
Result = `{"key1":{"key2":"value2", "key3":"value3"}}`Overwrite
overwrite action will overwrite a key:value with key:new_value or overwrite a list of values with the new list of values. In most cases this is similar to merge action.
In the following examples A is being patched into B to create the Result:
*Overwrite whole list*
A = `{"a":[{"id": "1"}], "blah":1}`
B = `{"a":[{"id": "2"}], "blah":"string"}`
Result = `{"a":[{"id": "1"}], "blah":1}`
*For non-id lists, overwrite will overwrite whole list*
A = `{"a":[{"blah": "1"}], "blah":1}`
B = `{"a":[{"blah": "2"}], "blah":"string"}`
Result = `{"a":[{"blah": "1"}], "blah":1}`Supported Types
The API supports the following image formats:
- JPEG
- PNG
- TIFF
- BMP
- WEBP
The API supports the following video formats:
- AVI
- MP4
- WMV
- MOV
- GIF
- 3GPP