The ability to process and understand different types of data is very useful. Think about it: what if you could take a picture of a sign and immediately have its text translated into another language? Or hear a voice recording and determine if the message is positive or negative? Sure, you can train complex models to do tasks like this, but a simpler way is just to chain models together where the output of one model is the input to the next. This is where Clarifai Community and Mesh, our workflow product, comes into play. It allows users to combine different tools, like image recognition and text translation, into one seamless multimodal system.
By creating these combined workflows, we can make computers more efficient and insightful. Clarifai Mesh offer a versatile framework for constructing your inference pipeline, and equips you with the fundamental components for sophisticated machine learning ensemble modeling and incorporating business logic. Clarifai simplifies the process of integrating diverse models, enabling you to execute intricate data operations and design solutions tailored to your precise business requirements.
One way to create workflows is using Clarifai Community's visual graph editor, however you might want to create them programmatically instead.
The Clarifai Python SDK empowers you to define and construct intricate workflows through a YAML configuration.
Install Clarifai Python SDK using the code snippet below.
Get started by retrieving the PAT token from the instructions here and setting up the PAT token as an environment variable. Signup here
To walk through the process of creating Workflows with YAML specifications let's consider two Tasks.
Task 1: Using a generative LLM model to perform text classification for Content moderation.
For this task, we would want to assemble the GPT 3.5 Turbo model (Explore Community models here. ) and create a prompt that performs text classification over an input.
The LLM model is a “text-to-text” model type within Clarifai and our current chosen model performs multiple text-based tasks in general. Here, we utilize the LLM to generate text.
To give more context on a prompter A prompt template serves as a pre-configured piece of text used to instruct a text-to-text model. It acts as a structured query or input that guides the model in generating the desired response.
Now, we are going to create a text sentiment classification prompter node,
Here is an example of a YAML specification for the task, stored as “prompter.yml”
Having specified the YAML, we can use the below SDK functionality to use the workflow created in the Clarifai platform.
Try experimenting by creating (summarisation, translation, named entity recognition..etc)
Task 2: Face Sentiment Classification
Multi-model workflow that combines face detection and sentiment classification of 7 concepts: anger, disgust, fear, neutral, happiness, sadness, contempt, and
Workflow contains three nodes:
Here is an example of a YAML specification for the task, stored as “face_sentiment.yml”
After defining the YAML configuration, we can employ the following SDK features to utilize the workflow established on the Clarifai platform.
Jump into the Workflow Create notebook to explore a variety of workflows designed to help you kickstart your projects. These workflows include Audio Sentiment, Vector Search, Language Aware OCR, and Demographics.
To begin or make rapid adjustments to existing Clarifai community workflows using an initial YAML configuration, the SDK offers an export feature.
An example of this pipeline is provided in the Clarifai/examples library.
We are bringing more data utilities for converting annotation formats before uploading or exporting, text splitting, model training and evaluation interfaces, and vector search interfaces.
Also, let us know what functionality you would like to see in the SDK in our discord channel.
For more information on Python SDK, refer to our Docs here and for detailed examples, we constantly strive to add more notebooks here.
