Our last post about the DeveloperWeek SF hackathon left out one very important project, i.e. the Clarifai challenge winner, Random Acts of Kindness by Genie! Their clever project won them the $500 grand prize, as they combined Clarifai and other technologies to create a system that allows retail brands to grant in-store freebies anywhere, by paying for certain items on behalf of the customer when they match that brand’s targeted marketing campaign demographics. I caught up with the team behind Genie after the hackathon to learn more about how they came together, nurtured a business idea, and hacked together an impressive project in just one (intense) weekend. Read on to see what they did and how they did it!
Introduction from Genie
We are Genie, and we've created a system to change the relationships a company can have with its customers, by providing offline targeted marketing through what we call Random Acts of Kindness. Once a marketing campaign in our platform goes live, that's when the magic happens: our low-cost hardware, installed in hundreds of stores' POS systems uses Clarifai to identify every purchasing customer, run them through all the active campaign demographics ("27-year-old women from the Bay Area") and then randomly distribute a financial prize to the ones that match. The result is that, if you're in the targeted group, and have a little bit of luck, Netflix might just pay for your coffee at Peet's one day.
https://app.near.ai/rq8jfjp89
Q&A with Genie
Who are the team and how did you come together at DeveloperWeek?
The four of us are Brazilian and found ourselves in the Bay Area for different reasons.
- [Full-Stack Developer] Erick Wendel is a Google Developer Expert and Microsoft MVP. He is currently working as an Independent Solutions Architect and Trainer focused on Node.js and Cloud Computing. His coding skills earn him keynote speaking opportunities at many conferences around the world. (erickwendel.com).
- [Front-End Developer] Igor Marinelli is a student from the University of SĂŁo Paulo (USP) currently at UC Berkeley for interchange studies. Having founded a growing non-profit in Brazil, he is now a full-time developer and innovator at Shawee, the largest hackathon organizer in Brazil. (@in.igormarinelli).
- [Business] Rodrigo Terron is a computer science graduate and now the CEO and Co-Founder of Shawee. He is looking to bring its activity to the US, starting from the San Francisco Bay Area (@in.rodrigoterron).
- [Design and Business] Rafael Rejtman is also a USP student on an exchange visit at UC Berkeley. Having done research in Belgium and China, Rafael now works at BNP at their Galvanize SF location. Recently, both he and Igor were selected to be finalists at The Brazil Conference at MIT in April. (@in.rafaelrejtman)
Our team came together as starting entrepreneurs, engineers and students in Silicon Valley looking for challenges and free food ;). DevWeek could not post a better offer!
Give us the pitch for your project!
The market has changed in many ways. We've seen the rise of huge tech companies and the ability that they provide to keep our lives entertained and full of new content, especially through advertising. But still, today, targeted advertising is almost exclusively done online. We've created a system that allows you to change the game, providing offline targeted marketing, by what we call Random Acts of Kindness.
We've developed a platform that allows you to create and manage a marketing campaign, choose your specific demographics of interest (say, women who are up to 27 years old in the Bay Area). Once your campaign goes live, that's when the magic happens: our low-cost hardware, installed in hundreds of point-of-sale (POS) terminals, uses Clarifai to detect and predict the demographics of every purchasing customer, compare that data to all the active campaigns’ demographic targets, and then randomly distribute a financial prize to some lucky ones who match.
The result is that if you're in a targeted group and have a little luck, a company might just surprise you by paying for your coffee one day, even if you’ve never been a paying customer before.
What inspired you to take on this project?
We heard a story from a friend that really inspired us to take on this project. The friend was at Newark Airport for a flight and wanted to get some dinner. Using the integrated system the airport provides, they ordered dinner from a restaurant. As they proceeded to pay by sliding their airline’s rewards card, they were surprised with a message "As a great customer, we'd like to offer you this dinner on us. Thank you for choosing us". The experience felt magical, and our objective was to bring this same magic to customers in every store. This is where the inspiration for the name "Genie" came from, we’re granting customers’ wishes, like magic.
Why did you use Clarifai?
Clarifai offers an incredibly easy and accessible solution to solve our core technical challenge: identify and match customers according to their physical characteristics and demographics. The API proved invaluable in developing our solution.
How does it work?
Our system has essentially three levels: 1) A customer-facing interface, at the POS terminal, which identifies the purchase being made, and has a camera that can see the customer's face. 2) An online platform and dashboard where companies can manage and create Marketing Campaigns. 3) A controlling layer to randomly select customers, match them to their particular demographics using Clarifai’s image recognition, and connect them to the correct marketing campaigns and budget.
Can you share technical implementation details?
We implemented this project in just eight hours! We used two devices, a Raspberry Pi, and a website to perform our flow. First, an administrator (e.g. a Netflix administrator) will register their marketing campaign parameters via our website (store, age range, gender, and location). At the store’s POS device (RaspberryPI 01), when the customer tries to pay, the device takes the customer's picture and sends the stream's image to the Clarifai’s Face Detection API. We register the data to Firebase in realtime. On the other device (RaspberryPI 02), we receive the event from Firebase and perform our match algorithm to choose if this customer is within target parameters and could be a winner. Supposing they are, our device sends an event to Firebase to say: "We've got a potential winner!". At the POS (RaspberryPI 01), the customer could see a picture with the message: "It's on us: Netflix". Our website receives Firebase events as well when we have a new winner so the administrator could see in real-time when we reach a new customer.
Technologies we used:
- OpenCV Node.js library: We used OpenCV to identify when we have a new person in front of our RaspberryPi 01.
- Node.js to handle the RaspberryPi's native modules such as the camera.
- ElectronJS to show the final result on the screen.
Clarifai Face Detection API: We extended Clarifai’s pre-trained face detection model to get information about gender and age from images taken by the camera. - Firebase database and real-time database: We used the real-time feature to send events between the devices, applications, and database to register our website campaign parameters.
An innovative idea with an impressive implementation! We’re excited to see what the future holds for Genie and the thoughtful idea that powers it. Congratulations again to the whole team!
Genie built their computer vision application in a weekend. It takes only minutes to implement Clarifai face detection and other pre-trained models. Try it out for free with 1,000 free operations per month on the house, and see if you can get as creative as the Genie team!
