Tutorial: Facial Emotion Recognition with Python
Emotion recognition is a part of machine learning that falls within the relatively new research area of artificial intelligence. Today, this technology is used to automate the recognition of facial expressions from images and videos, spoken expressions from audio, written expressions from text, and physiology measured by wearables.
Luxand offers a cloud-based Emotion Recognition API that provides a full range of AI-based computer vision tools for face recognition.
In this tutorial, we will describe how to implement Luxand.clod emotion detection API into your app, software or system using Python.
Emotion Detection with Python
Luxand.cloud offers emotion recognition technology in various programming languages. This guide will explain how to implement emotion detection using Python. According to Luxand.cloud research, Python is a popular choice among developers for implementing the Emotion Recognition API.
Recognizing the challenges that developers may face when integrating emotion recognition, we've created this guide to simplify the process.
What is Emotion Recognition
Facial emotion recognition is a subfield of computer vision that focuses on analyzing facial expressions and inferring the emotions they might represent. It uses advanced machine learning techniques, particularly deep learning, to train algorithms to identify these expressions and map them to specific emotions.
What emotions can be detected? The table below describes the link between emotions and the facial expressions often associated with them. Emotion recognition systems can be trained to identify these expressions, potentially providing insights into emotional states.
Luxand.cloud offers an emotion recognition API that you can easily integrate into your app, software, or system. Our API is trained to analyze the shape of the mouth, the position of the eyebrows, and other facial features to identify and classify different emotions.
Emotion Recognition Use Cases
You can use Luxand.cloud's emotion recognition API for various use cases. Here are some examples:
User Experience. Emotion recognition APIs are used to enhance user experiences in different industries, like entertainment and gaming. For example, virtual assistants can leverage emotion recognition to better understand customers, their mood and adjust their communication style.
Healthcare. Emotion recognition can be used to create a system to assist mental health professionals in identifying patients experiencing emotional distress. These crucial indicators of anxiety or depression can help to treat the symptoms. Additionally, the technology can help to detect signs of pain, which can facilitate faster and more effective treatment.
Security and Safety. Emotion recognition can be used in the security sector for different applications. For example, it can be used to identify potential security threats at airports or other public spaces by analyzing facial expressions of individuals.
Luxand.cloud Emotion Recognition API
Luxand.cloud's emotion recognition API offers a wide range of functionalities:
Upload photos from various sources. The API allows uploading photos from both the local Luxand.cloud database and from web resources.
Real-time processing. You can expect instantaneous processing and response from the API.
Multi-face detection and emotion recognition. The API can detect multiple faces in an image and analyze the emotions of each person separately.
Face location. The API also determines the coordinates of the detected faces within the image.
We'll demonstrate using the Luxand.cloud face emotion recognition API with Python as an example, and we'll use the standard synchronous algorithm for making requests to the REST API service, which is adaptable to various modern programming languages.
Implementing Emotion Recognition API using Python
Preparation
To use the REST API, you need to install any of the frameworks for making HTTP requests to the Luxand.cloud API.
In this example, we will use the requests module because Python Requests is a library that is designed to make working with requests quick and easy. Standard Python HTTP libraries, such as Urllib3, often require significantly more code to perform the same action, which makes it difficult to work with.
To install the library from the command line, you need to enter:
pip install requestsThe json module, which is included in the standard Python library, is also required by the API server to process responses.
Do not forget to include the specified libraries in your code.
import requests
import jsonAuthorization
You need to obtain authorization data for the Luxand.cloud API. To get access, you need a token, which is available in your Luxand.cloud dashboard. Each request to the Luxand.cloud API must be accompanied by an HTTP header with the "token" parameter set to the token value. The serialization of such a header in Python will look like this:
headers = {
"token": token
}Image Upload to Database
The Luxand.cloud API can process both web URLs of the analyzed image and work with images pre-uploaded to the Luxand.cloud service.
For full use of all Luxand.cloud functionality, it is more convenient to pre-upload your image database (dataset) that we will be working with. Uploading an image to the Luxand.cloud service is done by a POST request to the URL: https://api.luxand.cloud/subject/v2
This request must be accompanied by the "name" parameter in the request body with the name of the uploaded file, the file itself is attached to the request as a byte string (multipart/form-data body type). The implementation in Python will look like this:
headers = {
"token": token
}
data = {
"name": name
}
files = {
"photo": open(image_path, "rb")
}
response = requests.post(url, headers=headers, data=data, files=files)The request must be accompanied by the following parameters:
name - The name of the file.
image_path - The full path to the file on the system where the code is running.
token - The token for accessing the API.
The result of this request must be accepted in order to extract the 'id' field, which is the identifier of the uploaded file in the Luxand.cloud system.
In the event of an upload error (incorrect format or file size too large), the service will return an error with its definition in the 'error' field of the response.
In Python, the response can be processed as follows:
result = json.loads(response.text)
if response.ok:
print(result['id'])
else:
print(result['error'])The Use of Emotion Recognition Functionality
Now everything is ready to use the Luxand.cloud API. Accessing the API is done via the POST method. The request body is formatted similarly to how you would upload an image to a database (discussed earlier). You can learn more about the specific method for emotion recognition here: Recognize Emotions in a Photo. The pipeline for using this method, and others, is standardized:
Method POST
Header
headers = {
"token": token
}The request body (multipart/form-data) is the byte contents of the image file in the 'photo' field or a link to the image from a web resource.
The implementation of such a request in Python using the requests module will look like this:
headers = {
"token": token
}
files = {
"photo": open(image_path, "rb")
}
response = requests.post(url, headers=headers, files=files)image_path - The full path to the file in the system where the code is running.
token - The token for accessing the API.
If you are working with a web URL of an image (without first uploading it to the Luxand.cloud database), the body of the request will look like this:
files = {
"photo": “https://domen/image.jpg”
}"https://domen/image.jpg" is the web address of an image. As you can see, in this case all the other instructions remain the same, only instead of the byte content of the image we specify its URL.
In response, the service returns a JSON object:
{
"status": "success",
"faces": [
{
"rectangle": {
"top": 321,
"left": 329,
"bottom": 678,
"right": 686
},
"emotions": {
"happiness": 0.324,
"neutral": 0.676
}
}
]
}Field values are as follows:
'status' - status or result of the operation, example - "success"
'faces' - a list of detected faces in the image, each element of this list defines the following parameters:
'rectangle' - coordinates of the detected face square in parameters - top, left, bottom, and right
'emotions' - detected face emotion in terms of categorical parameters - happiness, neutral … with an indication of the degree of emotion correspondence of each of the found categories from 0 to 1.
In Python, you can process the response as follows:
result = json.loads(response.text)
if response.ok:
print(result['faces'])
else:
print(result['error'])In addition to the basic functionality of the Computer Vision API class, Luxand.cloud provides auxiliary methods that simplify the use of the main functionality. For example, the Luxand.cloud API allows you to upload your own collection of photos of people and their faces. You can create a photo gallery from this database and manage it.
The Full Code
The full version of the Python code implementing the example of emotion recognition discussed in the article is as follows:
import requests
import json
import sys
HEADERS = {
"token": 'XXXXXXXXXXXXXXXXXX'
}
def add_face_to_service(headers, image_path, image_name):
url = "https://api.luxand.cloud/subject/v2"
data = {
"name": image_name
}
files = {
"photo": open(image_path, "rb")
}
response = requests.post(url, headers=headers, data=data, files=files)
result = json.loads(response.text)
if response.ok:
print(f"Ok adding face: {result['id']}")
return result['id']
else:
print(f"Error adding face: {result['error']}")
return None
def recognize_face_emotions(headers, image_path):
url = "https://api.luxand.cloud/photo/emotions"
files = {
"photo": open(image_path, "rb")
}
response = requests.post(url, headers=headers, files=files)
result = json.loads(response.text)
if response.ok:
if result['status'] == "success":
print(f"Recognized face and emotions: {result['faces']}")
else:
print("Nothing found")
else:
print(f"Error recognizing face emotions: {result['error']}")
if __name__ == '__main__':
#Step1 - Prepare our image database to search faces within it
file_to_base_path = 'base_file.png'
file_to_base_name = 'Mr.Test'
id = add_face_to_service(HEADERS, file_to_base_path, file_to_base_name)
if not id:
sys.exit()
#Step2 - Recognize particular face emotions from image file
file_to_recognize_path = 'recognize_file.png'
recognize_face_emotions(HEADERS, file_to_recognize_path)
sys.exit()Conclusion
As you can see from this tutorial, to implement Luxand.cloud emotion recognition API is easy and fast following the steps. In conclusion, we want to provide some emotion recognition benefits.
The verification algorithms used to match objects to reference images (such as a passport photo or a snapshot) allow for accuracy up to 99.97% in standard benchmarks such as the NIST FRVT test.
This is comparable to the best results of iris scanners. Therefore, even the financial sector is switching to face recognition for customer authentication.
Image processing takes less than 1 second! Moreover, during this time the algorithm itself can query a database with photos at a speed of 100 million images in just one tenth of a second.
Luxand.cloud API does not impose high requirements on the resolution of the processed image, files with a small size and resolution are processed just as accurately, the main thing is to observe the condition that there are at least 75 pixels between the eyes of a person. The Luxand.cloud API can recognize images up to 12 KB in size.
And of course, all data and images you use in Luxand.cloud API are completely confidential. They are not stored in the service database as is, only their digital vector is stored, which cannot be converted back into an image. Thus, complete data security is achieved when using Luxand.cloud API. And yes, the service has the ability to deliver the solution locally, and then it can be used even without access to the Internet.
Here are some of the key benefits of Luxand.cloud API:
AI-powered. Luxand.cloud emotion recognition technology is AI-powered. Emotion recognition technology is based on the premise that human emotions can be expressed through facial expressions, and that these expressions can be recognized and classified using machine learning algorithms.
Easy to use. As you can see, Luxand.cloud emotion recognition API is easy to implement following the steps that we described.
Free plan. You can use Luxand.cloud for free! To build your app or software, just create an account and get 500 API requests monthly.
High accuracy. To identify emotional patterns, the algorithms are trained on large datasets containing labeled facial expressions. This process enables the system to learn how specific facial muscle movements correlate with different emotions.
Create a free account, follow the steps and enjoy the emotion recognition technology!