Face Recognition Node
The Face Recognition Node manages the attached camera device and uses the camera stream for face detection and recognition. As part of this, it uses AI functions for face detection and recognition.
loading...
Startup
During startup this node initialises the camera stream and starts the camera. If also initializes the face recognition storage. This storage is used during face recognition.
A reference to the Face AI service is created and a thread is created to start processing the camera stream.
Frame Processing
Under the control of a running thread, the next available frames are read from the camera and passed to the Face AI for face detection and alignment.
loading...
For each face found in the frame, face embeddings are found and these are then checked against the face recognition database in an attempt to identify the face:
loading...
For any identified faces, face name and match score meta data is added to the face data. After this, any identified faces are given an overlay label - this is added to the frame, showing the name of the identified person on the displayed frame:
loading...
The bounding box of the face in the frame is further processed to get the face position relative to the frame. This is useful when trying to track a face:
loading...
For any identified faces in the frame a person score is determined. This score tells the system if the identified person is someone of interest or someone to be avoided.
The person score, face name and bounding boxes are then published as a person-detected event:
loading...
Once all faces have been processed in the frame, the frame is published for display and a final check is made on the running status of the robot:
loading...
Face Storage
AI Robot uses lancedb to store face data for known faces:
loading...
The FaceStorage class is used to populate this database with detected faces from images in a given path. This process extracts face embeddings and stores these alongside the name of the person in the image. In this way we build a database of embeddings for a given person and this data can be used when detecting faces:
loading...
In the database we store records of type FaceRecognitionSchema:
loading...
The second part of the face storage class implements the function for identifying a face given face embedding data using a distance based algorithm. That is, for a list of embedding vectors, try to find the best match based on the similarity between the embedding and known embeddings for known faces:
loading...
Face AI
Face detection and recognition is performed using edge AI by the Hailo AI Hat on the Raspberry PI. On this AI device we run AI models found in the model zoo from DeGirum.
Face Detection is managed by the FaceAI class:
loading...
The implementation here is specific to the hardware we are using in the AI Robot - the Hailo AI Hat and is not abstracted.
For more details about the DeGirum Model Zoo, visit the DeGirum AI Hub
Model Initialization
To use the AI models we first initialize the models which results in fetching and loading the model into the Hailo AI:
loading...
Two models are used by the AI Robot:
- scrfd_10g--640x640_quant_hailort_hailo8_1 - used for face detection
- arcface_mobilefacenet--112x112_quant_hailort_hailo8_1 - used to get face embeddings from face image data
Face Detection
Face detection is performed by the detect_faces method:
loading...
This method is passed the camera frame and uses the face_det_model to return a collection of detected landmark objects.
Retuning Aligning Faces
Detected landmarks are further processed by the method get_aligned_faces_from_inference_result to get a list of aligned images from the detected landmarks:
loading...
Face alignment is important for face recognition as it ensures the face is oriented with the landmarks in expected position. For example, faces that are at an angle with respect to the frame are oriented so the face is perpendicular to the baseline. Image alignment is performed by the helper function align_and_crop:
loading...
Get Face Embeddings
The other major function of Face AI is to extract and return embeddings for an aligned face:
loading...
This function uses the face_rec_model to find embeddings in the face image. Note, for performance, this model is called with a batch of faces.
These embeddings are vectors describing key landmarks on the face such as where the eyes are in relation to each other and the other face landmarks. These embeddings are carefully modeled and provide a fast and accurate way to differentiate between one face and another and more importantly, these embeddings are consistent across different images for the same face.
Camera Stream
The Camera Stream class is responsible for initializing the camera and returning frames from the camera:
loading...
The camera steam supports both the PI Camera and a USB camera. On initialization, one of these is loaded based on configuration.
The frame_generator method is called to return the next frame from the loaded camera:
loading...
PI Camera
The class CamPi supports the PI Camera hardware using the Picamera2 library:
loading...
USB Camera
The class CamUsb supports the USB camera hardware using the opencv library:
loading...
Camera Base
The class CamBase acts as a base class to CamPi and CamUsb:
loading...