Openncc Nighthawk

Details

A night vision & face privacy protection application of OpenNCC

- Model-on-OpenNCC: face detection model, pixel substitution algorithm

- Hardware components: OpenNCC core board, IR LED kit

- Benefits: light pollution-free, face privacy safe

- Why this project?

By 2025, it is expected that there will be 1 billion cameras in the public and private fields. The edge-ai cameras with deep learning and visual analysis will also develop rapidly with the maturity of artificial intelligence technology and the improvement of computing power. At the same time, there will be a large number of application scenarios requiring privacy protection, such as:

• Privacy and identity protection in the public / private sphere
• Online protection of children (i.e. Blur the faces of minors when uploading photos)
• Hospital bed care for medical health
• Online fitness training guide
• Elderly care monitoring, fall detection alarm

Among them, in the field of medical treatment and nursing, cameras are often required to have good night vision ability, so that they can adapt to 24-hour coverage. Therefore, providing an edge AI face blurring night vision open camera that can meet privacy requirements can support developers of these scenarios to integrate their own applications.

Data security is being paid attention to by countries all over the world. More than 100 countries are actively dealing with data security issues and legislating for protection, such as:

• GDPR（Europe）
• Privacy Act 1988 (Australia)
• Personal Information Protection Act (Japan)
• Personal Data Protection Act (Singapore)
• The Cybersecurity Law of the People's Republic of China (China)

Vision is a kind of natural language. The face is our most personal feature of privacy. Protecting people's face information makes us more at ease and makes artificial intelligence better serve everyone in the field of vision.

How to implement a vision system with face privacy protection? We need to better understand the protection of face privacy from the composition of modern intelligent camera systems.

- How?

Generally speaking, a complete edge-ai camera is composed of a lens, photoelectric sensor, image processing chip, VPU/NPU processing unit, and some external interfaces. In daily applications, the camera will display photos and videos on the terminal computer, or transmit them to the cloud through the internet. 

From the above workflow, we can find the camera is the simplest place where the data can be desensitized directly. To develop a privacy protection camera with blurred human faces, first of all, we need it to have basic imaging functions and an open ability to allow developers to deploy and integrate algorithms.

Deploy Face blurring algorithm on Open Camera

OpenNCC(Open Neural Compute Camera) is an edge AI camera based on Intel Movidius VPU. It is compatible with the deployment of Intel OpenVINO, is also an open camera. There is a lot of source code for reference on GitHub so that developers can deploy different deep learning models to achieve differentiated functions and performance. This brings us a good development foundation for realizing such a human face blurring camera. We only need to care about the implementation and application of the algorithm, and we don't need to spend a lot of time on the optical and hardware design of the camera and the basic imaging of the camera.

From the perspective of embedded software, we need to complete the following steps to realize face blurring:

Deploy Face Detection Model on OpenNCC

Realize face detection on the Edge-AI camera, and also obtain the coordinates of the face.

OpenNCC camera supports OpenVINO. We select an object detection model from Intel's pre-trained model zoo of Intel OpenVINO Toolkit to implement face detection, of course, the model you trained yourself also could be used. To realize real-time privacy protection, we need to achieve a real-time face detection frame rate of 30 fps. Model training and clipping need to be balanced, as well as performance and efficiency. In the actual productization process, developers need to train and optimize a more ideal model to deploy.

Finally, we chose the face-detection-retail-0004 model to demonstrate the deployment for the demo. The performance of the model is as follows:

After deployment, this model can be used to realize real-time face detection of 30fps in the case of HD 1080p. For specific deployment steps, we refer to the OpenNCC blog article: How to deploy an intel open zoo model’.

Face blurring on camera

We get the coordinates of the detected single or multiple faces, which consists of the start X, end X, start Y, and end Y of the face bounding box. Then, we can use this information to extract the facial ROI itself, as shown in the figure below.

Finally, we use an efficient and simple pixel substitution algorithm to realize face pixel blurring on OpenNCC. The effects are as follows:

IR LED Night Vision Principle and Implementation

The CMOS sensor of the Edge-AI camera is like human eyes, which converts light into digital images. The frequency of the electromagnetic waves that can be perceived by our eyes is 380 ~ 750 THz and the wavelength is between 400 ~ 780 nm. As shown in the figure below, the sensing curve of a sensor used by OpenNCC has a good response rate in the band perceived by human eyes. However, at night, the external light is much weaker. For a camera with a fixed aperture, it can't get as much light as the human eyes. In this case, we need external lighting to help improve the lighting conditions and ensure the imaging quality.

One way is to use the visible light within the band of 400-780 nm to supplement the light, just like our home lighting. However, in some special application scenarios, strong supplementary light at night will form light pollution and affect everyone's rest. In this case, we need to use infrared light to fill the light. As shown in the figure below, we find that there is a small response peak near the 850 nm infrared band, so we can use the 850 nm infrared light to fill the light for the edge AI camera. This band exceeds the most sensitive sensing range of the human eyes and can well balance the problem of light filling and environmental light pollution.

OpenNCC camera provides such IR LED PCBA, which is easy to integrate into standard OpennNCC components, supports automatic detection of external light, supports automatic switching between day and night, and ISP optimization ready for day and night respectively.

OpenNCC SKU Option

Face blurring algorithm with IR LED night vision is deployable to the following camera:

• OpenNCC Nighthawk