The average person can identify a familiar face in a crowd with 97.53% accuracy. You will say that this is a lot and you will be right. But this is nothing compared to modern algorithms, which achieved 99.8% accuracy back in 2014. And in the last few years they have reached almost perfection! The modern algorithm used in video surveillance cameras in Moscow is capable of processing 1 billion images in less than half a second with an accuracy close to 100%.
This algorithm is so cool that already this year they are planning to introduce a facial recognition system in the Moscow Metro - FacePay. At the same time, we are promised that the system will work even if the person is wearing a medical mask.
As you understand, life will never be the same. So let's figure it out:
Computer vision and facial recognition technologies have developed very actively since the middle of the last century. But only now they started to work really well. There are three reasons for this:
All these events made it possible to create almost perfect face recognition algorithms. So let's figure out how they work.
First of all, in order to recognize a face, you must first detect it. The task is actually not trivial. For this we could use trained neural networks, but this is too time-consuming, expensive and resource-intensive. Therefore, a very simple Viola-Jones method, developed back in 2001, is used to detect faces.
How does this thing work?
This algorithm simply scans the image using these rectangles, they are called Haar primitives:
And also these rectangles:
The task of these objects is to find lighter and darker areas in the image that are specific to human faces.
For example, if you average the brightness values, the eye area will be darker than the cheeks or forehead, and the bridge of the nose will be lighter than the eyebrows.
In general, there are many such characteristic features and naturally, not only human faces can have similar patterns. Therefore, the algorithm works in several stages:
First, the first sign is found, the system understands: “There may be a face in this area.” Then she begins to look for the second sign there, and then the third. And if 3 signs are found in one area, you can already confidently say - yes, this is a face! After which the system receives an area of the image that contains only the face.
Having received an area for analysis, the main secret of each recognition system comes into play - the biometric algorithm.
He places anthropometric points on the face, from which the individual characteristics of a person will subsequently be calculated: the shape of the eyes, the shape of the nose, chin, the distance between them, etc. There can be many such signs, up to several thousand. But in general, there should be at least 68 such points.
And then the real magic begins. Ideally, we need a face that looks from the front, that is, directly into the camera. But such luck is rare, especially when it comes to recognizing a person in a crowd.
Therefore, the system performs additional image transformation: rotation and tilt of the head are eliminated. A 3D reconstruction of the face from a 2D image is also carried out. Thus, even if the person in the image was looking sideways, we can still get a clear frontal image, which significantly improves the quality of recognition.
Well, then the most important thing happens. A neural network enters the fray and assigns a feature vector to each face. What it is?
Essentially, it’s just some number that is the sum of the characteristics of the face: the distances between anchor points, the texture of certain areas on the face, etc. There can be many such characteristics. The basic rule: they must describe the face regardless of extraneous factors: makeup, hairstyle, age-related changes.
Well, then all that remains is to compare the resulting vector with a database of other vectors. And you're done. The system has identified you.
In addition to the obvious use cases, in addition to detecting offenders in public spaces and paying for subway tickets. Where and how can these technologies be applied?
Firstly, systems can be configured not for identification but for behavior or mood analysis. In a taxi, you can quickly identify inappropriate drivers or passengers. In stores, you can find sad customers and improve the level of service. Clothing retailers or grocery stores use behavioral cameras to analyze the customer's mood at the checkout. Or, for example, in schools, you can look for bored children and adjust the curriculum. This, by the way, is already being done in China. This is the world of the future, and we already live in it without knowing it.
What can we expect in the future? Facial recognition to unlock your iPhone, log into Windows, or during conferences is a great, convenient technology that makes life easier, and we already use it. But ubiquitous surveillance cameras in cities paint the darkest pictures in the spirit of George Orwell.
This raises the question: is it possible to protect yourself from video surveillance systems? Of course, as technology advances, so do the means to bypass these technologies.
People come up with makeup and jewelry that confuse the face detection algorithm, the same one from 2001, create infrared glasses that illuminate camera sensors, and also make all sorts of creepy clothes and masks.
But by and large, such a bow is likely to attract more attention, and the algorithms will adapt to the deceptions. Therefore, the only way to protect yourself is the law. Businesses do not actively implement facial recognition systems only because it incurs large legal costs. The EU is actively developing a new law, which has already been nicknamed GDPR 2: it will strictly regulate facial recognition systems and other artificial intelligence systems that raise legitimate concerns.
In Russia this is not so good yet. However, domestic companies that are present on the international market will also be forced to comply with the new rules of the game, as happened with the first GDPR.
That is, as you understand, there is a bright side of technology that makes our lives easier and a dark side that brings us closer to the world of big brother.
This algorithm is so cool that already this year they are planning to introduce a facial recognition system in the Moscow Metro - FacePay. At the same time, we are promised that the system will work even if the person is wearing a medical mask.
As you understand, life will never be the same. So let's figure it out:
- How do facial recognition algorithms work?
- Are these algorithms really scary and where are they used for good?
- We’ll also talk about what kind of future we can expect.
Causes
Computer vision and facial recognition technologies have developed very actively since the middle of the last century. But only now they started to work really well. There are three reasons for this:
- Really powerful computers have appeared that can cope with the task. Thanks to Moore's law for this.
- Databases have appeared with photographs of you and me. For that, thanks to social networks.
- And of course, there was a breakthrough in the field of neural networks.
All these events made it possible to create almost perfect face recognition algorithms. So let's figure out how they work.
Stage 1: Discovery
First of all, in order to recognize a face, you must first detect it. The task is actually not trivial. For this we could use trained neural networks, but this is too time-consuming, expensive and resource-intensive. Therefore, a very simple Viola-Jones method, developed back in 2001, is used to detect faces.
How does this thing work?
This algorithm simply scans the image using these rectangles, they are called Haar primitives:
And also these rectangles:
The task of these objects is to find lighter and darker areas in the image that are specific to human faces.
For example, if you average the brightness values, the eye area will be darker than the cheeks or forehead, and the bridge of the nose will be lighter than the eyebrows.
In general, there are many such characteristic features and naturally, not only human faces can have similar patterns. Therefore, the algorithm works in several stages:
First, the first sign is found, the system understands: “There may be a face in this area.” Then she begins to look for the second sign there, and then the third. And if 3 signs are found in one area, you can already confidently say - yes, this is a face! After which the system receives an area of the image that contains only the face.
Stage 2. Anthropometric points
Having received an area for analysis, the main secret of each recognition system comes into play - the biometric algorithm.
He places anthropometric points on the face, from which the individual characteristics of a person will subsequently be calculated: the shape of the eyes, the shape of the nose, chin, the distance between them, etc. There can be many such signs, up to several thousand. But in general, there should be at least 68 such points.
Stage 3. Correction of distortions
And then the real magic begins. Ideally, we need a face that looks from the front, that is, directly into the camera. But such luck is rare, especially when it comes to recognizing a person in a crowd.
Therefore, the system performs additional image transformation: rotation and tilt of the head are eliminated. A 3D reconstruction of the face from a 2D image is also carried out. Thus, even if the person in the image was looking sideways, we can still get a clear frontal image, which significantly improves the quality of recognition.
Stage 4. Face vector
Well, then the most important thing happens. A neural network enters the fray and assigns a feature vector to each face. What it is?
Essentially, it’s just some number that is the sum of the characteristics of the face: the distances between anchor points, the texture of certain areas on the face, etc. There can be many such characteristics. The basic rule: they must describe the face regardless of extraneous factors: makeup, hairstyle, age-related changes.
Stage 5. Identification
Well, then all that remains is to compare the resulting vector with a database of other vectors. And you're done. The system has identified you.
Where and how is it used?
In addition to the obvious use cases, in addition to detecting offenders in public spaces and paying for subway tickets. Where and how can these technologies be applied?
Firstly, systems can be configured not for identification but for behavior or mood analysis. In a taxi, you can quickly identify inappropriate drivers or passengers. In stores, you can find sad customers and improve the level of service. Clothing retailers or grocery stores use behavioral cameras to analyze the customer's mood at the checkout. Or, for example, in schools, you can look for bored children and adjust the curriculum. This, by the way, is already being done in China. This is the world of the future, and we already live in it without knowing it.
What will happen in the future?
What can we expect in the future? Facial recognition to unlock your iPhone, log into Windows, or during conferences is a great, convenient technology that makes life easier, and we already use it. But ubiquitous surveillance cameras in cities paint the darkest pictures in the spirit of George Orwell.
This raises the question: is it possible to protect yourself from video surveillance systems? Of course, as technology advances, so do the means to bypass these technologies.
People come up with makeup and jewelry that confuse the face detection algorithm, the same one from 2001, create infrared glasses that illuminate camera sensors, and also make all sorts of creepy clothes and masks.
But by and large, such a bow is likely to attract more attention, and the algorithms will adapt to the deceptions. Therefore, the only way to protect yourself is the law. Businesses do not actively implement facial recognition systems only because it incurs large legal costs. The EU is actively developing a new law, which has already been nicknamed GDPR 2: it will strictly regulate facial recognition systems and other artificial intelligence systems that raise legitimate concerns.
In Russia this is not so good yet. However, domestic companies that are present on the international market will also be forced to comply with the new rules of the game, as happened with the first GDPR.
That is, as you understand, there is a bright side of technology that makes our lives easier and a dark side that brings us closer to the world of big brother.
