Modern computer vision has provided states with unprecedented opportunities in the field of video surveillance. Authoritarian (or authoritarian-leaning) regimes seem to be the best at exploiting this. They are trying to fight deanon in a variety of ways - from laser pointers to special makeup and jewelry. However, if in Hong Kong protesters wear masks and scarves, in Moscow law enforcement officers themselves are massively switching to balaclavas. However, against new, only emerging face recognition systems, neither one nor the other, apparently, will not help.
The SenseVideo video surveillance system from the Chinese SenseTime Group is one of the most expensive computer vision startups. The system not only detects and classifies objects, but also automatically recognizes pedestrians.
More than 200 million cameras are currently in operation in China, and by 2020 their number is expected to reach 400 million. The Moscow government is also trying to keep up with China. In June 2019, the mayor of the capital, Sergei Sobyanin, said that in the near future more than 200 thousand video cameras throughout the city will be connected to the face identification system. IVA Cognitive and NTechLab are participating in the competition for its development. The latter is widely known for its algorithm and (now closed to ordinary users) FindFace service.
Experiments on the use of a face recognition system in the capital, carried out by the Moscow Department of Information Technology (DIT) in conjunction with the Ministry of Internal Affairs, have already shown impressive results. According to the police, since 2017, thanks to the installation of thousands of cameras near the entrances of residential buildings, more than 90 wanted criminals have been detained. The use of video monitoring and face identification systems at several Moscow metro stations results in five to ten arrests every month.
However, face recognition does not have to be tied to fixed cameras. Systems capable of identifying the right people in a crowd can be mobile. For example, in the same China, portable recognition systems resembling smart glasses Google Glass have been tested since the beginning of 2018. Functionally similar, but significantly less futuristic mobile systems will soon be received by the Russian police: in May 2019, the company Zhejiang Dahua Technology and NTechLab, a part of Rostec, offered the security forces wearable camera with face recognition function. The solution is quite practical: the camera-video recorder has an Android 6.0.1 operating system, face detection takes place inside the device itself, and already cut "portraits" with additional metadata are sent to the server for identification. According to a Vedomosti source, the police have already begun testing the devices; similar devices have been announced by NTechLab's competitors, VisionLabs.
The introduction of such systems can have far-reaching consequences for the participants in mass actions. According to statistics presented in the framework of testing the same pilot projects of the Department of Information Technology and the Ministry of Internal Affairs, fixed cameras alone made it possible to identify and detain more than one and a half hundred criminals “on the run” at mass events; if data from portable cameras were added to them, the number of detainees could be much higher.
How recognition works
Human face recognition technology consists of two separate and very different stages of complexity: detection and recognition itself. In the first of them, a computer algorithm analyzes the video sequence and tries to find the area in the image where the face is. In principle, this can be done manually - if there is a snapshot of a specific suspect who needs to be “pushed through the database”. However, in most cases, recognition systems work with a continuous video stream, so simple automatic algorithms are used to find a face in a frame.The main one is the Viola-Jones method, created back in 2001, which is now built into every smartphone and many cameras. The algorithm pays attention to the ratio of bright and dark areas in the picture and looks for areas in it where the relative position of the areas is similar to a face. This is a very simple method - its simplicity, on the one hand, makes the algorithm very fast and economical in terms of computational resources, but on the other hand, it makes it easy to deceive automation and avoid detection. This can be done with the help of makeup (which changes the position of bright and dark spots on the face), a deep hood, a hat pulled down over the eyes, and other simple means.
Whether or not Viola-Jones can be deceived is easy to check for yourself: just try to take a selfie on your phone or upload a picture to a social network - the automatically detected face is usually highlighted with a frame. However, even if detection does not work, it is important to remember that your face can be manually cut out of the frame. And then it will still go to the next stage - recognition.
After detecting the face area, the image is first converted from color to gray (brightness matrix), then the location of the eyes is determined, cropped along the face contour (in fact, the contour features do not play any role for recognition, so standard "ovals" are loaded into the system). Finally, this already rather schematic image turns into a numerical vector - a sequence containing at most several hundred parameters that distinguish the face of one person from another.
Converting a face image into a numerical vector is the most difficult stage of the technology. It is usually assigned to convolutional neural networks, whose task is to neutralize differences in illumination, foreshortening, facial expressions and other features of a particular photograph and to reduce the image of a person to a set of features that are as similar as possible for different images of one person and differ as much as possible for different people.
After this set of features is found, the recognition problem becomes trivial: the obtained values are compared with other vectors stored in the database, and the closest one is found among them. If the proximity exceeds a certain predetermined threshold, the system signals a match, and then gives the user an identifier and a file with a personal file attached to it.
In order for the system to detect a person, his photographs must already be loaded in the database and specific vectors of signs created for them. The more such photographs there are, the more accurate the resulting vector - and the easier it is for the system to recognize you. The best sources for finding them are social networks, the bases of the Ministry of Internal Affairs, the FSB, the Federal Migration Service or companies. Bookmakers own huge databases of photographs of faces (registration requires taking a selfie with a passport in hand); banks offering pay-by-face service; services for the selection of hairstyles and makeup; applications like Msqrd or FaceApp.
Countermeasures: lasers and balaclavas
If your pictures do not exist and have never been in any database, they will not be able to identify you. But for this it is necessary not to leave digital traces at all, which is practically impossible - if only because for this it will be necessary, for example, to refuse to obtain a passport. Can you try to protect yourself from being identified?Methods for countering identification can be conditionally divided into two groups: the first do not allow algorithms to detect a face in the video, the second do not allow it to be correctly recognized.
For example, participants in recent protests in Hong Kong have come up with two simple and practical Chinese ways to avoid face detection. One is to wear balaclavas and face masks. It is important to note that regular gauze bandages for the mouth and nose will not help, as the key information for the algorithms is related to the area of the eyes, brow ridges, eyebrows and the upper part of the bridge of the nose ... Standard sunglasses won't help either. When worn individually, the systems successfully identify the identity of the rioters. However, their combination and the use of a hood allows in some cases to avoid automatic detection and identification - at least with the help of modern industrial systems.
However, this method has been inapplicable for Russian protesters for some time now: according to the law of the Russian Federation "On Assemblies, Rallies, Demonstrations, Processions and Picketing", their participants are prohibited from using "masks, disguises, and other items specially designed to make it difficult to establish their identity. "
Another method is to illuminate lenses with laser pointers. In Hong Kong, they are even used against police officers, shining in the eyes and interfering, for example, with aiming. The effectiveness of this method was confirmed to Meduza on condition of anonymity by a video analytics expert from a Moscow firm that develops automated systems for working with clients in banks and retail. Even simple exposure to bright sunlight on a fine summer day confuses algorithms, he said. And the use of lasers gives, instead of a normal frame with an image, only a few blurry light spots on a dark background.
An indirect confirmation of the effectiveness of simple illumination is also in the design of automatic vehicle classification systems (ACTS): when entering toll roads, cameras are always duplicated on both sides in order to avoid errors due to frontal illumination during the rising or setting sun ...
Countermeasures: makeup and jewelry
In addition to masks explicitly prohibited in Russia, as well as laser pointers, the use of which can be interpreted as insubordination or even an attack on police officers, there are other methods to reduce the likelihood of their identification - which are in the "gray "legal zone. These include the use of makeup, wigs, glasses, bright makeup - in a word, everything that is called disguised face identification in the field of computer vision. Unlike “conventional” recognition, in this case there are no well-developed, high-precision and ready-to-implement industrial systems so far. Different teams of engineers test different approaches, and sometimes they produce very different results.The modern level of detection quality by industrial systems allows one to evaluate, for example, the report of the Russian company Videomax, in which specialists tested the Face-Intellect analytical software package for face recognition (developed by Axxon, a division of the British ITV -Group). It turned out that no false mustaches, beards, dark and transparent glasses could deceive the algorithms. But wearing a voluminous wig reduced the identification accuracy by almost half. The joint use of a wig with long hair, a headdress, adhesive bandages and imitation of bruises on the face allowed the system to identify the user with an accuracy of only 51%.
Oddly enough, the most impressive result was achieved with the use of patriotic makeup: the application of the colors of the Russian tricolor at an angle of 45 degrees to the face and the use of a cap with the inscription "Russia" completely excluded even the detection of a face in the frame. The system just didn't see anything. Whether patriotic make-up refers to means of disguise is not yet clear - in the determination of the Constitutional Court, the use of make-up on the face is allowed, but only for propaganda purposes, and not in order to conceal the identity.
In the same 2017, when Face Intellect was tested, Grigory Bakunov, Director of Technology Dissemination at Yandex, came up with a much more complex method of countering identification. He took into account the principles of neural networks when recognizing the specific features of faces and suggested applying makeup in the form of several, at first glance, chaotically located lines and points. His method not only helped to avoid identification, but also allowed another person to slip into the system. Certain makeup schemes tricked the algorithm into thinking it was a completely different person's face. Bakunov and his colleagues then decided not to create a service based on the invented technology - referring to the fact that "there is too great a chance to use it not for good, but for other purposes altogether."
According to Bakunov himself, the method has two weaknesses. Firstly, such makeup is rather difficult to choose and apply correctly. And secondly, it looks so strange and ridiculous that it can attract undue attention.
A couple of years ago, Polish designers from the Nova company managed to come up with an approach similar to makeup. They have developed so-called face jewelry to confuse identification systems. Their most popular product consists of a kind of spectacle frame, to the lower arcs of which are attached two copper circles covering the cheekbones, and a copper horn rises above the bridge of the nose across the forehead. This aesthetic accessory practically does not cover a person's face, but blocks the facial recognition system - at least the one used by Facebook at the time of testing by the designers.
Countermeasures: glasses and reflectors
One of the most promising approaches to countering face identification can be the use of special high-tech glasses. Some look quite aesthetically pleasing and don't attract too much attention. The person wearing them will not arouse suspicion among the police, they can be worn at rallies and pickets. But the effectiveness of such glasses in some cases is very high, especially against surveillance cameras with infrared illumination of the field of view.So, engineers from the Japanese National Institute of Informatics (National Institute of Informatics) back in 2015 tested glasses in which a set of infrared LEDs was built, invisible to the human eye, but capable of illuminating the area around the eyes and nose for the lenses of photosensitive video recorders. This simple trick made it impossible for the system to even detect a face in the frame.
Tests of one of the first prototypes of glasses with LEDs that interfere with the work of video surveillance systems
A similar approach is taken by engineer and designer Scott Urban. His Reflectacles Ghost and Phantom goggles are framed by many reflectors. They reflect the infrared rays emitted by many types of surveillance cameras (especially those designed for night mode) and turn the face into a large bright spot of light. Another model of glasses from Reflectacles (IRPair) uses the opposite principle: they have built-in special filters that do not allow infrared radiation to reach the face and illuminate it for the camera. In addition, they prevent iris scans and 3D face mapping.
Countermeasures: clothing of many faces
The most extravagant way to get away from the surveillance of "Big Brother" was offered by fashion designers. This is the so-called HyperFace smart textile made for the Hyphen-Labs NeuroSpeculative AfroFeminism project. The task of the HyperFace clothing is to overload the identification algorithm and thereby prevent it from working. To do this, a special pattern is applied to the fabric, imitating many schematic faces. At the same time, each of the patterns is aimed at counteracting one specific algorithm, for example, the Viola-Jones method, or HoG / SVM (Histogram of Oriented Gradients / Support Vector Machines), which is included by default in the OpenCV library ...Strictly speaking, the use of “many-sided clothing” is an exploitation of the old psychological problem of figure and background. The face stands out from the rest of the landscape as a figure, but if the whole landscape consists of faces, then it is no longer possible to distinguish and identify one of them. Everything becomes the background and the algorithm stops working.
Countermeasures: inverse problem
It is not only the state that can use face identification systems against its citizens. Some protesters have recently begun using video and photographic footage from protest actions to de-anonymize police officers who abuse their authority. In response to this, the security forces themselves began to massively use balaclavas, the use of which by ordinary citizens at rallies is prohibited. It seems like this is a guaranteed way to maintain your anonymity. But whether this is really so is a difficult technical question. Based on the results of recent competitions in the field of machine vision, it seems that the answer is rather negative.Before talking about these results, it is necessary to clarify the concept of the so-called. recognition accuracy, which is usually talked about when discussing algorithms. Unfortunately, the “accuracy” itself, that is, the total number of false positive and false negative responses of the system divided by the number of its tests, says almost nothing about whether this algorithm will cope with a specific task. To do this, you must at least know the size of the sample of people, among whom you want to find the owner of the face in the image (or even better understand the cost of the error and represent the area under the ROC curve) ...
Simply put, if we are talking about monitoring millions of people in a continuous stream, for example, video surveillance systems in the subway or at a mass action, then the proportion of false positives in such systems should be very low (and the "accuracy "- high) - otherwise the total cost of responding to each trigger would be too high. If, however, the number of people among whom the system searches can be significantly reduced in some way (for example, to search not among all citizens, but among representatives of a known profession, gender, age, and so on), then the accuracy requirements will be fundamentally lower. ...
One of the main platforms for testing the most modern face recognition systems is contests like Disguised Faces in the Wild . Here and in other similar contests, development teams are invited to solve the problem of identifying a person among images of faces unrecognizably altered by makeup, masks, professional movie make-up, and so on. There is no special data set in balaclavas in this competition, but for greater complexity among the wrong answers there are even images of parodists - that is, the system is asked to guess the face, for example, of Lady Gaga among the actors portraying her ...
The latest results of competitions for the identification of hidden persons indicate the following figures. For a given false positive rate of 1%, the best algorithms give the correct answer more than 90% of the time. If the level of false positives is reduced by an order of magnitude, the result is only 10% worse.
This, of course, is not enough to enable the introduction of hidden face recognition systems in the metro today. But if the task is to search in a much smaller sample, the accuracy of modern methods of recognizing even faces under make-up or mask will, apparently, be quite enough to detect potential candidates for deanonymization. And although there is no ready-made service based on these algorithms now, it may appear very soon.
What to do with the information obtained in this way is another big question, but this is already a matter of ethics, not technology. The principle of ethical neutrality applies to facial recognition algorithms just as much as it does to encryption methods or weapons development.
