Assistant or home spy: Do smart speakers eavesdrop on their owners?

[Father]

In recent years, the network has received a huge amount of data about Russians. No one is surprised that scammers know not only the full name of their "client", but also the list of accounts, some personal data, up to the place of residence.

Against this background, the demand for privacy is growing, and along with it, distrust of various modern technologies, the so – called "Internet of things". The biggest questions are raised by the "flagships "of IoT – smart speakers, because they" hear " literally everything that happens in a room or the entire apartment.

This article will discuss the main aspects of smart speaker privacy and the likelihood that the data they process may be accessed by third parties or used by the manufacturer for third-party purposes.

What the smart speaker hears​

Ordinary users of smart speakers often have a question – at what point their device "activates" and starts listening to conversations. It is widely believed that the device is in inactive mode before the code phrase is uttered, and this is true.

Nikolay Khechumov
Information Security Engineer, Avito

There are no miracles, and to turn on the speaker by voice command, you need to continuously analyze the stream from the microphone. Another question is what exactly the column does with this stream. The most obvious and correct answer is that speech is recognized locally, but the sound itself is not transmitted or stored in the background. However, nothing prevents any manufacturer from making changes to this logic with the next update.

However, the background mode does not mean that the device does not hear what is happening around, and does not analyze this speech. A case from five years ago, when a smart speaker from Google independently called the police during a domestic quarrel, was widely distributed on the web.

Therefore, we can assume that any speaker has a set of trigger words, for each of which there is a certain algorithm of action, and it is not limited to activation when directly "accessing" the device.

Based on this information, we can distinguish three algorithms for interacting with a smart speaker:

1. Zero trust. After interacting with the speaker, the user must disconnect it from the network.
2. The best option. No manufacturer will record, store, or process all the speech of all users, since the "efficiency" of this process will always be low. Accordingly, as long as the user does not discuss "trigger" topics in the action field of the column, there are no risks.
3. Absolute trust. In this case, the smart speaker can be safely installed directly in the CEO's office or meeting room.

It is also important to separate real risks and conspiracy theories. Theoretically, it is very likely that any speaker manufacturer is somewhat dependent on one state or a whole group of countries in whose jurisdiction it promotes its products. It can be assumed that, at the request of the special services, they will be required to provide conditionally legitimate access to systems, possibly even to data from a specific device. However, the security services are guaranteed to have exactly the same access to a mobile phone, social media data, search queries, a suspect's PC, and a number of other data devices.

The capabilities of the special services are very extensive, but at the same time they are quite limited, since setting up a "panopticon" with total surveillance of everyone is quite simple in theory and extremely difficult in practice, when the entire amount of data received needs to be stored somewhere, archived, but most importantly-quickly processed in order to search for "target content".

Alexey Lazarev
Head of the Rutoken Module department at Aktiv

Can special services use these features upon special request? We don't know for sure. Whether this can be used by an ordinary attacker who does not have administrative access to the global system or access to your personal account. No. Any modern system, especially one such as the Yandex infrastructure, contains reliable mechanisms that prevent external attacks, both at the endpoint level and at the system level as a whole. Leaks of user information usually occur through insiders who have access to the processing of user data. And any owner of a database with users ' personal data is fiercely fighting them, since incidents with leaks are a powerful blow to the reputation of a business.

In the arsenal of attackers, there are a million other, cheaper ways to get hold of user data, about themselves, location data, and voice biometrics. These methods are not without success used by phone scammers. However, in most cases, this data is provided by the users themselves. And to prevent your account from being hijacked by getting a username and password, we recommend using strong two-factor authentication or authentication with a one-time password.

Can a hacker use a smart speaker to listen in​

If you approach this question as a mathematical problem, then, with a high degree of probability, it has a solution. That is, a cybercriminal, in theory, can hack the device and start "listening" to the user's speech. However, if you return from the world of equations to reality, the question arises: who needs it and why?

If an attacker was able to break into an internal network, private or corporate, he has dozens of easier and more effective ways to benefit from this hacking than messing with a smart speaker, siphoning data from it and, most importantly, spending a huge amount of time analyzing the information received, which may well turn out to be "white noise" – unless the user is in the habit of saying all their passwords, logins, and corporate accounts out loud over their morning coffee.

Dmitry Ovchinnikov
Chief Specialist of the Integrated Information Security Systems Department of Gazinformservis

The danger of smart speakers is greatly exaggerated. If an attacker gets into your home network, then there are more interesting things there than eavesdropping on household conversations through a smart speaker. In addition, there are easier ways to deceive people for money - these are phishing, social engineering, cryptographic viruses and much more. Well, if we assume that your speaker has been hacked and I'm eavesdropping on you, then this is absolutely not effective for intruders, since tracking via audio takes a long time, and the result is not guaranteed.

The only major danger may be the integration of a smart speaker into a smart home system. However, with this approach, an attacker who has access to the speaker will also have access to the smart home infrastructure. But protecting a smart home is a completely different story and a topic for a separate article.

It can be assumed that hacking a smart speaker would be relevant in the case of a targeted attack, when the target is, for example, a politician or a top manager of a large company, or just a well-known person. The threat model for such individuals differs from the cyber risks of a standard network user. However, as practice shows, even in such cases, attackers are more likely to "target" smartphones, which is banal because the user interacts with them more often, which means that they are more likely to implement an attack through phishing mailing lists or other social engineering methods.

What about Russian speakers?​

The leak of source code from the repository of one of the major manufacturers of smart devices, which the IT community has already managed to call "the biggest contribution to open source", has again revived the debate about how safe it is to use their product. The reason for this was found in the analysis of the leak phrases that users use to activate the column.

Alexey Drozd
Head of the Information Security Department at Serchinform

Technically, there are no obstacles to this. For example, "Alice" listens to the user while waiting for the activation command. But after analyzing the leaked source codes from Yandex, the researchers found out that this happens locally, that is, the data is not transmitted anywhere. Voice commands are transmitted externally only when the device is active. On the other hand, it all depends on the implementation – there are also reverse examples. You can recall the scandal with smart doorbells Ring, which automatically sent data without the knowledge of users, and they could not affect it in any way.

You need to rely on the integrity of the manufacturer. It is good if there is an "analog" way to make sure that the device does not "write" anything. For example, a physical switch, a button that disables the microphone. If the manufacturer also transparently declares this – like the same Yandex, which openly published the microphone operation scheme in Yandex. Stations-this is an additional success.

Based on the leak data, we can conclude that the Russian bigtech works only with targeted requests that are necessary for training the device itself. Everything else, in the absence of supporting information, is just a private opinion of individuals, which, in the end, rests on "I believe – I don't believe".

User data as a "white" product​

It has long been no secret that many big techs and large IT companies are extremely interested in analyzing user data for the subsequent creation of targeted and contextual advertising. Whether this is good or bad is a dialectical question, since Bigdata has long been a part of objective reality and an important market segment.

In fact, if a company uses depersonalized data – it doesn't cause any problems for the user, except for the appearance of more ads in their search feed.

Sergey Belov
CEO, AtreIdea

If the data is stored and processed correctly, then depersonalized data must be protected from identifying a specific user. Usually, when a company says that it anonymizes data, it means that it removes information that can identify the user (for example, name, address, phone number, etc.). However, you can use other data (for example, IP address, location, device usage, etc.) in order to: identify the user.

It cannot be ruled out that the company may try to get additional information about users by using depersonalized data and use it to improve its products. However, in a well-managed system that adheres to modern privacy and security standards, there must be measures in place to prevent the user from being identified based on depersonalized data.

In addition, data such as recordings of voice requests may be at risk of unauthorized access or leakage. Therefore, it is important that companies that collect and process user data follow appropriate privacy and security standards to minimize risks to users.

As applied to Yandex and their smart column, the latest leak may indirectly indicate that the company has data depersonalization processes, and they work quite efficiently, since no one was able to identify existing queries with specific users.

However, this does not mean that the process of working with data in this company, as in any other, is a priori ideal. The company itself, according to the results of an internal investigation, stated that it found facts of violation of corporate ethics and a number of other violations related to compliance with the rules for working with code or data.

Results​

A smart speaker can certainly hear a lot of things around it. However, these data, in 99% of cases, are of extremely low interest to the company itself, to law enforcement agencies, or to cybercriminals, so the cost of processing them will always be ineffective, which does not indicate the risks of "targeted wiretapping" for a number of "VIP users".

From the point of view of hacker activity, a smart column is far from the most attractive target, since the required user data can be obtained from other, less hacking – resistant sources, provided that you have access to the network.

The main issue from the point of view of operating a smart speaker is trust in the manufacturer, since it is quite difficult to check the device yourself. However, regardless of the level of trust, you should not put a smart speaker in the middle of the meeting room of the CII subject or in the office of the company director.