Supervised web surfing. What data do browser developers collect about us?

[Carding 4 Carders]

When you look at websites, someone is watching you. This has become almost commonplace: the collection of statistics is now embedded not only in web pages, but also in many programs. We conducted a study to find out what exactly the developers of popular browsers learn about you and how much it violates your privacy.

Methodology​

When it comes to web surfing, two fundamentally different types of data collection should be distinguished: one that is performed by the browser itself, and one that is produced by scripts on sites.

OpenStat statistics helped us to identify the circle of the main suspects. We have selected the most popular browsers in Russia, highlighting the versions for Windows desktop computers. The most widespread were Google Chrome (its share is almost half), Yandex Browser, which is annoyingly installed for a company with other programs, Mozilla Firefox and Opera.

The OpenStat list also includes the Apple Safari browser, but its Windows version stopped being updated in 2012 and is almost never used. The Edge browser preinstalled in Windows 10 barely gained one and a half percent of fans, but it was from it that we most expected manifestations of "spyware". From Edge and its big brother Internet Explorer, which is always overstated in popularity due to the ability of different programs to identify themselves as IE.

Browser popularity rating

The "spyware" behavior of browsers was assessed in several stages. First, we downloaded the latest distributions from the official sites, installed them in clean OS and launched them with the default settings. Then they changed the home page to a blank one and repeated the experiment. At the final stage, they arranged an hour of sitting in ambush, during which the browser was simply opened with a blank page (about: blank) and did not have to perform any network requests, except for checking the availability of its own updates.

All tests were run in virtual machines. We had to use both Windows 10 and old Windows XP in order to filter out all background traffic. As you can remember from the article where we examined Windows 10 in a similar way, this OS itself very closely monitors the user and sends all the data that it technically can collect to Microsoft's servers. In this traffic flow, browser activity is simply lost, because Edge (and, as it turned out, not only it) is able to send part of the requests on behalf of system processes, using them as intermediaries. Therefore, simple tools (for example, installing a web proxy and filtering traffic by process names) did not guarantee the ability to catch all the traffic of interest to us.

We had to play it safe and use several tools at once to track the network activity of browsers. TCPView manager showed all network connections in real time. With its help, it was convenient to determine which actions cause the appearance of new connections and which IP addresses are used by the browser most often.

The lion's share of traffic is sent encrypted by browsers. Therefore, using MakeCert, we generated and installed the left security certificate into the system, thanks to which we decrypted all intercepted HTTPS traffic.

In some cases, it was necessary to use the AppContainer Loopback Exemption utility in order to bypass the application isolation technology built into Windows 10 and to ensure that traffic is intercepted using Fiddler. First of all, it was necessary to do this for Edge and Internet Explorer.

Redirect Edge traffic to a local proxy bypassing Windows protection

We also used the Wireshark sniffer for detailed analysis of logs and finding patterns. This is a powerful tool that, among other things, is able to collect individual packets into streams. Therefore, having found one suspicious package, we easily restored the entire process of exchanging the browser with the selected remote host.

These programs have already become the de facto standard for performing tests. However, the programs are limited by the operating system. The Internet Explorer and Edge browsers are so tightly integrated into Windows 10 that they can use Windows 10 components to send data in a workaround way. Therefore, to ensure that no packet went unnoticed, we additionally used a hardware sniffer.

Intermediate router as a sniffer.
It was a portable router TP-Link MR3040 v. 2.5, which we re-flashed with the latest version of OpenWrt and plugged in "in the gap" by selecting the WISP mode. All traffic from test systems went through it. The router showed all network connections in real time and kept a detailed log.

All connections in real time (fragment of the list)

Legalized surveillance​

The very idea that the user's actions at the computer are becoming known to someone else has become familiar to many. In part, people are so calm about this, because they do not understand the volume and nature of the data sent about their activity. The converse is also true: fanatical human rights activists are ready to see a violation of privacy in any dispatch of a log with purely technical information. As usual, the truth is somewhere in between, and we tried to get as close to it as possible.

Most of the users we interviewed believe that everything is limited to some kind of abstraction - "anonymous statistics collected in order to improve the quality of the product." This is exactly what is indicated in the formal warning that browsers (and other programs) display on the screen during installation. However, the wording in them is rather florid, and the long list often ends with the words "... and other information", which completely frees the hands of the developer's lawyers.

Google knows about all contacts, addresses of its users and their health status. Microsoft also identifies handwriting by “handwriting samples”. Free antiviruses (and many paid ones too) can legitimately send any file to their developers as suspicious. Browsers against this background do not look like spies, but rather harmless voyeurs. However, there can be tangible consequences from their peeping. Let's see what and where they send.

Google Chrome​

On first launch, Chrome 56.0 makes nine connections to Google servers across four subnets.

Chrome connections on browser launch

One of the subnets is located in Russia and is served by the Rostelecom provider.

Chrome always connects to these IPs

The browser sends information about its version, OS version and recent network activity of the user to the 173.194.44.0/24 subnet. If it was not there (the first launch of Google Chrome), then the entry "No recent network activity" appears in the log.

Chrome sends a log of its activity

A certificate request is sent to the 46.61.155.0/24 subnet to authenticate the Google.com site and dozens of its mirrors (including statistics collection sites * .gstatic.com, google-analytics.com and others). In the course of further work of the browser, separate connections are periodically established with them.

If you logged in to Google through a browser, then additional traffic will go to the 74.125.232.0/24 subnet and to servers with addresses of the form http://clients#.google.com, where # is the sequence number of the pool. Chrome establishes similar connections with the 46.61.155.0/24 subnet, probably to distribute the load.

When opening a new tab, Chrome would always connect to servers on the same subnets.

Chrome connections when creating a new tab

In this case, the browser generates a unique view identifier X-Client-Data: CJC2yPGIprbJAQjBtskBCK2KygEIwcdKAQj6nMoBCKmdygE=, and the site google.ruadditionally uses cookies with the identifier NID=. All tabs open in the same browser get a common ID X-Client-Data.

From time to time, Chrome established a connection to the server storage.mds.yandex.net, but in our test, except for empty packets with a header connection keep alive, nothing was sent to it. The rest of the traffic, not related to user actions in Chrome, was due to the work of the Google SafeBrowsing anti-phishing system and checking the availability of updates.

Yandex Browser​

Yandex Browser 17.3 has been more active from the very beginning. On the first start, it establishes dozens of connections.

Yandex.Browser and forty connections

Interestingly, many of them lead not to Yandex sites, but to servers of other companies. Mail.ru, VKontakte and even Google. Apparently, this is due to different partnership agreements, under which Yandex Browser provides alternative search options and advertises third-party resources on the quick access panel in each new tab.

Yandex Browser connects to a dozen subnets already at start

Please note that some of the traffic comes on behalf of a system process with a zero PID. The addresses of the remote nodes to which this process establishes a connection coincide with those to which Yandex Browser simultaneously connects.

Yandex.Browser connections in TCPView

The most detailed information "Yandex Browser" sends to api.browser.yandex.ru. They describe the configuration of the computer, browser and all its components, including the state of the password manager and the number of saved bookmarks. Separate lines indicated the result of detecting other browsers and their status (which is running in parallel and which is selected by default). The total amount of this data in our case was 86 KB in plain text format. This is despite the fact that the browser was just installed and did not contain any traces of user activity. Our video card was indicated in this log as VirtualBox Graphics Adapter - theoretically, this allows Yandex Browser to easily determine that it is running in a virtual environment.

Detailed statistics "Yandex Browser" (fragment of the list)

In the intercepted traffic, there are amusing lines like morda-logo or X-Powered by: Cocaine - you can't deny developers a sense of humor. In addition to the OS version and other technical information, Yandex Browser determines the physical location of the device on which it is running. And he does it implicitly - via HTTPS and through the explorer process. Longitude and latitude are calculated using the Wi2Geo geolocation service. In addition to the coordinates themselves, the wi2geo.mobile.yandex.net error in their determination is always calculated through the server. Naturally, we changed the real address, but it will be funny if someone tries to find an editorial yacht in the Arabian Sea.

Geolocation by IP in Yandex Browser

Edge​

In our tests, we used Microsoft Edge 38.14, preinstalled in Windows 10 build 1607. This browser is interesting because it is almost always active. Even if you don't start it, it appears in memory and establishes connections to Microsoft servers. An MSN bot is predominantly running in the background, and when you launch Edge, connections to seven of Microsoft's major networks are momentarily visible.

Edge connections at startup

These are the networks 40.74.0.0–40.125.127.255, 68.232.32.0–68.232.47.255, 93.184.220.0–93.184.223.255, 104.40.0.0–104.47.255.255, 104.244.40.0–104.244.47.255, 111.221.29.0/24 and 207.46. 0.0 / 16. Their numbering capacity is enormous. Judging by the balance of incoming and outgoing traffic, they serve not only for content delivery, but also for large-scale data collection.

Edge connectivity with Microsoft and partner networks

Oddly enough, there was no obvious suspicious browser activity while running Edge. The maximum that indirectly identified the user is the stingy telemetry lines, User-Agent and cookies.

Basic telemetry in Edge

When configuring to launch Edge from a blank page, traffic was generally minimal. The only thing that slightly alarmed was the line containing the records DefaultLocation= and MUID=. The values sent to the server in it msn.comare encoded.

From past research, we have a strong sense that Edge's humble behavior is just an illusion. It is part of Windows 10, and Microsoft in this OS (and now in others too) has implemented many ways to collect detailed information about the user and his network activity. As we discussed in the second part of the article on Windows 10 "spy" habits, it is not necessary to send this information directly through the browser.

Opera​

Already during the installation of the Opera 43.0 browser, traffic goes not only between the computer and the opera.com website.

Connections during Opera installation

Requests are also sent to BitGravity and EdgeCast servers, but they contain exclusively anonymized identifiers, browser version and OS.

Opera connect to different networks at startup

At each start, Opera 43.0 showed a page with advertisements of different brands - from IT (Google, Yandex, Rambler) to completely pop. What to do! This is the modern monetization scheme. When we looked at the intercepted traffic in Wireshark, we saw this line of comment from Opera's partner, Booking.com hotel booking service: “x-content-Type-options: nosniff. Do you know that you might be paid to tinker with our code? We are hiring designers and developers to work in Amsterdam. "The offer is tempting, but perhaps it's better to relax in Amsterdam than to work. Otherwise, the code will be too funny.

Don't pick the code for free!

In addition to servers in the opera.com domain, the browser of the same name often connects to the nodes of the Dutch network WIKIMEDIA-EU-NET (91.198.174.0/24). The intercepted traffic to these servers contained only security certificate (SSL) verification packages, and all "personal data" was limited to a mean line User-Agent: … OPR/36.0.2130.80. Strange, since the version of Opera was 43.0.

Data compression by the Opera Turbo service is performed through a system process with a zero PID, and the traffic goes to the servers opera-mini.net.

Opera Turbo function

During our test, the Opera browser behaved modestly. In the default settings, it downloaded a lot of adware at startup, but soon these left connections were closing. Opera did not divulge any intimate details.

Firefox​

Firefox developer - The Mozilla Foundation makes heavy use of Amazon's cloud web services. This can be seen from the multitude of connections to servers compute.amazonaws.comthat appear immediately at the start of the browser.

Firefox browser automatic connections

They occur every time Firefox 51.0 is started, even if it's just installed. Besides Amazon, traffic goes to Akamai, Cloudflare, EdgeCast and Google subnets. This is necessary in order to balance the load when downloading updates to the browser itself and its add-ons, as well as to ensure the ability to quickly send search queries. In addition, by default, a new browser tab displays links to other projects in the Mozilla community, for which images are also downloaded from the web.

Basic statistics about Firefox performance are sent to telemetry.mozilla.org. It looks scanty and rather harmless.

The physical location of the device running the Firefox browser is determined through the open system Mozilla Location Service, but only if the user has allowed it in the settings: "Menu → Tools → Page Information → Permissions → Know your location."

No matter how hard we tried to find at least some suspicious Firefox activity, it was not found. All traffic was completely within the scope of the user agreement.

Encryption ≠ encryption​

The encryption procedure is fundamentally different from encryption. If encryption transforms data and makes it unreadable without knowing the key and decryption algorithm, then encryption serves to shorten the records and standardize them. This is achieved by using both code pages and some kind of conventionally written method. There are not so many encodings and formats. Finding the right ones is a matter of technology. However, without knowledge of the conventions, the encoded notation becomes difficult to understand in any format.

Encoded browser information

In the case of browsers, decrypted traffic is often additionally encoded. Some of the variables used in it have obvious meaning. For example, the entry s:1440x900x24 reports on the set screen resolution and color depth (8 bits for each RGB channel). Other variables are more or less easy to guess from the context. For example, we can assume that _ym_uid=1488623579201112390 is the user ID in the Yandex.Metrica system. However, there are many other meanings, the meaning of which is not so obvious. For example, a record fpr:335919976901 or rqnl:1:st1488642088 is also some kind of data. In their raw form, their meaning may be incomprehensible even to the specialists of the developer company. Simply because they are usually analyzed by an automatic system that turns such gibberish into some kind of visual information.

Draw the curtains​

It is quite easy to prevent Chrome, Firefox and Opera from sending most of the statistics. All you need to do is uncheck the Send Usage Information box or something similar. Usually, the corresponding item is in the installation wizard and in the browser settings in the "Privacy" section.

You can slightly increase privacy already at the stage of installing the browser.
In the same place, you can check the items "Send sites a tracking ban", "Ask permission to send my geodata" and uncheck the "Automatically send information about possible problems" checkbox.

The wording of these points varies slightly from browser to browser, but the gist is the same. Disable Tracking means that the browser will add a header to outgoing traffic do_not_track. How to handle it is entirely on the conscience of the owner of a particular site.

Please do not follow sites.
Sending geodata on demand means that sites will not be able to automatically determine your current location. As always, this slightly increases safety at the expense of convenience.

By "information about problems" in a browser, we mean a sudden termination of its work or the inability to connect to its own site (for example, to check for updates). In this case, extended information about the problem that has occurred can be sent to the developer. If you do not have any "hacker" extensions and settings, you can help the developers to make the browser better and more stable.

Conclusions​

During the research, we tested popular browsers, intercepted and analyzed the traffic they automatically generated. The conclusions here can be drawn very carefully. Some of the data sent is encrypted and additionally encrypted, so their purpose remains unknown. In addition, the situation can change at any time: a new version of the browser will be released, one company will absorb another, and the privacy policy will change, which most users are too lazy to read.

The browsers we tested do send their developers and partners the data they collect while working on any device - be it a computer or a smartphone. However, there is no data among these that could be called personal and sensitive to disclosure. In most cases, this is just a set of technical information, and rather laconic.

For example, the screen resolution is specified in outgoing packets, but the monitor type is not specified. The general architecture of the processor is identified, but neither the specific model nor its serial number is recorded. The number of open tabs is calculated, but their addresses are not transmitted. The same with passwords: the telemetry information sent by the browser does not contain the saved passwords themselves - only their total number within the framework of the password manager settings synchronization.

Prior to authorization in any network service, the user can be remotely identified while surfing the web only indirectly. However, it should be understood that even general technical details form unique combinations. It is unlikely that you will be able to find many people with the same version of OS, browser, date and time of its installation, set of installed plugins and extensions, number of bookmarks, monitor resolution, processor type, amount of RAM and fifty other small signs. This digital fingerprint does not reveal the secret of the identity, but it allows you to distinguish one user from others quite reliably.

Initially, browsers and websites assign impersonal identifiers to users. They look like alphanumeric strings. They are needed to bring together all the statistics of work and not to mix data from different users. What are their names and what they like - browser developers don't care. This is of interest to the marketing departments of large companies that are actively promoting various social chips. Search engines, social networks, online games, dating and job search sites are the main hunters for personal data, but that's another story altogether.

(c) xakep.ru