Mutt
Professional
- Messages
- 1,459
- Reaction score
- 1,105
- Points
- 113
Good day, this article will focus on one of the ways to hide data from an Internet provider
Net neutrality
If you are not familiar with the term "Net neutrality", I will try to quickly bring you up to date.
Not so long ago, America passed a law that annuls net neutrality. That is, now the provider can legally collect and analyze information about the user's activities on the network without obtaining consent. In practice, this means that IP addresses will be collected, which can be easily converted to physical location, site history and content of pages viewed outside the HTTPS protocol, including information submitted via web forms. The Electronic Frontier Foundation (EFF) believes that ISPs can go even further and start selling the collected information to third parties.
Hiding data from the provider with Noisy
To hide data from the provider, the Noisy utility will help us, which is a simple script written in Python. It can generate random HTTP / DNS traffic in the background while you visit websites. Thus, your information loses its uniqueness, becomes useless for marketers and companies involved in the analysis of this kind of information.
The ideas behind the Noisy script are nothing new and have been taken from the Squawk tool, which is a script embedded in web pages to send additional random requests. However, Noisy is easier to use and modify.
Let's take a step-by-step look at how to work with Noisy.
Installing dependencies
You must have Python installed. Kali Linux has everything you need by default. Noisy is compatible with versions 2.7 and 3.6. To check the Python version on your system, enter the following command:
Noisy uses the Requests module, which can be installed using the following command:
In addition, you can get even more benefit from Noisy if you install Docker, as this way you can isolate traffic in a limited environment as a separate channel. However, using Docker is optional.
Cloning Noisy from GitHub
After Python and all the required dependencies are installed, download Noisy from the repository using the git command:
Alternative option: go to the repository and download the .zip file. First, click on the "Clone or download" button and in the context menu that opens, select the "Download ZIP" item. After downloading, unpack the contents of the archive.
Familiarity with the configuration file
Now you need to edit the configuration file. Despite the fact that you do not need to change the settings for Noisy to work correctly, along the way we will be able to figure out how the program works.
First, go to the directory with the unpacked archive using the cd command:
Then open the config.json file in your favorite text editor. Let's use Vim as an example:
Alternatively, go to the "Files" folder on the desktop, and then to the "noisy" directory and open the config.json file in the Notepad ++ editor.
The information in the file is stored in JSON format. Noisy takes the first URL from the "root_urls" list and waits for a random amount of time between "min_sleep" and "max_sleep" (in seconds) before clicking on the random link on the page. The viewing depth is specified in the "max_depth" parameter. After completing page views on the current site, the next address from the list is taken, and so on.
As you may have noticed, there are some specifics related to the settings. First, there are only 11 addresses in the "root_urls" list. Secondly, when visiting some of the sites in the above list, problems may arise.
When visiting sites, the content is not displayed on the screen, but it can attract the attention of someone who monitors the local network or if the viewing of these sites is prohibited by the proxy server settings. In short, edit this list as you see fit. The more addresses are added, the more random the traffic will be.
Since there are few addresses in the config.json file, the entire list will be traversed in not very long time. As a result, there could potentially be a recognizable piece of information that can be easily filtered out.
Adding sites to the config file
If you decide to make your own list of sites, remember that the information is stored in JSON format and each URL must be enclosed in double quotation marks with a comma at the end. Also, I always add the https: // prefix to try to access the site over HTTPS first. Example:
Traffic randomization
After updating the config.json file, you're ready to randomize your traffic. Using the cd command, go to the noisy directory and run the noisy.py script with an argument in the form of a configuration file:
After running this command, random traffic will be generated. If everything is working correctly, you should see an outgoing stream of different addresses as in the example below:
To stop the script, press the key combination Ctrl-C.
Other options in Noisy
When Noisy is not running, use the h or help argument to familiarize yourself with other options, such as setting a timeout.
Full list of arguments:
Optional arguments:
Running Noisy via Docker (optional)
As mentioned above, you can put Noisy in a Docker-based container using the following command (but you need to install Docker first):
You can even build a Raspberry Pi device running a Raspbian operating system if you want to move the traffic randomizer to a separate location.
After the build is complete, launch is carried out using the following command:
Increasing random traffic volumes (optional)
By examining the configuration file, you may have noticed that the delay is set in seconds. This fact means that the smallest delay can be equal to one second, if you configure the configuration like this:
In some cases, this latency may not be enough, such as in situations where there are many people on your network and you need to generate large amounts of random traffic.
Using Docker-compose, we can run multiple containers at the same time to generate unlimited traffic. You need to go to the examples folder, build a new image and run the build on the number of containers that you need:
However, in this case, you need to track the amount of traffic generated, since with a large number of working containers, the Internet speed can slow down. Also, if you have traffic restrictions, you need to be very careful about how much random data is generated.
As you can see, Noisy is an extremely simple utility, and after some tweaking, you have an incredibly powerful tool that should be your everyday tool if you care about your privacy.
Net neutrality
If you are not familiar with the term "Net neutrality", I will try to quickly bring you up to date.
Not so long ago, America passed a law that annuls net neutrality. That is, now the provider can legally collect and analyze information about the user's activities on the network without obtaining consent. In practice, this means that IP addresses will be collected, which can be easily converted to physical location, site history and content of pages viewed outside the HTTPS protocol, including information submitted via web forms. The Electronic Frontier Foundation (EFF) believes that ISPs can go even further and start selling the collected information to third parties.
Hiding data from the provider with Noisy
To hide data from the provider, the Noisy utility will help us, which is a simple script written in Python. It can generate random HTTP / DNS traffic in the background while you visit websites. Thus, your information loses its uniqueness, becomes useless for marketers and companies involved in the analysis of this kind of information.
The ideas behind the Noisy script are nothing new and have been taken from the Squawk tool, which is a script embedded in web pages to send additional random requests. However, Noisy is easier to use and modify.
Let's take a step-by-step look at how to work with Noisy.
Installing dependencies
You must have Python installed. Kali Linux has everything you need by default. Noisy is compatible with versions 2.7 and 3.6. To check the Python version on your system, enter the following command:
Code:
python -VNoisy uses the Requests module, which can be installed using the following command:
Code:
pip install requestsIn addition, you can get even more benefit from Noisy if you install Docker, as this way you can isolate traffic in a limited environment as a separate channel. However, using Docker is optional.
Cloning Noisy from GitHub
After Python and all the required dependencies are installed, download Noisy from the repository using the git command:
Code:
git clone https://github.com/1tayH/noisy.gitAlternative option: go to the repository and download the .zip file. First, click on the "Clone or download" button and in the context menu that opens, select the "Download ZIP" item. After downloading, unpack the contents of the archive.
Familiarity with the configuration file
Now you need to edit the configuration file. Despite the fact that you do not need to change the settings for Noisy to work correctly, along the way we will be able to figure out how the program works.
First, go to the directory with the unpacked archive using the cd command:
Code:
cd noisyThen open the config.json file in your favorite text editor. Let's use Vim as an example:
Code:
vim config.jsonAlternatively, go to the "Files" folder on the desktop, and then to the "noisy" directory and open the config.json file in the Notepad ++ editor.
The information in the file is stored in JSON format. Noisy takes the first URL from the "root_urls" list and waits for a random amount of time between "min_sleep" and "max_sleep" (in seconds) before clicking on the random link on the page. The viewing depth is specified in the "max_depth" parameter. After completing page views on the current site, the next address from the list is taken, and so on.
Code:
{
"max_depth": 25,
"min_sleep": 3,
"max_sleep": 6,
"timeout": false,
"root_urls": [
"http://4chan.org",
"https://www.reddit.com",
"https://www.yahoo.com",
"http://www.cnn.com",
"https://p---hub.com",
"https://www.ebay.com",
"https://wikipedia.org",
"https://youtube.com",
"https://github.com",
"https://medium.com",
"https://thep-----bay.org",
],
"blacklisted_urls": [
"https://t.co",
"t.umblr.com",
"messenger.com",
"itunes.apple.com",
"l.facebook.com",
"KIDALA",
"mediawiki",
".css",
".ico",
".xml",
"intent / tweet",
"twitter.com/share",
"dialog / feed?",
".json",
"zendesk",
"clickserve",
".png"
],
"user agents": [
"there are many user agents here!"
]
}As you may have noticed, there are some specifics related to the settings. First, there are only 11 addresses in the "root_urls" list. Secondly, when visiting some of the sites in the above list, problems may arise.
When visiting sites, the content is not displayed on the screen, but it can attract the attention of someone who monitors the local network or if the viewing of these sites is prohibited by the proxy server settings. In short, edit this list as you see fit. The more addresses are added, the more random the traffic will be.
Since there are few addresses in the config.json file, the entire list will be traversed in not very long time. As a result, there could potentially be a recognizable piece of information that can be easily filtered out.
Adding sites to the config file
If you decide to make your own list of sites, remember that the information is stored in JSON format and each URL must be enclosed in double quotation marks with a comma at the end. Also, I always add the https: // prefix to try to access the site over HTTPS first. Example:
Code:
"https://website.com",Traffic randomization
After updating the config.json file, you're ready to randomize your traffic. Using the cd command, go to the noisy directory and run the noisy.py script with an argument in the form of a configuration file:
Code:
cd noisy
python noisy.py --config config.jsonAfter running this command, random traffic will be generated. If everything is working correctly, you should see an outgoing stream of different addresses as in the example below:
Code:
INFO: root: Visiting https://azerbaijantourism.az/about
INFO: requests.packages.urllib3.connectionpool: Starting new HTTPS connection (1): azerbaijantourism.az
INFO: root: Visiting https://azerbaijantourism.az/
INFO: requests.packages.urllib3.connectionpool: Starting new HTTPS connection (1): azerbaijantourism.az
INFO: root: Visiting https://azerbaijantourism.az/booking-conditions
INFO: requests.packages.urllib3.connectionpool: Starting new HTTPS connection (1): azerbaijantourism.azTo stop the script, press the key combination Ctrl-C.
Other options in Noisy
When Noisy is not running, use the h or help argument to familiarize yourself with other options, such as setting a timeout.
Code:
python noisy.py –helpFull list of arguments:
Code:
noisy.py [-h] [--log -l] --config -c [--timeout -t]Optional arguments:
- -h, --help help
- -Log -l event logging level
- -Config -c configuration file
- -Timeout -t duration of the spider in seconds
Running Noisy via Docker (optional)
As mentioned above, you can put Noisy in a Docker-based container using the following command (but you need to install Docker first):
Code:
docker build -t noisyYou can even build a Raspberry Pi device running a Raspbian operating system if you want to move the traffic randomizer to a separate location.
Code:
docker build -f Dockerfile.pi -t noisyAfter the build is complete, launch is carried out using the following command:
Code:
docker run -it noisy --config config.jsonIncreasing random traffic volumes (optional)
By examining the configuration file, you may have noticed that the delay is set in seconds. This fact means that the smallest delay can be equal to one second, if you configure the configuration like this:
Code:
"min_sleep": 0,
"max_sleep": 1,In some cases, this latency may not be enough, such as in situations where there are many people on your network and you need to generate large amounts of random traffic.
Using Docker-compose, we can run multiple containers at the same time to generate unlimited traffic. You need to go to the examples folder, build a new image and run the build on the number of containers that you need:
Code:
cd examples / docker-compose
docker-compose build
docker-compose up --scale noisy = <number-of-containers>However, in this case, you need to track the amount of traffic generated, since with a large number of working containers, the Internet speed can slow down. Also, if you have traffic restrictions, you need to be very careful about how much random data is generated.
As you can see, Noisy is an extremely simple utility, and after some tweaking, you have an incredibly powerful tool that should be your everyday tool if you care about your privacy.
