A useful hacker search engine!
Salute to everyone, dear friends!
Today we will tell you about a solid hacker search engine CENSYS.
Probably everyone has heard about the SHODAN search engine and search for various network devices with its help. As you know, an interesting business niche is never alone for a long time, and not so long ago the Censys search engine appeared, with the help of which you can also find on the network a lot of everything that is hidden from the masses. Today we will introduce you to her work.
Content
1. FIRST FINDINGS
2. EXAMPLES OF SEARCH
3. CENSYS VS SHODAN
4. NOT INSTEAD, BUT TOGETHER
The Censys search engine began its history at the University of Michigan as a network utility Zakir Durumerik made from ZMap to collect statistics on the prevalence of known vulnerabilities on the Web. Initially, Censys was a little-known fork of Scans.io, a repository for polling network nodes with an IPv4 address.
In the past year, Censys technologies have developed: they screwed the Google search engine to the database, wrote additional tools and used powerful servers dedicated to academic research.
However, even in its current form, Censys is only part of a larger plan. If Google itself indexes mainly web pages and files, then the authors of Censys set the task of creating and maintaining a "base of everything on the Internet." They seem to be doing quite well. We found absolutely all devices - from ATMs to automated process control systems of power plants.
FIRST FINDINGS
Censys is already showing significant benefits. The search engine helped to assess the prevalence of the FREAK and Heartbleed vulnerabilities , helped uncover the fraudulent use of encryption keys by manufacturers of embedded systems and find invalid X.509 certificates , and on the sites of the first million most visited.
Using Censys, it quickly became clear that Dell Inspiron 14 laptops were sold with the same encryption key pre-installed and two certificates, including the eDellRoot root. One of these compromised certificates was used for HTTPS authorization and SCADA control at a water treatment plant in Kentucky.
Initially, Censys kept on three whales. The first was ZMap, an open source network scanner. The second was ZGrab, an application-level scanner needed to identify active services running on a remote host. The third is ZDb, a constantly updated database of scan results. Recently, another one has been added to them - ZTag. It is a tool for tagging devices based on their specific network responses. Thanks to ZTag, the current spread of a particular vulnerability can be identified simply by typing its name. For example, a request for heartbleed found 229,138 nodes in the morning, and 229,134 nodes by lunchtime of the same day. You can see in almost real time how they are slowly patched.
Tagged search by vulnerability name
The drilldown provides details for each node found. Here you can see that the version of encryption in HTTPS that is vulnerable to Heartbleed is being used.
Detailing Vulnerable Hosts
EXAMPLES OF SEARCH
Censys supports full text search, boolean operators, conditionals, and filters. In general, the search word and optional pointers of where it should appear are specified. To filter the issue, you can list port, protocol, method, IP address range, geographic location, or date constraints. The detailed syntax is provided in the help and manual on the censys.io site.
For example, a SCADA query will show all found ICS controlled via the Internet. Now there are nearly 46 thousand of them around the world. Among them are controllers with access via the insecure HTTP protocol. For example, I found a steam boiler in Australia.
Steam boiler with HTTP control
SCADA is similarly controlled at a nuclear power plant in Kansas. There was such a state when I was writing this article.
Write 80.http.get.headers.www_authenticate: netcam and you will get a list of about three thousand web-based network cameras. It is easy to guess that 80 is the open eightieth port,
http is the protocol corresponding to it,
get is the method of receiving data,
header is the header, and
www_authenticate: netcam corresponds to the representation of the device as a network camera.
Addresses and access to network cameras
Enter metadata.manufacturer: "Cisco" and you will see all active equipment manufactured by Cisco and currently working.
Surely among it there will be a lot of unpatched routers with known vulnerabilities.
Active network equipment Cisco
Typical error messages are also useful. Type certificate has expired and you will get a list of everyone using expired certificates. Additionally, specifying the range of a specific subnet, you can perform its express audit.
List of expired certificates
Would you like to know how the internet has changed while you were celebrating the New Year? Request * [2019-12-31 TO 2020-01-01] will show all new and updated network nodes added to the Censys database during this period.
Wondering how many watchers are on one of the Microsoft subnets? Set the range of its IP-addresses: ip: [137.116.81.1 TO 137.117.235.255]. At the time of this writing, there were 22,848 active IPs in it.
Witness hosts on one of the Microsoft subnets
Conveniently, Censys automatically handles DNS records. Enter mx: gmail. com and get a list from google mail servers. DNS A records are also immediately translated to IP addresses. For example, a: facebook.com will return the host description of the main server with IP 173.252.120.68.
CENSYS VS. SHODAN
Censys was created by the legendary HD Moore, who enjoys a special subscription to expanded access to Shodan. These search engines have a similar purpose, but different data collection methods and usage policies. Without registration, they only allow you to briefly familiarize yourself with the found. Despite its formal open status, Censys has more restrictions on unregistered users. If Shodan without authorization limits the depth of search results, then Censys cuts off access to entire sections and almost all additional functions. In addition, when guest logs in, Censys only handles five requests per day from a single IP address, and deleting the cookie does not help.
If you ask "cloud crawlers" to tell about themselves in the same way as they tell about other sites, you will receive an error message. However, Shodan willingly gives out all known information about Censys, and vice versa.
For using filters in Shodan and shifting search results limits from 20 to 10,000 in the account, "credits" are deducted, which can be purchased for money. Censys is a free project with a different constraint model. To remove them, you will not only need to register, but also send a letter to the developers. In it, you need to try to convince them of the ethics of their research (a CEH certificate is great) and the responsible use of the data obtained. For example, by submitting a certificate of their publications, educational institution or company. It is advisable to describe the planned research and make a commitment to include a link to Censys in a scientific article or analytical report.
Shodan always produces standardized search results. Censys has the ability to get the data you are looking for in raw and JSON format. To process complex queries and deep filtering in Censys, you can use the SQL Query Engine, as well as access to Google BigQuery through the API. These tools are of course only available to trusted users. Censys allows you not only to view search results, but also generate reports with a built-in tool.
The authors of Censys declare that they are updating the scan results of the entire IPv4 address range on a daily basis. This is 3.7 billion network nodes, taking into account the losses for the ranges reserved in RFC6890. For some reason, checking the total number of records in Censys itself only yields 189.5 million. It displays exactly how many IPv4 nodes if you enter an asterisk as a universal mask in the request.
NOT INSTEAD, BUT TOGETHER
In their publication, the authors argue that Censys, unlike Shodan, always produces a fresh set of search results. However, I see no practical point in opposing search engines to each other. Shodan scans the Internet much more thoroughly - across all ports and protocols, respecting secure timeouts.
In my opinion, now the Shodan results are cleaner, and there is more valuable information in each result. Therefore, for the latest statistics for global security research, it makes sense to turn to Censys with its tagged search, and for your own practical research - mainly to Shodan.
Salute to everyone, dear friends!
Today we will tell you about a solid hacker search engine CENSYS.
Probably everyone has heard about the SHODAN search engine and search for various network devices with its help. As you know, an interesting business niche is never alone for a long time, and not so long ago the Censys search engine appeared, with the help of which you can also find on the network a lot of everything that is hidden from the masses. Today we will introduce you to her work.
Content
1. FIRST FINDINGS
2. EXAMPLES OF SEARCH
3. CENSYS VS SHODAN
4. NOT INSTEAD, BUT TOGETHER
The Censys search engine began its history at the University of Michigan as a network utility Zakir Durumerik made from ZMap to collect statistics on the prevalence of known vulnerabilities on the Web. Initially, Censys was a little-known fork of Scans.io, a repository for polling network nodes with an IPv4 address.
In the past year, Censys technologies have developed: they screwed the Google search engine to the database, wrote additional tools and used powerful servers dedicated to academic research.
However, even in its current form, Censys is only part of a larger plan. If Google itself indexes mainly web pages and files, then the authors of Censys set the task of creating and maintaining a "base of everything on the Internet." They seem to be doing quite well. We found absolutely all devices - from ATMs to automated process control systems of power plants.
FIRST FINDINGS
Censys is already showing significant benefits. The search engine helped to assess the prevalence of the FREAK and Heartbleed vulnerabilities , helped uncover the fraudulent use of encryption keys by manufacturers of embedded systems and find invalid X.509 certificates , and on the sites of the first million most visited.
Using Censys, it quickly became clear that Dell Inspiron 14 laptops were sold with the same encryption key pre-installed and two certificates, including the eDellRoot root. One of these compromised certificates was used for HTTPS authorization and SCADA control at a water treatment plant in Kentucky.
Initially, Censys kept on three whales. The first was ZMap, an open source network scanner. The second was ZGrab, an application-level scanner needed to identify active services running on a remote host. The third is ZDb, a constantly updated database of scan results. Recently, another one has been added to them - ZTag. It is a tool for tagging devices based on their specific network responses. Thanks to ZTag, the current spread of a particular vulnerability can be identified simply by typing its name. For example, a request for heartbleed found 229,138 nodes in the morning, and 229,134 nodes by lunchtime of the same day. You can see in almost real time how they are slowly patched.
Tagged search by vulnerability name
The drilldown provides details for each node found. Here you can see that the version of encryption in HTTPS that is vulnerable to Heartbleed is being used.
Detailing Vulnerable Hosts
EXAMPLES OF SEARCH
Censys supports full text search, boolean operators, conditionals, and filters. In general, the search word and optional pointers of where it should appear are specified. To filter the issue, you can list port, protocol, method, IP address range, geographic location, or date constraints. The detailed syntax is provided in the help and manual on the censys.io site.
For example, a SCADA query will show all found ICS controlled via the Internet. Now there are nearly 46 thousand of them around the world. Among them are controllers with access via the insecure HTTP protocol. For example, I found a steam boiler in Australia.
Steam boiler with HTTP control
SCADA is similarly controlled at a nuclear power plant in Kansas. There was such a state when I was writing this article.
Write 80.http.get.headers.www_authenticate: netcam and you will get a list of about three thousand web-based network cameras. It is easy to guess that 80 is the open eightieth port,
http is the protocol corresponding to it,
get is the method of receiving data,
header is the header, and
www_authenticate: netcam corresponds to the representation of the device as a network camera.
Addresses and access to network cameras
Enter metadata.manufacturer: "Cisco" and you will see all active equipment manufactured by Cisco and currently working.
Surely among it there will be a lot of unpatched routers with known vulnerabilities.
Active network equipment Cisco
Typical error messages are also useful. Type certificate has expired and you will get a list of everyone using expired certificates. Additionally, specifying the range of a specific subnet, you can perform its express audit.
List of expired certificates
Would you like to know how the internet has changed while you were celebrating the New Year? Request * [2019-12-31 TO 2020-01-01] will show all new and updated network nodes added to the Censys database during this period.
Wondering how many watchers are on one of the Microsoft subnets? Set the range of its IP-addresses: ip: [137.116.81.1 TO 137.117.235.255]. At the time of this writing, there were 22,848 active IPs in it.
Witness hosts on one of the Microsoft subnets
Conveniently, Censys automatically handles DNS records. Enter mx: gmail. com and get a list from google mail servers. DNS A records are also immediately translated to IP addresses. For example, a: facebook.com will return the host description of the main server with IP 173.252.120.68.
CENSYS VS. SHODAN
Censys was created by the legendary HD Moore, who enjoys a special subscription to expanded access to Shodan. These search engines have a similar purpose, but different data collection methods and usage policies. Without registration, they only allow you to briefly familiarize yourself with the found. Despite its formal open status, Censys has more restrictions on unregistered users. If Shodan without authorization limits the depth of search results, then Censys cuts off access to entire sections and almost all additional functions. In addition, when guest logs in, Censys only handles five requests per day from a single IP address, and deleting the cookie does not help.
If you ask "cloud crawlers" to tell about themselves in the same way as they tell about other sites, you will receive an error message. However, Shodan willingly gives out all known information about Censys, and vice versa.
For using filters in Shodan and shifting search results limits from 20 to 10,000 in the account, "credits" are deducted, which can be purchased for money. Censys is a free project with a different constraint model. To remove them, you will not only need to register, but also send a letter to the developers. In it, you need to try to convince them of the ethics of their research (a CEH certificate is great) and the responsible use of the data obtained. For example, by submitting a certificate of their publications, educational institution or company. It is advisable to describe the planned research and make a commitment to include a link to Censys in a scientific article or analytical report.
Shodan always produces standardized search results. Censys has the ability to get the data you are looking for in raw and JSON format. To process complex queries and deep filtering in Censys, you can use the SQL Query Engine, as well as access to Google BigQuery through the API. These tools are of course only available to trusted users. Censys allows you not only to view search results, but also generate reports with a built-in tool.
The authors of Censys declare that they are updating the scan results of the entire IPv4 address range on a daily basis. This is 3.7 billion network nodes, taking into account the losses for the ranges reserved in RFC6890. For some reason, checking the total number of records in Censys itself only yields 189.5 million. It displays exactly how many IPv4 nodes if you enter an asterisk as a universal mask in the request.
NOT INSTEAD, BUT TOGETHER
In their publication, the authors argue that Censys, unlike Shodan, always produces a fresh set of search results. However, I see no practical point in opposing search engines to each other. Shodan scans the Internet much more thoroughly - across all ports and protocols, respecting secure timeouts.
In my opinion, now the Shodan results are cleaner, and there is more valuable information in each result. Therefore, for the latest statistics for global security research, it makes sense to turn to Censys with its tagged search, and for your own practical research - mainly to Shodan.
