At the moment, there are only a few sites for finding vulnerabilities. With the help of this article, I will need to check their work for flaws, as well as find vulnerabilities under the CVE number. And based on the results of this, we will decide which are the best sites for finding vulnerabilities sites for finding vulnerabilities should be used first.
Content
1. CVE Details
2. NVD search engine
3. Vulners
4. Summary
I usually solve the problem of identifying packages installed on the system manually or using simple scripts. So I will omit that side of the question. Let's just say we already have a list of all the software - product names and versions.
What are the options for finding vulnerabilities for these components? In this case, we will consider only free options - The budget, as always, is limited:
CVE Details
This is a popular site, statistics of which can be seen in many articles and reports. But can it be used as a vulnerability search tool? Sure! And at first I was in awe of the clever side that left less than a hundred of the 1,500 CVEs of the Linux kernel. Only later it turned out that not everything is so simple here.
We try to look for vulnerabilities, use a version search and enter the name of the kernel according to CPE ( )linux:linux_kernel and its version.
The result is No matches. As far as I understand, the site of our version does not know either. Using a step-by-step downgrade method, we were able to determine that the oldest supported version of the branch we are interested in is 4.14.141. We look at the result: there are just over 50 vulnerabilities.
This is much less than the NVD output (470 for version 4.14.141). Let's compare the lists of the vulnerabilities found. I will demonstrate the differences in search results using two examples: CVE-2019-17351 and CVE-2019-17133.
For some reason, vulnerabilities with the same description are handled differently. Web Search NVD is both vulnerabilities, while CVE Details finds only the CVE-2019-17351 . CVE-2019-17133 is not listed in search results, and version 4.14.141 is not listed as a vulnerable version in the description of the CVE information. Moreover, on the NVD page, the vulnerable versions of these two CVEs are displayed equally correctly.
The reason for this behavior of CVE Details can be found on the page describing how the site works. It turns out that the site receives data from an RSS feed about changes in the XML description of vulnerabilities. NVD officially stopped updating XML descriptions on October 9, 2019, and switched exclusively to JSON. But the omission of vulnerabilities is not related to this. Let's take a deeper look and find in the archives of the Internet the file nvdcve-2.0-2019.xml - this is exactly the description of all CVEs of 2019 in XML format. By the way, the pub_date field contains the value 2019-10-15T03: 00: 00, that is, in fact, NIST continued to update XML for some time after the deadline.
So, the meaning of the fields is strikingly fully consistent with what we see on cvedetail.com. - vulnerable-software-list
The fact is that the NVD developers used to experiment and for some records they tried to indicate all vulnerable versions, and for some they simply indicated the version range (the last vulnerable version). CVE Details just doesn't know how to parse these version ranges.
For CVE-2019-17133, the version range is indeed specified initially. If you upload another file - nvdcve-2019.xml (the same description, but using the field format version 1.2, not 2.0), then you will see the following there.
The prev flag simply means that all previous versions are vulnerable, but for some reason this flag was lost when moving from v1.2 to v2.0, and in v2.0 it was no longer possible to distinguish where which revisions were listed and where their volume is there. NIST has updated both versions of the description, but who wants to use version 1.2 when it's 2.0? So it's hard to blame the CVE details here.
The transition of NVD to JSON had a bad effect on CVE Details, the statistics of vulnerabilities on the site are eloquent.
In 2019, the CVE details lacked 5,000 vulnerabilities and there were no vulnerabilities for the period 2020-2021, the site simply stopped updating. But it still has fans, and in 2020 some of my friends continued to use it, trying not to notice all the accumulated shortcomings.
Incidentally, surprisingly, as soon as I finished working on this article, the CVE Details site was updated for the first time in the last year and a half!
Unfortunately, even the existing version support was removed with this update. And now, for example, both CVE-2019-17133 and CVE-2019-17351 look like this.
I'm afraid now it will be completely impossible to use CVE Details for any efficient version search.
Search engine NVD
The NVD website is a good option to search the NVD database. Who knows better how to search the NVD database if not the compiler? But finding vulnerabilities in the Linux kernel, as it turns out, is not so easy.
Let's say we are using the good old 4.14 kernel, do not forget and patch it to the latest version 4.14.230 for today (April 11). We are using the advanced search option (Search type: Advanced). You will not be able to check this version on a search engine. Even version 4.14.200 is unfamiliar to NVD and the last one available is 4.14.194 (21 Aug 2020). NVD honestly reports this in a note.
The fact is that the web interface of the search engine allows you to use only product versions from the CPE (Common Platform Enumeration) dictionary. If we ignore this warning and enter the correct version 4.14.230, then by clicking the search button, we will get more than four thousand vulnerabilities of all versions of the Linux kernel.
The search engine returned 460 results, which is very good for such a large product.
But let's try to skip the search for only known CPEs, because NVD also provides a REST API (see PDF documentation).
We form the following search query:
The results are displayed in the form of a JSON file, by default 20 pieces per page, if you don't want to combine all this, you can add the following line to the search query:
But here you have to count on luck, the NVD server for DDoS protection sometimes does not respond to such greedy requests. For the most part I was lucky and managed to get up to 2000 results per page.
But in the end, we get a strange result - 1315 vulnerabilities.
Search results via REST API
Let's try to find out. We save all pages with the search result. Then we search for all HTML and JSON CVE IDs right in Notepad ++, sort and compare them.
Even if you selectively scan for a vulnerability that does not appear in web search results, you can immediately see its characteristic symptom.
The web version does not display vulnerabilities in which the searched product is marked as Running on / with, that is, it just works together with the vulnerable one. The web version only gives us Linux kernel vulnerabilities, and the REST API also gives us vulnerabilities in all software running on Linux. This is perhaps the advantage of web search. Let's take NVD's web search results as a guide and see what the other two sites show.
Vulners
I think everyone who has been on the topic has heard of this search engine. I myself am close to it because of the openness that its authors advocate. Both separate tools for searching for vulnerabilities and the database itself are openly available (but only for researchers).
Now let's try to study the vulnerability of our experimental kernel versions 4.14.194, leaving only NVD entries. The result is impressive: just one vulnerability!
I will try to reproduce everything the same, but through the Vulners API Python wrapper. Let's take a script for searching by CPE ID as an example .
To get a free key, you need to register on the site, and here is the result, but again a little thin - only 55 vulnerabilities.
I tried to search in a different way.
But this is even worse.
In my opinion, the first of the search options is the most probable, it is also the most correct. Let's compare with the results of the NVD web search engine.
Take CVE-2018-7754 and CVE-2018-7755 for example. The first is in both lists, the second is only in NVD. Let's compare them with each other.
At first glance, there is not much difference, linux_kernel is there and there.
I also compared the descriptions of these two vulnerabilities in JSON, they can be downloaded from the site for the selected vulnerabilities (see example and figure below).
But, frankly, I did not find a significant difference between the descriptions of CVE-2018-7754 and CVE-2018-7755 (ranges of vulnerable versions are present both in the added fields affectedSoftware and in those taken from NVD cpeConfiguration).
Outcomes
We've looked at NVD vulnerability research for distributions where you may be the only maintainer, and where vulnerability research has not yet been rolled out, like Red Hat or Ubuntu.
Based on the comparison results, I have to conclude that at the moment for split distributions it is better to use the standard NVD search, but the web version gives the best results.
And you can define any version, even one that is not in the CPE, right in the URL bar, like this (version number at the end):
But for widely used OS distributions, Vulsner will be the best option.
(c) cryptoworld.su
Content
1. CVE Details
2. NVD search engine
3. Vulners
4. Summary
I usually solve the problem of identifying packages installed on the system manually or using simple scripts. So I will omit that side of the question. Let's just say we already have a list of all the software - product names and versions.
What are the options for finding vulnerabilities for these components? In this case, we will consider only free options - The budget, as always, is limited:
- regular search engine NVD;
- CVE Details - almost more popular than CVE itself;
- Vulners is a specialized vulnerability search engine whose developers claim to continually add new sources and even use AI to determine the severity of a vulnerability.
CVE Details
This is a popular site, statistics of which can be seen in many articles and reports. But can it be used as a vulnerability search tool? Sure! And at first I was in awe of the clever side that left less than a hundred of the 1,500 CVEs of the Linux kernel. Only later it turned out that not everything is so simple here.
We try to look for vulnerabilities, use a version search and enter the name of the kernel according to CPE ( )linux:linux_kernel and its version.
The result is No matches. As far as I understand, the site of our version does not know either. Using a step-by-step downgrade method, we were able to determine that the oldest supported version of the branch we are interested in is 4.14.141. We look at the result: there are just over 50 vulnerabilities.
This is much less than the NVD output (470 for version 4.14.141). Let's compare the lists of the vulnerabilities found. I will demonstrate the differences in search results using two examples: CVE-2019-17351 and CVE-2019-17133.
For some reason, vulnerabilities with the same description are handled differently. Web Search NVD is both vulnerabilities, while CVE Details finds only the CVE-2019-17351 . CVE-2019-17133 is not listed in search results, and version 4.14.141 is not listed as a vulnerable version in the description of the CVE information. Moreover, on the NVD page, the vulnerable versions of these two CVEs are displayed equally correctly.
The reason for this behavior of CVE Details can be found on the page describing how the site works. It turns out that the site receives data from an RSS feed about changes in the XML description of vulnerabilities. NVD officially stopped updating XML descriptions on October 9, 2019, and switched exclusively to JSON. But the omission of vulnerabilities is not related to this. Let's take a deeper look and find in the archives of the Internet the file nvdcve-2.0-2019.xml - this is exactly the description of all CVEs of 2019 in XML format. By the way, the pub_date field contains the value 2019-10-15T03: 00: 00, that is, in fact, NIST continued to update XML for some time after the deadline.
So, the meaning of the fields is strikingly fully consistent with what we see on cvedetail.com. - vulnerable-software-list
The fact is that the NVD developers used to experiment and for some records they tried to indicate all vulnerable versions, and for some they simply indicated the version range (the last vulnerable version). CVE Details just doesn't know how to parse these version ranges.
For CVE-2019-17133, the version range is indeed specified initially. If you upload another file - nvdcve-2019.xml (the same description, but using the field format version 1.2, not 2.0), then you will see the following there.
The prev flag simply means that all previous versions are vulnerable, but for some reason this flag was lost when moving from v1.2 to v2.0, and in v2.0 it was no longer possible to distinguish where which revisions were listed and where their volume is there. NIST has updated both versions of the description, but who wants to use version 1.2 when it's 2.0? So it's hard to blame the CVE details here.
The transition of NVD to JSON had a bad effect on CVE Details, the statistics of vulnerabilities on the site are eloquent.
In 2019, the CVE details lacked 5,000 vulnerabilities and there were no vulnerabilities for the period 2020-2021, the site simply stopped updating. But it still has fans, and in 2020 some of my friends continued to use it, trying not to notice all the accumulated shortcomings.
Incidentally, surprisingly, as soon as I finished working on this article, the CVE Details site was updated for the first time in the last year and a half!
Unfortunately, even the existing version support was removed with this update. And now, for example, both CVE-2019-17133 and CVE-2019-17351 look like this.
I'm afraid now it will be completely impossible to use CVE Details for any efficient version search.
Search engine NVD
The NVD website is a good option to search the NVD database. Who knows better how to search the NVD database if not the compiler? But finding vulnerabilities in the Linux kernel, as it turns out, is not so easy.
Let's say we are using the good old 4.14 kernel, do not forget and patch it to the latest version 4.14.230 for today (April 11). We are using the advanced search option (Search type: Advanced). You will not be able to check this version on a search engine. Even version 4.14.200 is unfamiliar to NVD and the last one available is 4.14.194 (21 Aug 2020). NVD honestly reports this in a note.
The fact is that the web interface of the search engine allows you to use only product versions from the CPE (Common Platform Enumeration) dictionary. If we ignore this warning and enter the correct version 4.14.230, then by clicking the search button, we will get more than four thousand vulnerabilities of all versions of the Linux kernel.
The search engine returned 460 results, which is very good for such a large product.
But let's try to skip the search for only known CPEs, because NVD also provides a REST API (see PDF documentation).
We form the following search query:
Code:
https://services.nvd.nist.gov/rest/json/cves/1.0?cpeMatchString=cpe:2.3:o:linux:linux_kernel:4.14.194.&startIndex=0The results are displayed in the form of a JSON file, by default 20 pieces per page, if you don't want to combine all this, you can add the following line to the search query:
Code:
&resultsPerPage=1400But here you have to count on luck, the NVD server for DDoS protection sometimes does not respond to such greedy requests. For the most part I was lucky and managed to get up to 2000 results per page.
But in the end, we get a strange result - 1315 vulnerabilities.
Search results via REST API
Let's try to find out. We save all pages with the search result. Then we search for all HTML and JSON CVE IDs right in Notepad ++, sort and compare them.
Even if you selectively scan for a vulnerability that does not appear in web search results, you can immediately see its characteristic symptom.
The web version does not display vulnerabilities in which the searched product is marked as Running on / with, that is, it just works together with the vulnerable one. The web version only gives us Linux kernel vulnerabilities, and the REST API also gives us vulnerabilities in all software running on Linux. This is perhaps the advantage of web search. Let's take NVD's web search results as a guide and see what the other two sites show.
Vulners
I think everyone who has been on the topic has heard of this search engine. I myself am close to it because of the openness that its authors advocate. Both separate tools for searching for vulnerabilities and the database itself are openly available (but only for researchers).
Now let's try to study the vulnerability of our experimental kernel versions 4.14.194, leaving only NVD entries. The result is impressive: just one vulnerability!
I will try to reproduce everything the same, but through the Vulners API Python wrapper. Let's take a script for searching by CPE ID as an example .
Code:
import vulners
import json
vulners_api = vulners.Vulners (api_key = "There should be a_key here")
cpe_results = vulners_api.cpeVulnerabilities("cpe:/o:linux:linux_kernel:4.14.194")
cpe_NVD_list = cpe_results.get('NVD')
print("cpe:/o:linux:linux_kernel:4.14.194")
print(json.dumps(cpe_NVD_list, indent=2, sort_keys=False))To get a free key, you need to register on the site, and here is the result, but again a little thin - only 55 vulnerabilities.
I tried to search in a different way.
But this is even worse.
In my opinion, the first of the search options is the most probable, it is also the most correct. Let's compare with the results of the NVD web search engine.
Take CVE-2018-7754 and CVE-2018-7755 for example. The first is in both lists, the second is only in NVD. Let's compare them with each other.
At first glance, there is not much difference, linux_kernel is there and there.
I also compared the descriptions of these two vulnerabilities in JSON, they can be downloaded from the site for the selected vulnerabilities (see example and figure below).
But, frankly, I did not find a significant difference between the descriptions of CVE-2018-7754 and CVE-2018-7755 (ranges of vulnerable versions are present both in the added fields affectedSoftware and in those taken from NVD cpeConfiguration).
Outcomes
We've looked at NVD vulnerability research for distributions where you may be the only maintainer, and where vulnerability research has not yet been rolled out, like Red Hat or Ubuntu.
Based on the comparison results, I have to conclude that at the moment for split distributions it is better to use the standard NVD search, but the web version gives the best results.
And you can define any version, even one that is not in the CPE, right in the URL bar, like this (version number at the end):
Code:
https://nvd.nist.gov/vuln/search/results?form_type=Advanced&results_type=overview&search_type=all&cpe_vendor=cpe%3A%2F%3Alinux&cpe_product=cpe%3A%2F%3Alinux%3Alinux_kernel&cpe_version=cpe%3A%2F%3Alinux%3Alinux_kernel%3A4.14.999But for widely used OS distributions, Vulsner will be the best option.
(c) cryptoworld.su
