Today, modern technological approaches to solving cybersecurity problems lead to the creation of software or hardware-software solutions that rely on high-speed algorithms and automated control of information flows. One of the classic representatives of promising and innovative means of ensuring information security are hardware-software platforms Next-Generation Firewalls (NGFW).
Since the number of different attack scenarios on computer networks has increased significantly in recent years – mostly due to a significant increase in the activity of hacker groups and their unification into large units – the load on the first-line hardware and software platforms – NGFW – has also increased. Despite a number of advantages and capabilities, modern NGFWs require high stability and speed of their own software for their successful operation. The key task in the development of such tools was to reduce restrictions and optimize the functionality of NGFW subsystems.
When analyzing the functionality of NGFW platforms already presented or just entering the market, problems and aspects were identified that NGFW does not solve or solves partially. Here are some of them:
- Complete protection against zero-day vulnerabilities - although NGFW provides protection against some zero-day vulnerabilities, it cannot provide complete protection against all new and unknown threats, anomalies that are not in signature databases
- Detection and prevention of advanced targeted attacks (APT) - Advanced persistent threats (APT) can bypass NGFW protections because they often use social engineering, phishing and other sophisticated techniques that are difficult to detect at the network level, it should be noted that hackers are increasingly joining forces in large groups to implement attack scenarios of this kind
- Automatic control of any services - practice has shown that both company system administrators and attackers often use non-standard ports for network services, therefore, an intelligent system is required that can identify and automatically control connections to any services.
The results of this analysis lead to the conclusion about the need for a completely new - intelligent subsystem as part of the NGFW, which would use advanced analysis methods, including machine learning - heterogeneous artificial intelligence specialized for cybersecurity tasks, allowing to identify and respond to distributed and complex attacks, detect anomalies and suspicious activity, classify new types of traffic.
The Next-generation Intrusion Detection System (NGIDS) can serve as such a subsystem, complementing the functionality of NGFW, based on modern specialized AI tools. The development of this subsystem in NGFW will significantly increase the speed, accuracy and efficiency of detecting hacker attacks, targeted attacks and anomalous impacts.
A study and detailed analysis of classic modern Intrusion Detection Systems (IDS) showed that standard solutions require constant access to the signature database, have a low response rate to incidents, and are integrated into a small number of hardware platform architectures.
This article discusses the results of research on the integration of algorithms for intelligent identification and classification of a wide range of hacker attacks into NGFW systems and NGIDS subsystems.
As a result of studying the applicability of classical common AI models to this task, it turned out that they do not solve the problem of classifying network traffic, have a tendency to overfitting, and work very slowly. For example, the support vector machine (SVM) and the adaptive boosting model over the decision tree (AdaBoost) did not provide the necessary convergence during training; algorithms such as the k-nearest neighbors (KNN) and random forest (RF), on the contrary, were prone to overfitting on the test sample and cannot be used by themselves to classify arbitrary traffic.
To solve the problems of accurate classification of network traffic and combat overtraining of AI models, it was necessary to create our own high-performance system for collecting network traffic statistics and its subsequent classification.
For these purposes, a low-level mechanism for intercepting network traffic was developed and implemented using WFP libraries for Windows and AF_PACKET sockets for Linux. The solution allowed avoiding copying unnecessary data and capturing only the necessary data blocks for analysis.
The collected network traffic statistics were then fed to a multi-layer specially prepared AI for traffic classification. At the first layer, errors were reduced and a nodal data structure was formed using algorithms such as Gini importance and Mean Decrease Accuracy (MDA) based on the Random Forest classifier, as well as the MANOVA multivariate variance analysis algorithm to form data for abnormal zones.
The second AI layer performed an analysis using a modified Kohonen self-organizing map (SOM) algorithm, which effectively groups statistical data in a multidimensional field, simplifying further evaluation. In their canonical form, maps are unable to assign input data to a specific class, so the learning process was modified with a map coloring algorithm, allowing each network neuron to be assigned to a specific data class.
To increase the accuracy of traffic classification by self-organizing maps, an algorithm for optimizing weights during training was implemented, which allows for updating the weight of the winning neuron to take into account not only the Euclidean distance to the input vector from the test sample, but also the normalized deviation from the weights of the remaining neurons in the area determined by the exponential neighborhood function. This method made it possible to avoid first-order discontinuities in the multidimensional map, which, along with the optimizing gradient descent method to the boundaries of areas, made it possible to create the densest intensity map used to assign class labels to individual neurons.
The third layer is the final one in evaluating the input data and the data obtained as a result of the previous layer. The approaches used are based on the feedforward neural network (FNN) for classifying statistical data and the network with recurrent connections (RNN) for evaluating the statistics of the network connection time series.
To process statistical data, a multilayer perceptron with ultra-precise layers was developed, allowing to identify the correlation of input data with the results of SOM evaluation. To reduce the effect of overtraining, dropout layers were introduced, complicating the formation of strong connections between network neurons. Also, to normalize the results of the network and provide a weighted assessment of the attitude to the classes of attacks, a Softmax output layer was introduced. For parallel evaluation of time series, a model based on LSTM cells is used, allowing to identify anomalies in the sequence of network packet statistics. This approach allows to identify long-term network attacks, as well as to determine malicious traffic disguised as normal traffic.
Fig. 1. Scheme of classifier layers
To test the AI operation on real traffic within the test network, a stand was assembled. The structural diagram of the stand is shown in the following figure:
Fig. 2. Network protection
As external malicious influences, a specialized code library was assembled, containing basic and specialized programs for pentesting network infrastructure. To guarantee the integrity of the approach, all influences were divided into attack groups:
- DDoS (utility types: golden_eye, slow_loris, MHDDoS) – 61 utilities
- Brute-force (utility types: patator, pydictor, hydra) – 52 utilities
- Scan (utility types: nmap, nikto) – 22 utilities
- Malware (types of utilities: CoronaVirus, AvosLocker, Koxic, BlackMatter, BazarBackdoor in terms of network infection) – 19 utilities
- Exploit (utility types: metasploit+meterpreter, cobalt strike) – 9 utilities.
The classification was carried out according to the forming features - an attack group was assigned. Each group contained a subset of attacks that differed in patterns and characteristics.
Table of results of testing on the layout:
| Attack group | Accuracy | False positives | False negatives | Anomalous |
| DDoS | 91.9% | 4.3% | 3.1% | 0.7% |
| Brute force | 91.2% | 3.9% | 4.3% | 0.6% |
| Scan | 93.4% | 1.5% | 2.2% | 2.9% |
| Malware | 93.1% | 4.3% | 2.2% | 0.4% |
| Exploit | 89.8% | 6.1% | 3.2% | 0.9% |
The results of classifier testing revealed that in an extremely low percentage of cases the model confuses the attack type. Also, normal traffic can be recognized as an attack in 0.5%. There is an assumption that these results can be improved by optimizing the collection of parameters and enriching the neural network with feedback from the HoneyPot, WAF, DLP subsystems.
For a visual assessment of the classification quality, a 128x128 map was introduced into the model to display a slice of layers from a multidimensional space onto a plane, and the following hyperparameters were selected to correctly display attacks: sizes of hidden layers (74, 33, 17), the initialization function of the map layer weights, the activation function (Relu6), the probability of excluding a neuron from the training sample to counteract overfitting (0.04).
Fig. 3. AI slice map to display the classification process and generated regions
The introduction of such a visual tool is useful for transparent monitoring and control of AI: it can be used to select a control zone - this can be a network area or a group of users, select a cutoff level, enable and disable hard or soft response in the implementation of a real system.
Fig. 4. – AI cross-section map in real time
Red dots mean “fallen” on attack areas, the areas and borders themselves are a cut of the AI onto a plane.
Based on the results of the study, we can talk about the successful testing of such AI architectures for a comprehensive assessment of network attacks of selected groups or other abnormal behavior of the network infrastructure. As we can see, the system also gains the ability to provide visualization data (Kohonen map) for further investigation.
All functionality for collecting and classifying network statistics was implemented in C/C++, which allows the system to be used on any hardware and software platforms, including those with limited computing resources, ensuring the most efficient use of available hardware resources to achieve high performance.
The result of the work on creating and training a network traffic classification system were the following accuracy rates: up to 97% of network attack detection taken into account when training classifiers, and up to 92% of accuracy when determining a zero-day attack, the statistics of which were not used when training neural network models.
This technology was integrated into the next-generation firewall. The result of the joint work of NGFW and NGIDS functionality with an improved network attack classifier implemented in C++ was that the declared classification accuracy values in training classifiers and in determining a zero-day attack were confirmed and correspond to the data obtained during the experiments, which made it possible to solve the problems identified during the analysis of the functionality of NGFW platforms.
Fig. 5. Map of the neural network in a 2-dimensional slice of the multidimensional space of operators
Fig. 6. Recognition of Brute-force, DoS/DDoS, Scan attacks
Fig. 7. Recognition of anomalies in traffic and service operation
Conclusions:
Integrating AI-powered NGIDS into NGFW provides a powerful solution to protect networks from modern cyber threats. Using artificial intelligence, detection accuracy can be significantly improved, false positives can be minimized, and proactive protection against attacks can be provided. Such solutions are becoming an integral part of the cybersecurity strategy for organizations of all sizes, ensuring reliable protection of their data and infrastructure.
(c) Alexey Vysotsky, Vladimir Chuyanov
Source
