Tesis doctoral de Ignasi Paredes Oliva
Network monitoring has always been a topic of foremost importance for both network operators and researchers for multiple reasons ranging from anomaly detection to traffic classification or capacity planning. Nowadays, as networks become more and more complex, traffic increases and security threats reproduce, achieving a deeper understanding of what is happening in the network has become an essential necessity. In particular, due to the considerable growth of cybercrime, research on the field of anomaly detection has drawn significant attention in recent years and tons of proposals have been made. All the same, when it comes to deploying solutions in real environments, some of them fail to meet some crucial requirements. Taking this into account, this thesis focuses on filling this gap between the research and the non-research world. Prior to the start of this work, we identify several problems. First, there is a clear lack of detailed and updated information on the most common anomalies and their characteristics. Second, unawareness of sampled data is still common although the performance of anomaly detection algorithms is severely affected. Third, operators currently need to invest many work-hours to manually inspect and also classify detected anomalies to act accordingly and take the appropriate mitigation measures. This is further exacerbated due to the high number of false positives and false negatives and because anomaly detection systems are often perceived as extremely complex black boxes. analysing an issue is essential to fully comprehend the problem space and to be able to tackle it properly. Accordingly, the first block of this thesis seeks to obtain detailed and updated real-world information on the most frequent anomalies occurring in backbone networks. It reports on the performance of different commercial systems for anomaly detection and analyses the types and the characteristics of the network anomalies detected. on the whole, the presence of sampling in large networks for monitoring purposes has become almost mandatory as opposed to full packet captures and, therefore, all anomaly detection algorithms that do not take that into account might report incorrect results. In the second block of this thesis, the dramatic impact of sampling on the performance of well-known anomaly detection techniques is analysed and confirmed. However, we show that the results change significantly depending on the sampling technique used and also on the common metric selected to perform the comparison. Furthermore, we propose a sampling technique that obtains the same good performance for scan detection than a recently proposed technique but works on a per-packet basis instead of keeping all flows in memory, thus using much less resources and being implementable in routers to work online. although the literature is plenty of techniques for detecting anomalous events, research on anomaly classification and extraction (e.G., To further investigate what happened or to share evidence with third parties involved) is rather marginal. This makes it harder for network operators to analyse reported anomalies because they depend solely on their experience to do the job. Furthermore, this task is an extremely time-consuming and error-prone process. The third block of this thesis presents a system for automatic anomaly detection, extraction and classification with high accuracy and very low false positives. We deploy the system in an operational environment and show its usefulness in practice. the fourth and last block of this thesis presents a generalisation of our system that focuses on analysing all the traffic, not only network anomalies. This new system seeks to further help network operators by summarising the most significant traffic patterns in their network. In particular, we generalise our system to deal with big network traffic data.
Datos académicos de la tesis doctoral «Addressing practical challenges for anomaly detection in backbone networks«
- Título de la tesis: Addressing practical challenges for anomaly detection in backbone networks
- Autor: Ignasi Paredes Oliva
- Universidad: Politécnica de catalunya
- Fecha de lectura de la tesis: 29/07/2013
Dirección y tribunal
- Director de la tesis
- Pere Barlet Ros
- Tribunal
- Presidente del tribunal: nicholas geoffrey Duffield
- rubén Cuevas rumín (vocal)
- (vocal)
- (vocal)