Stream4Flow: Real-time IP Flow Host Monitoring using Apache Spark

Warning

This publication doesn't include Faculty of Economics and Administration. It includes Institute of Computer Science. Official publication website can be found on muni.cz.
Authors

JIRSÍK Tomáš

Year of publication 2018
Type Article in Proceedings
Conference NOMS 2018 - 2018 IEEE/IFIP Network Operations and Management Symposium
MU Faculty or unit

Institute of Computer Science

Citation
Web https://ieeexplore.ieee.org/document/8406132
Doi http://dx.doi.org/10.1109/NOMS.2018.8406132
Keywords host monitoring; situation awareness; real-time; Stream4Flow
Attached files
Description In this paper, we present Stream4Flow, a framework for cyber situational awareness based on Apache Spark Streaming. We demonstrate utilization of Stream4Flow for real-time IP flow host monitoring in a large campus network. Contemporary IP flow analysis systems are not designed for the continuous host monitoring. Gaining the detailed overview of each host is not straightforward with these systems due to connection-based paradigm and performance challenges. We show that distributed stream processing is a natural solution for detailed IP flow host monitoring. Moreover, we describe a real-time host monitoring workflow in data streams in detail and present advantages of flow-based host monitoring in Apache Spark including real-time host profiling, dynamic level of detail and granularity.
Related projects:

You are running an old browser version. We recommend updating your browser to its latest version.