NEMI Toolkit
NEMI’s data substrate is flexible, composable and scalable. The toolkit brings together various associated technologies as a turn key solution for network and edge management by adopting an evolved MAPE approach. The toolkit is composed of the following building blocks:
1. Data Substrate: Scalable data exchange systems powered by industry standard data flow mechanisms, composed of edge piper (edge node manager) and central piper (central node manager) stack. Following describe the data flow pipeline used by NEMI:
- Container management and orchestration based on Docker, Kubernetes and Portainer.
- Configurable and flexible data flow pipeline based on Apache Ni-Fi.
- Message brokering system based on on nats.
2. Data Lake: The user can apply pre-processing on the data harvested from the active system before storing into the data lake for further analytics. The data persisted for further usage from the Savant modules. Data is stored in the following forms:
- Time series management based on Influxdb.
- LRU cache based on redis.
- Semi-structured data management based on mongoDB.
- Logs management based on the ELK Stack
- RDF triple store based on Apache Jena Fuseki.
3. AI/ML and Knowledge Substrate (Edge and Central Savants): Algorithms leveraging the data lake, to augment and decouple data exchange from edge management. Each Savant module focuses on a research topic that is of interest in the domain of data and network management. The Savants are built on top of the data substrate and data lake so that they can leverage data for fulfilling an AI/ML pipeline. Following Savants are currently available under incremental releases:
- Intent Based Networking: A flexible AI/ML pipeline focusing on transforms human readable intents into suitable actions performed on the network. This approach reduces the network management complexity exposed to the user.
- Federated Learning: NEMI’s reliable data exchange substrate is leveraged to exercise federated learning algorithms on devices at remote locations that experience non-reliable connectivity.
- ML based anomaly detection: An ML based approach that is trained to detect anomalies in the 5G core network when core interacts with external components. The training is based on analysing key metrics provided by the core’s network functions and system run time environment.
