# Intrusion Detection System (IDS)

## **Introduction**

This project is an **Intrusion Detection System (IDS)** that allows users to upload network data files for Machine Learning (ML) inference. The system pre-processes the uploaded file, runs it through an ML model trained on the NSL-KDD dataset, and returns the classification results to the frontend. Processed results are stored in **S3**, and job status is tracked in **DynamoDB**.

The architecture is built using a **Microservices architecture** hosted on **AWS Fargate (ECS)** with an internal and external load balancer setup, ensuring secure and scalable operations.

---

## **Objectives**

The primary goals of the IDS project are:

* **Data Pre-processing**: Transform raw network traffic data (based on the NSL-KDD dataset) into a structured format for machine learning.
    
* **Model Training**: Compare multiple machine learning models to identify the most accurate and efficient one for intrusion detection.
    
* **User Interaction**: Build a user-friendly web interface using Flask for users to upload network data for classification.
    
* **File Classification**: Build the application that will classify the uploaded file based on the machine learning model.
    
* **Cloud Deployment**: Deploy the entire system using **AWS ECS Fargate**, **Terraform**, and a robust CI/CD pipeline for automation.
    
* **Scalability**: Ensure the system can handle increased traffic and adapt to evolving threats.
    
* **CI/CD Pipeline:** Create a CI/CD pipeline that’ll ensure seamless deployment of code changes.
    

---

### 🛠️ **Tech Stack**

1. **Machine Learning**: Pre-processing with Pandas and NumpPy, and training models with SciKit-learn.
    
2. **Frontend**: NextJS frontend
    
3. **Backend**: Flask-based web application for user interaction.
    
4. **Infrastructure as Code (IaC)**: Terraform for provisioning AWS resources.
    
5. **CI/CD**: Automated deployment pipeline (To Be Determined).
    
6. **Cloud Deployment**: Containerized application.
    
7. **Visualization**: Generate real-time classification insights with Matplotlib.
    

---

### 🚦 **Key Features**

* **Efficient Data Processing**: Leverages advanced preprocessing techniques to prepare data for model training.
    
* **Multi-Model Comparison**: Tested models such as Random Forest, Decision Tree, Naive Bayes, and Logistic Regression to ensure the best performance.
    
* **Interactive User Interface**: Upload network traffic data and receive immediate threat classification.
    
* **Cloud-Native Architecture**: Deployed using AWS services for high availability and scalability.
    

---

### 🔍 **Project Workflow**

1. **Data Preprocessing**: Cleaned and transformed the NSL-KDD dataset into machine-readable formats.
    
2. **Model Training**: Trained multiple models to classify threats, comparing metrics like **accuracy**, **precision**, **recall**, and **F1 score** to select the best model.
    
3. **Model Deployment**: Packaged the trained model as a Docker container and pushed it to Amazon ECR.
    
4. **Web App Development**: Created a Flask-based app for user interactions.
    
5. **Cloud Deployment**: Deployed the app on AWS ECS Fargate, ensuring security and scalability.
    

---

### 🏆 **What’s Next?**

This is just the beginning! In future posts, I’ll dive deeper into:

* Enhancing the dataset by incorporating real-world scenarios.
    
* Improving the Flask app with user authentication and a database for result tracking.
    
* Scaling the system with Kubernetes for even greater flexibility.
    
* Exploring real-time anomaly detection using advanced ML techniques.
    

---

### 🔗 **Stay Tuned!**

This post is part of my **Projects** series. For all updates and technical details on this Intrusion Detection System, check out my [**Intrusion Detection System**](https://victoriliya.hashnode.dev/series/intrusion-detection-system) series, where I'll break down each component and share insights along the way.

Thank you for following my journey! Feel free to share your thoughts or ask questions in the comments. 😊