This project is part of the HNG DevOps internship (Stage 3), and trust me, it sounds way more complicated than it actually is. Let's break it down together.

What Are We Building?

Think of it like hiring a smart security guard for your website who:

1. Watches the door - Keeps track of everyone visiting your site
2. Learns the pattern - Figures out what "normal" traffic looks like
3. Spots the troublemakers - Detects when something fishy is happening
4. Takes action - Blocks suspicious visitors automatically
5. Sends you alerts - Notifies you on Slack when there's trouble
6. Shows a dashboard - Gives you a live view of what's happening

Cool, right? Let's see how it all works!

Understanding the Key Concepts (No Jargon, I Promise!)

1. The Sliding Window (Think: Security Camera Footage)

Imagine you have a camera recording your front door. Instead of keeping all footage forever, you only keep the last 60 seconds. When a new second is recorded, the oldest one gets deleted.

In our system:

• We track how many people visited in the last 60 seconds
• Every second, we add new data and remove the oldest
• This gives us a "rolling" view of recent activity

# Simple example (Python makes this easy!)
from collections import deque

# This automatically keeps only the last 60 items
recent_traffic = deque(maxlen=60)

# Every second, we add new data
recent_traffic.append(current_visitors)

# Old data automatically disappears!

2. The Baseline (Your "Normal" Traffic Pattern)

Your baseline is like knowing your daily routine. If you usually get 10-15 visitors per second, that's your "normal."

How we calculate it:

• Watch traffic for 30 minutes
• Calculate the average (mean)
• Calculate how much it varies (standard deviation)

Example:

• Normal traffic: 13 visitors/second ± 2
• This means 11-15 is totally normal
• But 50 visitors/second? That's suspicious!

import statistics

# Collect 30 minutes of data
traffic_data = [12, 15, 13, 14, 16, 11, 12, ...]

# Calculate your "normal"
average = statistics.mean(traffic_data)  # Example: 13.5
variation = statistics.stdev(traffic_data)  # Example: 2.1

3. The Z-Score (How Weird is This Traffic?)

The z-score is just a fancy way of asking: "How unusual is this compared to normal?"

Simple formula:

z-score = (current traffic - normal traffic) / variation

What it means:

• Z-score of 0 = Perfectly normal
• Z-score of 1-2 = A bit high, but okay
• Z-score of 3+ = ALERT! Something's wrong!

Real example:

normal = 13.5 visitors/second
variation = 2.1
current = 30 visitors/second

z_score = (30 - 13.5) / 2.1
# Result: 7.86 — Definitely an attack!

if z_score > 3.0:
    print("🚨 ATTACK DETECTED!")

4. The 5x Rule (Backup Check)

Sometimes the z-score isn't enough. The 5x rule is simple:

If traffic is 5 times higher than normal, it's an attack — no matter what!

Why? If you normally get 2 visitors/second, even 10 might not trigger the z-score, but it's still 5x your normal traffic!

Setting Up Your Project

What You'll Need

1. A Google Cloud account (free tier works!)
2. A Slack account (to receive alerts)
3. Basic knowledge of:
   • Running commands in terminal
   • What Docker is (even just a basic idea)
   • Python basics (if statements, loops)

Step 1: Create Your Cloud Server

1. Go to Google Cloud Console
2. Create a new VM (virtual machine):
   • Name: hng-stage3
   • Type: e2-medium (2 CPU, 4GB RAM)
   • Disk: Ubuntu 22.04 LTS, 20GB
   • Firewall: Allow HTTP and HTTPS traffic
3. Click "Create" and wait a minute

Don't forget: Open port 8080 for your dashboard!

• Go to VPC Network → Firewall
• Create a new rule called allow-dashboard-8080
• Allow TCP port 8080 from anywhere (0.0.0.0/0)

Step 2: Connect to Your Server

Click the "SSH" button next to your VM in Google Cloud Console. A terminal will open — this is where the magic happens!

Step 3: Install the Tools

Run these commands one by one:

# Update your system
sudo apt update && sudo apt upgrade -y

# Install Docker (the easy way!)
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

# Install Docker Compose
sudo apt install docker-compose -y

# Install Python
sudo apt install python3 python3-pip git -y

# Create your project folder
mkdir ~/hng-stage3
cd ~/hng-stage3

Congrats! Your server is ready! 🎉

Building the Detection System

Project Structure

Here's how we'll organize everything:

hng-stage3/
├── docker-compose.yml          # Starts all our services
├── nginx/
│   └── nginx.conf              # Web server config
├── detector/
│   ├── main.py                 # Main program
│   ├── monitor.py              # Watches the logs
│   ├── baseline.py             # Calculates "normal"
│   ├── detector.py             # Spots attacks
│   ├── blocker.py              # Blocks bad IPs
│   ├── notifier.py             # Sends Slack alerts
│   ├── dashboard.py            # Web dashboard
│   ├── config.yaml             # Your settings
│   └── requirements.txt        # Python packages needed

The Core Logic (In Plain English!)

1. Monitor the Traffic (monitor.py)

This script watches your web server logs and counts visitors every second.

# Every second, check how many people visited
def count_visitors():
    # Read the log file
    # Count new entries
    # Return the number
    return visitor_count

2. Learn What's Normal (baseline.py)

After collecting data for 30 minutes, calculate your baseline.

def calculate_baseline(data):
    average = sum(data) / len(data)
    # Calculate how much traffic varies
    variation = calculate_standard_deviation(data)

    return {
        'mean': average,
        'stddev': variation
    }

3. Detect Attacks (detector.py)

Compare current traffic to your baseline using the z-score.

def is_attack(current_traffic, baseline):
    z_score = (current_traffic - baseline['mean']) / baseline['stddev']

    # Check z-score rule
    if z_score > 3.0:
        return True

    # Check 5x rule
    if current_traffic > (baseline['mean'] * 5):
        return True

    return False

4. Block the Attacker (blocker.py)

Use iptables (a firewall tool) to block the bad IP address.

def block_ip(ip_address):
    # Add firewall rule to block this IP
    os.system(f"iptables -A INPUT -s {ip_address} -j DROP")
    print(f"🚫 Blocked {ip_address}")

5. Send an Alert (notifier.py)

Post a message to Slack so you know what's happening.

import requests

def send_slack_alert(message):
    webhook_url = "YOUR_SLACK_WEBHOOK_URL"

    payload = {"text": message}
    requests.post(webhook_url, json=payload)

Testing Your System

Step 1: Let It Learn (5-10 minutes)

Generate some normal traffic so the system can build a baseline:

# Install the testing tool
sudo apt install apache2-utils -y

# Send normal traffic (10 requests/second for 60 seconds)
ab -n 600 -c 1 -t 60 http://localhost/

Watch your detector logs:

docker logs hng-detector -f

You should see it learning and updating the baseline!

Step 2: Simulate an Attack

Now let's test if it can detect attacks:

# Send 100 requests/second for 60 seconds
ab -n 6000 -c 10 -t 60 http://localhost/

What should happen:

1. Detector spots the unusual traffic
2. Calculates a high z-score
3. Blocks the IP address
4. Sends you a Slack alert
5. Shows the attack on your dashboard

If you see all of this, you did it! 🎊

The Dashboard

Your dashboard runs on port 8080 and shows:

• Live traffic graph - See requests per second
• Baseline tracker - Your "normal" traffic pattern
• Recent alerts - What attacks were detected
• Blocked IPs - Who's been banned

Common Problems (And How to Fix Them)

Problem 1: "My logs are empty!"

Solution:

# Make sure the web server is running
docker ps

# Test by visiting your site
curl http://localhost/

# Check if logs are being created
docker exec hng-nginx tail /var/log/nginx/hng-access.log

Problem 2: "The detector isn't blocking anything!"

Solution:

# Make sure the detector has permission to use the firewall
docker inspect hng-detector | grep -i privileged
# Should show: "Privileged": true

# If not, add this to your docker-compose.yml:
# privileged: true
# network_mode: "host"

Problem 3: "Dashboard won't load!"

Solution:

# Check if the port is open on GCP
# Go to: VPC Network → Firewall → allow-dashboard-8080

# Test from your computer:
curl http://YOUR_SERVER_IP:8080/api/metrics

What I Learned

Building this project taught me:

1. How real security systems work - It's not magic, just math!
2. The power of baselines - Understanding "normal" helps you spot "abnormal"
3. Why automation matters - Blocking attacks manually would be impossible
4. Docker makes deployment easy - Everything runs in containers
5. Monitoring is crucial - You can't fix what you can't see

Key Takeaways

Resources

• 🐍 Python Documentation
• 🐳 Docker Getting Started
• 🔔 Slack Webhooks Guide

Good luck, and happy coding! 🚀

P.S. If you found this helpful, give it a ❤️ and follow me for more DevOps tutorials!