Introduction
Imagine you're running a cloud storage platform powered by Nextcloud, serving users around the clock. One day, suspicious traffic starts flooding in — thousands of requests per second from unknown IPs. How do you detect it? How do you stop it automatically?
That's exactly the challenge I tackled as a DevSecOps Engineer. I built a real-time anomaly detection engine in Python that watches all incoming HTTP traffic, learns what "normal" looks like, and automatically blocks attackers using iptables — no Fail2Ban, no rate-limiting libraries, just raw Python and math.
The Architecture
The system runs as a Docker stack with four components:
- Nginx — reverse proxy in front of Nextcloud, writing JSON access logs
- Nextcloud — the cloud storage platform we're protecting
- PostgreSQL — database for Nextcloud
- Detector Daemon — my Python tool that monitors, detects, and blocks
Nginx writes every request to a JSON log file. The detector continuously tails this log, parses each line, and feeds the data into sliding windows and a rolling baseline.
How the Sliding Window Works
A sliding window is like a 60-second memory of recent traffic. I used Python's collections.deque to store timestamps of every request.
python
from collections import deque
class SlidingWindow:
def __init__(self, window_seconds=60):
self.window_seconds = window_seconds
self._deque = deque()
def add(self, timestamp):
self._deque.append(timestamp)
self._evict(timestamp)
def _evict(self, now):
cutoff = now - self.window_seconds
while self._deque and self._deque[0] < cutoff:
self._deque.popleft()
Every time a new request comes in, we add its timestamp. We also remove any timestamps older than 60 seconds from the left side of the deque. Since timestamps arrive in order and popleft() is O(1), this is very efficient.
I maintain two pairs of windows:
Per-IP windows — one for each IP address, tracking how many requests that IP is making
Global window — tracking total traffic across all IPs
How the Baseline Learns from Traffic
The baseline is the engine's understanding of "normal." It's a rolling 30-minute window of per-second request counts, recalculated every 60 seconds.
Every second, the detector counts how many requests came in and records that number. From those numbers, it computes:
Mean — the average requests per second over the last 30 minutes
Standard deviation — how much the traffic varies
The clever part: the baseline maintains separate statistics for each hour of the day. If it's 2 PM and the engine has seen enough 2 PM traffic, it uses the 2 PM baseline specifically. This means quiet overnight hours don't inflate daytime detection thresholds.
There are also floor values (minimum mean of 2.0, minimum stddev of 1.0) to prevent false positives during very quiet periods.
How the Detection Logic Makes a Decision
When a new request arrives, the detector checks two conditions for the source IP:
Z-score test: Is the IP's request rate more than 3 standard deviations above the mean?
Rate multiplier test: Is the IP's request rate more than 5x the baseline mean?
Whichever fires first triggers the detection. The z-score formula is simple:
z-score = (current_rate - mean) / standard_deviation
If an IP is also generating lots of error responses (4xx/5xx status codes) — more than 3x the normal error rate — the thresholds tighten automatically. The z-score threshold drops from 3.0 to 2.0, making it easier to catch the attacker.
The same logic applies globally. If total traffic across all IPs spikes, a global anomaly alert fires.
How iptables Blocks an IP
When a per-IP anomaly is detected, the engine runs an iptables command to block the attacker:
iptables -A INPUT -s <attacker_ip> -j DROP
This tells the Linux kernel: "Any packet coming from this IP? Drop it. Don't even respond." The attacker's connection just times out — they get nothing back.
The ban follows an escalating schedule:
1st offense: 10 minutes
2nd offense: 30 minutes
3rd offense: 2 hours
4th offense: permanent
A background thread checks every 30 seconds for expired bans and removes them with:
iptables -D INPUT -s <attacker_ip> -j DROP
Every ban and unban sends a notification to Slack and writes to a structured audit log.
The Live Dashboard
The engine serves a web dashboard that refreshes every 3 seconds, showing:
Current global requests per second
Effective baseline mean and standard deviation
Banned IPs with duration and offense count
Top 10 source IPs by traffic
CPU and memory usage
Baseline history chart
What I Learned
Building this from scratch taught me that DDoS detection isn't magic — it's statistics. A sliding window gives you real-time awareness, a rolling baseline gives you context, and the z-score tells you when something deviates from normal. The hardest part was getting the thresholds right so the system catches real attacks without banning legitimate users.
If you're interested in security tooling, I'd encourage you to build something similar. Start simple: tail a log file, count requests per IP, and alert when something looks off. You'll be surprised how far basic math can take you.