Anybody who runs an internet-facing webserver has seen their fair share of spammy scanners in the logs. It varies server to server, but some of mine get up to 15,000 scans per day.
Almost all of these are harmless network mappers, but they still annoy me. Many are compromised hosts or belong to hackers & organized crime rings. While it’s possible to create false positives, it’s probably safe to block all of these.
So I’m going to ban these spammy scanners from my servers automatically, for several weeks at a time. Once a malicious IP has been confirmed, it will stay blocked indefinitely Here’s how:
Configure Nginx to log default virtualhost traffic
First and foremost, we need to separate the legit traffic from the scans. The easiest way to do this is to point the default virtualhost at a separate log file. For nginx, a default server block can be added to the end of nginx.conf:
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
server_name_in_redirect off;
location / {
return 404;
access_log /var/log/nginx/spam.log;
}
}
server {
listen 443 ssl default_server;
listen [::]:443 ssl default_server;
server_name _;
ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
ssl_protocols TLSv1.3 TLSv1.2;
server_name_in_redirect off;
location / {
return 404;
access_log /var/log/nginx/spam.log;
}
}
With this configuration, all traffic that does not have a valid hostname will be 404’d and logged to spam.log, keeping the actual server logs much cleaner. This logfile will also be used to generate our block list later.
Unless you have a well-known or high traffic site, this will filter about 95% of the scanner traffic, since they primarily scan IP ranges rather than by hostname.
Configure logrotate to disable compression
To make parsing historical data easier, we will disable compression on old web server log files. Make sure you have sufficient storage before doing this. For nginx, the file /etc/logrotate.d/nginx will have two lines commented out:
...
rotate 14
#compress
#delaycompress
notifempty
...
Once applied, new log files will only be rotated, not gzipped. It is possible to unzip the compressed log files for analysis, but it is much easier to have them uncompressed.
Install and configure ipset
Ipset is a tool for managing hash tables in the kernel for fast lookup. Rather than creating hundreds or thousands of individual firewall rules, a single rule can perform a quick lookup on the table without impacting performance in a meaningful way.
First, the tools must be installed:
sudo apt install ipset ipset-persistent
sudo systemctl enable --now ipset.service
Then, the table can be created:
ipset create scanners hash:ip hashsize 4096 counters
This creates an ipset list with counters enabled, which will allow us to check the hit rate on each rule later.
Process Nginx logs to build banned IP list
A simple script is used to process the nginx log and generate the ipset list:
After execution, the ipset list will contain several IPs to block:
# ipset list -t
Name: scanners
Type: hash:ip
Revision: 5
Header: family inet hashsize 4096 maxelem 65536 counters bucketsize 12 initval 0xe1ff9cd2
Size in memory: 27632
References: 0
Number of entries: 359
Additionally, a cron job should be used to regularly refresh the list with any new malicious IPs from the nginx log file:
/etc/cron.d/droplist
00 * * * * root /opt/blocklist_nginx.sh
Configure iptables to lookup & block banned IPs
The iptables rules for the server should be modified to include the set lookups. For performance reasons, established connections should be allowed to ‘bypass’ the rules - only new connections should be examined.
An example for a web server:
/etc/iptables/rules.v4
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -m state --state INVALID -j DROP
-A INPUT -m set --match-set scanners src -j DROP
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 22 -s 192.168.1.0/24 -j ACCEPT
-A INPUT -j DROP
COMMIT
Load the ruleset, either using iptables-restore or by restarting the system (if iptables-persistent is installed).
sudo iptables-restore /etc/iptables/rules.v4
Before too long, the rules will begin to get matches, and therefore block requests.
# iptables -L -v -n
Chain INPUT (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
1007K 1173M ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
12 1344 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 state INVALID
268 15228 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set scanners src
23196 1392K ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0
524 27891 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:80
2829 170K ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:443
7 412 ACCEPT tcp -- * * 192.168.1.0/24 0.0.0.0/0 tcp dpt:22
7826 688K DROP all -- * * 0.0.0.0/0 0.0.0.0/0
Likewise, the ipset list will begin to show hits on the counters:
# ipset list scanners | tail -n +9 | sort -k 3 -g -r | awk '{print $1"\t"$3"\t"$5}' | less
83.97.73.87 18477 1108620
195.226.194.242 17321 1039260
195.226.194.142 17012 1020720
141.98.11.11 13762 825400
143.110.222.166 11474 458960
This will grow over time as more requests are logged and dropped.
After 14 days, the nginx logs will be removed, and IPs will be flushed from the blocklist shortly after. Effectively, a banned IP will be blocked for just over two weeks.
It’s also possible to configure logrotate to retain additional logs, which would allow the list to grow over time and maintain a much larger list of banned hosts. This may or may not be desirable, depending on the threat model of the server.