Skip to content

Bad Bot Blocker

The Nginx Ultimate Bad Bot Blocker is a set of configuration files for Nginx that block over 4000 bad referers, spam referrers, user-agents, bad bots, bad IP's, porn, gambling and clickjacking sites, lucrative seo companies, and wordpress theme detectors. It is regularly maintained and easily updateable.

Installing Bad Bot Blocker

Obtain the globalblacklist.conf file and place into your /etc/nginx/conf.d folder.

sudo wget -O /etc/nginx/conf.d/globalblacklist.conf

Create a config directory to store the include files:

sudo mkdir /etc/nginx/bots.d

Download and place the following files into that folder:

sudo wget -O /etc/nginx/bots.d/blockbots.conf
sudo wget -O /etc/nginx/bots.d/ddos.conf

Download the whitelist configuration files:

sudo wget -O /etc/nginx/bots.d/whitelist-ips.conf
sudo wget -O /etc/nginx/bots.d/whitelist-domains.conf

Add your static IPs' to the whitelist-ips.conf file and your domains to the whitelist-domains.conf file.

Now download custom blacklist files for bad user-agents, bad referrers, and bad IP addresses or ranges:

sudo wget -O /etc/nginx/bots.d/blacklist-user-agents.conf
sudo wget -O /etc/nginx/bots.d/custom-bad-referrers.conf
sudo wget -O /etc/nginx/bots.d/blacklist-ips.conf
sudo wget -O /etc/nginx/bots.d/bad-referrer-words.conf

These files are for you to edit with your own additions and won't be overwritten when you update the lists. The instructions on how to use them are within the comments in the files. Just open in nano to read.

Now we get the botblocker-nginx-settings.conf file. This file contains Nginx settings for rate limiting:

sudo wget -O /etc/nginx/conf.d/botblocker-nginx-settings.conf

Open up the Nginx config file:

sudo nano /etc/nginx/nginx.conf

and ensure it contains the following directive. Add it if not:

include /etc/nginx/conf.d/*;

And ensure you have the following 2 lines added to your virtual host files within a server block:

include /etc/nginx/bots.d/blockbots.conf;

include /etc/nginx/bots.d/ddos.conf;

Adding a dedicated log file

Now we'll create a seperate log in nginx specifically for badbots, which we can then use in fail2ban. First make the following changes in the blockbots.conf file:

sudo nano /etc/nginx/bots.d/blockbots.conf
#if ($remote_addr ~ "(|(" ) {
    #set $bad_bot  '0'; #Uncommenting this line will disable bad_bots functionality for specified IP(s)
    #set $validate_client '0'; #Uncommenting this line will disable validate_client  ip blocking functionality for specified IP(s)
set $bbcode '0';

# --------------
# --------------

# Section bot_1 Unused
#limit_conn bot1_connlimit 100;
#limit_req  zone=bot1_reqlimitip burst=50;

limit_conn bot2_connlimit 10;
limit_req  zone=bot2_reqlimitip burst=10;
if ($bad_bot = '3') {
  set $logbb 1;
  set $bbcode 'Type: Bad Bot';
  return 444;

# ---------------------
# ---------------------

if ($bad_words) {
  set $logbb 1;
  set $bbcode 'Type: Bad Word';
  return 444;

# ------------------
# ------------------

if ($bad_referer) {
  set $logbb 1;
  set $bbcode 'Type: Bad Referrer';
  return 444;

# -----------------------------
# -----------------------------

if ($validate_client) {
  set $logbb 1;
  set $bbcode 'Type: Bad IP Address';
  return 444;

# Additional log for Bad Bots
access_log  /var/log/nginx/access_badbots.log bb if=$logbb;

# If we have a global access log, we need to re-declare it here
access_log  /var/log/nginx/access.log main;

Now create the custom log format in /etc/nginx/nginx.conf and also create a default value for the $logbb variable:

sudo nano /etc/nginx/nginx.conf
http {

    log_format  bb  '$remote_addr - $remote_user [$time_local] $host "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for" "$bbcode"';


    map $host $logbb {
      default 0;

Check the configuration with:

sudo nginx -t

And apply changes (if no errors) with:

sudo service nginx reload


Run the following commands one by one from a terminal on another linux machine against your own domain name. Substitute in the examples below with your real domain name

curl -A "googlebot"

Should respond with 200 OK

curl -A "80legs"
curl -A "masscan"

Should respond with: curl: (52) Empty reply from server

curl -I -e
curl -I -e

Should respond with: curl: (52) Empty reply from server

Check the log:

sudo tail -n 10 /var/log/nginx/access_badbots.log


To update to the latest version automatically, create a script:

sudo nano /usr/local/sbin/

and enter the following:

wget -O /etc/nginx/conf.d/globalblacklist.conf
service nginx reload
exit 0

Schedule it with a cron job:

sudo crontab -e
0 22 * * * /usr/local/sbin/ > /dev/null 2>&1

Blocking bots with fail2ban

We can block persistent bad bots at firewall level using fail2ban in order to prevent excess stress on the Nginx server.

Create a filter:

sudo nano /etc/fail2ban/filter.d/nginx-badbots.conf
failregex = ^<HOST> .* 444 .*"Type: Bad Bot"$
ignoreregex =

Create a jail:

sudo nano /etc/fail2ban/jail.d/nginx-badbots.local
enabled = true
port = http,https
filter = nginx-badbots
logpath = /var/log/nginx/access_badbots.log
maxretry = 1
findtime = 36000
bantime = 86400
action = iptables-allports

Reload fail2ban:

sudo fail2ban-client reload