Implementing a Tarpit in Nginx: Trap Malicious Scanners

If you’ve ever looked at your server logs, you’ve likely seen countless attempts to access files like /.env, /wp-login.php, /.git/config, or /admin. These are automated scanners probing your server for vulnerabilities. While simply blocking them with a 403 or 404 response works, there’s a more satisfying approach: trapping them in a tarpit.

What I found when I actually measured my own tarpit

The tarpit above isn’t a lab exercise — it’s been running on this site in production, logging 418 for every capture while serving 200 to keep the client hooked, exactly as described above. Recently I found out it had stopped doing the second half of that job for a while, and nothing in the logs gave it away: every capture still showed the expected 200/418 split, so on paper it looked like it was working perfectly.

The bug: compression undid the throttle

My production tarpit doesn’t serve the guide’s random Perl string — it serves an animated decoy page, nedry.html (255,864 bytes), followed by roughly 10 MB of zero-byte padding, for a combined payload of 10,255,882 bytes. limit_rate only throttles bytes that actually cross the wire, and Nginx applies it after compression runs. Brotli reduced that entire ~10 MB response down to 189,208 bytes (the 189,368 you’ll see in the raw log includes HTTP headers), and my limit_rate_after was set to 300 KB — comfortably above the compressed size, but nowhere near the true payload. The throttle simply never engaged: every scanner downloaded the full 10.25 MB at full speed, in a fraction of a second, and moved on. The 300 KB threshold itself was correctly chosen — nedry.html alone is ~250 KB, sized so the visible animation still loads fast for anyone who isn’t being tarpitted — the padding is what quietly broke it.

The fix, and what changed

The fix was one line, gzip off; brotli off;, in the tarpit snippet, validated with nginx -t and reloaded.

The log-level picture agrees. Comparing the 14 hours before the fix against a 9-hour window afterward, the median bytes served per capture jumped from 189,368 to 306,948 — for the first time, above the limit_rate_after threshold, which is exactly what “the throttle is finally engaging” looks like. I’d also expected captures-per-hour to fall sharply as scanners got stuck instead of firing dozens of quick requests, and it did (266/hour before, 14/hour after) — but I’m not presenting that number as proven on its own: the “after” window runs into the night, when scan traffic is naturally lower, so part of the drop could simply be the daily scanning cycle rather than the fix. Confirming it properly means comparing the same hours across different days, not two consecutive windows. The byte figures don’t have that problem — they’re a direct, mechanical consequence of turning compression off, not something traffic variation could produce.

How to verify your own tarpit is actually working

The transferable lesson here: a tarpit serving compressible content can be completely inert while its logs look perfectly healthy, because the status code and the capture count never change. If you run one, check these three things instead of trusting the log alone.

Two ways to pad the payload, and the trade-off

There are two honest ways to build the multi-megabyte filler, and neither is a free lunch — I’m presenting both rather than pretending the one I run today is simply “the right one”:

1. Zero-byte padding with compression disabled (what’s running now). Simple to generate and easy to reason about, but fragile: it depends on nothing downstream ever re-compressing the response. Any intermediate proxy that applies its own gzip or Brotli collapses the 10 MB back down to ~189 KB and silently defeats the tarpit again — no error, no alert, just a throttle that quietly stops throttling.
2. Random bytes instead of zeros. Incompressible by construction, so the tarpit works whether or not compression is active anywhere in the chain, and survives intermediate proxies that zero padding doesn’t. The cost: the file never compresses even in cases where that would help (say, serving it over a slow uplink), and a few megabytes of genuine randomness is more work to generate and regenerate than a stream of zeros.

I’m running option 1 today because it’s what I could ship immediately after finding the bug — but it’s the more fragile of the two, and now I know exactly what “fragile” looks like when it fails silently.

What is a tarpit?

A tarpit (also known as a “tar pit” or “sticky honeypot”) is a network security mechanism designed to slow down attackers by deliberately responding to their requests at an extremely slow rate. The name comes from the natural phenomenon of tar pits—geological formations where animals become trapped in viscous tar and cannot escape.

In cybersecurity, a tarpit does the same thing digitally: it accepts incoming connections from malicious actors but drip-feeds data so slowly that the attacker’s tools become stuck, wasting their time and resources while they wait for a response that never fully arrives. For more background, see Hedgehog Security’s excellent overview.

History and origins

The concept of tarpits in cybersecurity emerged in the late 1990s and early 2000s during a period when network worms and automated scanning tools began proliferating. The most famous early tarpit was LaBrea, created by Tom Liston around 2001.

LaBrea worked at the network layer, responding to TCP SYN packets for unused IP addresses and creating “virtual sticky machines” that would trap worm scanners. When the Code Red and Nimda worms were spreading rapidly, LaBrea proved remarkably effective at slowing their propagation.

Evolution of tarpits

The tarpit concept has evolved beyond network-layer implementations:

Why use a tarpit instead of blocking?

Tarpits offer several advantages over simple blocking:

1. Resource Exhaustion: Automated scanners have limited connections. Keeping them stuck reduces their scanning capacity.
2. Intelligence Gathering: Logged tarpit connections reveal attacker IP patterns and targeted paths.
3. Psychological Impact: Attackers who realize they’ve been tarpitted may avoid your server in the future.
4. Crash Poorly-Written Tools: Some scanning tools don’t handle slow responses well and may crash.

Implementation in Nginx

Let’s implement a complete, production-grade tarpit solution in Nginx. The strategy involves three components:

1. Generate slow content: Create a large random payload
2. Throttle delivery: Use limit_rate to control bandwidth
3. Log for analysis: Use a specialized log format that forces the status code to 418 (even when we return 200) so parsers like CrowdSec can easily detect it.

Step 1: prepare Nginx (http block)

We need to define our log formats and map variables in the main nginx.conf. This setup allows us to whitelist trusted IPs (like your home IP or VPN) so you don’t tarpit yourself!

http {
    # ... existing config ...

    # 1. Map to identify trusted IPs (Localhost, VPN, etc.)
    map $remote_addr $is_trusted_ip {
        127.0.0.1 1;
        ::1       1;
        # Add your static IP here if you have one
        # 1.2.3.4 1;
        default   0;
    }

    # 2. Dynamic Tarpit Rate
    # Trusted IPs get 0 (unlimited), others get 10 bytes/s
    map $is_trusted_ip $tarpit_rate {
        1 0;
        0 10;
    }

    # 2b. Only log tarpitted requests (avoid trusted IPs)
    map $is_trusted_ip $tarpit_loggable {
        1 0;
        0 1;
    }

    # 3. Special Log Format for Tarpit
    # We force the status code to 418 in the log, even though we return 200 to the client.
    # This allows CrowdSec to easily identify tarpit hits without complex parsing.
    log_format tarpit_418_fixed escape=none '$remote_addr - $remote_user [$time_local] "$request" 418 $body_bytes_sent "$http_referer" "$http_user_agent" "$host"';

    # 4. Generate random content (requires nginx-mod-http-perl)
    perl_set $slow_content 'sub {
        my $chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
        my $result = "";
        for (1..100000) {
            $result .= substr($chars, int(rand(length($chars))), 1);
        }
        return $result;
    }';
}

Step 2: create the tarpit snippet

Create a reusable snippet that handles the throttling and logging.

# Log to dedicated tarpit log only for tarpitted requests
access_log /var/log/nginx/tarpit_access.log tarpit_418_fixed if=$tarpit_loggable;

# Bypass for Trusted IPs - Return standard 403 instead of Tarpit
# This saves you from waiting 2 hours if you accidentally hit a protected path!
if ($is_trusted_ip) {
    return 403;
}

# Dynamic bandwidth throttling (0 for trusted, 10 for others)
limit_rate $tarpit_rate;

# Start throttling after the first byte
limit_rate_after 1;

# Send the slow response
add_header Content-Type text/plain;
add_header Cache-Control "no-store, no-cache, must-revalidate, max-age=0";
add_header Pragma "no-cache";
return 200 $slow_content;

Step 3: define sensitive paths (the “traps”)

Instead of adding logging logic to every location (which is messy), we simply return 418 for any sensitive path. We will handle this error code globally in the server block.

# =============================================================================
# Sensitive Files Tarpit Protection
# =============================================================================

# Environment and configuration files
location ~ /\.env { return 418; }
location ~ /\.git { return 418; }
location ~ /\.svn { return 418; }
location ~ /\.hg { return 418; }

# Server configuration files
location ~ /\.(htaccess|htpasswd) { return 418; }

# Backup and sensitive file extensions
location ~* \.(bak|backup|old|orig|save|swp|tmp|sql|db|sqlite|sqlite3)$ { return 418; }

# PHP/CMS exploit paths
location ~* (phpinfo|adminer|phpmyadmin|wp-login|xmlrpc|wp-admin|wp-config)\.php$ { return 418; }

# Common scanner/exploit paths
location ~* ^/(config|admin|administrator|login|cgi-bin|scripts|shell|cmd|console) { return 418; }

# Catch-all for dotfiles (excluding .well-known)
location ~ /\.(?!well-known) {
    return 418;
}

Step 4: configure the server block

Now we tie it all together using error_page 418. This is the “magic glue” that makes the architecture clean.

server {
    listen 443 ssl;
    server_name example.com;

    # ... SSL and other configuration ...

    # =========================================================
    # TARPIT HANDLER
    # =========================================================
    
    # 1. Catch 418 errors
    error_page 418 = @tarpit;
    
    # 2. Redirect them to the tarpit snippet
    location @tarpit {
        include /etc/nginx/snippets/tarpit.conf;
    }

    # =========================================================
    # SITE CONFIGURATION
    # =========================================================
    
    # Include the traps
    include /etc/nginx/snippets/sensitive_files_tarpit.conf;
    
    location / {
        try_files $uri $uri/ =404;
    }
}

Step 5: test the configuration

Validate and reload Nginx:

Test the tarpit with a timeout to avoid waiting forever:

The connection was established, received 50 bytes (5 seconds × 10 bytes/second), and then our timeout kicked in. In reality, an attacker’s scanner would wait much longer!

Integration with CrowdSec

While the tarpit wastes attacker time, we can go further by automatically banning repeat offenders. CrowdSec can parse the tarpit log and create firewall rules.

Create a CrowdSec parser

CrowdSec can read the /var/log/nginx/access.tarpit file to identify malicious IPs:

name: crowdsec/nginx-tarpit
description: "Parse Nginx tarpit access logs"
filter: "evt.Parsed.program == 'nginx-tarpit'"
onsuccess: next_stage
statics:
  - meta: service
    value: http
  - meta: log_type
    value: tarpit
  - meta: source_ip
    expression: "evt.Parsed.remote_addr"
  - meta: http_path
    expression: "evt.Parsed.request"
nodes:
  - grok:
      pattern: '%{IPORHOST:remote_addr} - %{DATA:user} \[%{HTTPDATE:time}\] "%{WORD:method} %{DATA:request} HTTP/%{NUMBER:http_version}" %{NUMBER:status} %{NUMBER:bytes}'
      apply_on: message

Create a scenario

Define when to ban an IP (e.g., after 3 tarpit hits in 5 minutes):

type: leaky
name: crowdsec/nginx-tarpit-scan
description: "Detect and ban IPs triggering the Nginx tarpit"
filter: "evt.Meta.service == 'http' && evt.Meta.log_type == 'tarpit'"
leakspeed: 5m
capacity: 3
groupby: "evt.Meta.source_ip"
blackhole: 1h
labels:
  service: http
  type: scan
  remediation: true

Reporting malicious IPs: contributing to collective defense

One of the most powerful aspects of running a tarpit is the intelligence you gather. Every trapped connection reveals an attacker’s IP, their target paths, and timing patterns. But this data becomes exponentially more valuable when shared with the security community.

Why report malicious IPs?

Benefits of reporting include:

1. Early Detection: Other organizations can block threats you’ve identified before being targeted
2. Pattern Recognition: Aggregated data reveals coordinated attacks, botnets, and malware campaigns
3. Reduced Attack Surface: Reported IPs are often blocked by ISPs and security tools globally
4. Community Reciprocity: Contributors receive access to larger, more comprehensive blocklists

AbuseIPDB integration

AbuseIPDB is a community-driven threat intelligence platform with over 100,000 contributors. You can both check IP reputation and report malicious IPs via their API.

Check IP reputation

Before blocking, verify the threat level:

Report malicious IPs from tarpit logs

Create a script to automatically report tarpit captures:

#!/bin/bash
# Report IPs from tarpit log to AbuseIPDB
# Usage: Run via cron every hour

set -euo pipefail

ABUSEIPDB_KEY="${ABUSEIPDB_KEY:-}"
LOGFILE="/var/log/nginx/access.tarpit"
REPORTED="/var/log/nginx/reported_ips.txt"
REPORT_LOG="/var/log/nginx/abuseipdb_reports.log"

# Validate API key
if [[ -z "$ABUSEIPDB_KEY" ]]; then
  echo "[$(date -Iseconds)] ERROR: ABUSEIPDB_KEY not set" >> "$REPORT_LOG"
  exit 1
fi

# Ensure files exist
touch "$REPORTED" "$REPORT_LOG"

# Get unique IPs from last hour
awk -v d1="$(date --date='-1 hour' '+%d/%b/%Y:%H')" \
  '$4 ~ d1 {print $1}' "$LOGFILE" | sort -u | while read -r ip; do
  
  # Skip if already reported today
  grep -q "$ip" "$REPORTED" 2>/dev/null && continue
  
  # Report to AbuseIPDB with error handling
  response=$(curl -s -w "\n%{http_code}" "https://api.abuseipdb.com/api/v2/report" \
    -H "Key: $ABUSEIPDB_KEY" \
    -H "Accept: application/json" \
    --data-urlencode "ip=$ip" \
    --data-urlencode "categories=21,15" \
    --data-urlencode "comment=Vulnerability scanner trapped in HTTP tarpit." \
    2>&1) || true
  
  http_code=$(echo "$response" | tail -n1)
  body=$(echo "$response" | sed '$d')
  
  if [[ "$http_code" == "200" ]]; then
    echo "$ip $(date +%Y-%m-%d)" >> "$REPORTED"
    echo "[$(date -Iseconds)] OK: Reported $ip" >> "$REPORT_LOG"
  else
    echo "[$(date -Iseconds)] FAIL: $ip (HTTP $http_code) $body" >> "$REPORT_LOG"
  fi
done

CrowdSec community blocklists

Unlike AbuseIPDB (which is passive lookup), CrowdSec operates as a “massively multiplayer firewall”. When your Security Engine detects a threat, it shares the signal with the network, and you receive blocklist updates containing threats detected by other users.

The Community Blocklist updates in real-time and is tailored to your stack—if you run WordPress, you’ll receive WordPress-specific threat IPs automatically.

Advanced techniques

Combining with rate limiting

For extra protection, combine tarpit with rate limiting to prevent overwhelming your server:

# Define rate limit zone
limit_req_zone $binary_remote_addr zone=tarpit_zone:10m rate=1r/s;

location @tarpit {
    limit_req zone=tarpit_zone burst=5 nodelay;
    limit_rate 10;
    # ... rest of tarpit config
}

Tarpit for specific user-agents

Target known malicious scanners by user-agent. Common scanners include sqlmap (SQL injection), Nikto (web vulnerability scanner), Nmap (network scanner), masscan (port scanner), and ZGrab (banner grabber):

map $http_user_agent $is_scanner {
    default                     0;
    "~*sqlmap"                  1;
    "~*nikto"                   1;
    "~*nmap"                    1;
    "~*masscan"                 1;
    "~*zgrab"                   1;
}

server {
    if ($is_scanner) {
        return 418;
    }
    # ... rest of server
}

Monitoring your tarpit

Analyze tarpit logs

Since we force the status code to 418 in our tarpit_418_fixed log format, filtering for attacks is incredibly simple, even if mixed with other logs:

  847 /.env
  312 /wp-login.php
  201 /.git/config
  156 /admin/
   98 /phpmyadmin/
   87 /.htaccess
   65 /config.php
   43 /backup.sql
   38 /shell.php
   29 /xmlrpc.php

Top offending IPs

Identify the most persistent scanners:

  423 185.234.xx.xx
  287 45.148.xx.xx
  156 194.169.xx.xx
   98 23.94.xx.xx
   67 89.248.xx.xx

Real-world results

Implementing the production-grade architecture with forced 418 logging has significantly improved visibility:

• Clear Signal: The 418 status code acts as a high-fidelity signal for “confirmed malicious intent”.
• Zero False Positives: By using the trusted IP map, admin traffic is never accidentally tarpitted.
• Resource Protection: The rate limit bypass ensures valid traffic flows smoothly while attackers are stuck in the slow lane.
• CrowdSec Integration: The parser logic became trivial—simply matching status == 418 is enough to ban an IP, regardless of the requested path.

Conclusion

A tarpit is a creative and effective addition to your security toolkit. While it shouldn’t replace proper hardening and access controls, it provides:

1. Resource waste for automated scanners
2. Intelligence gathering on attack patterns
3. Automatic banning when combined with CrowdSec or Fail2Ban
4. A psychological deterrent for attackers

The implementation is straightforward in Nginx using just a few directives (limit_rate, error_page, and location blocks). Combined with logging and a security tool like CrowdSec, you create a layered defense that not only blocks attackers but makes them pay for their intrusion attempts.