Blocking Malicious Bots

Web scrapers, aggressive AI crawlers, and malicious bots consume bandwidth, steal content, and slow down your origin.

What does HEIMWALL do automatically?

For every request, HEIMWALL automatically calculates a Bot Score (0–100) by combining signature, behaviour, and infrastructure signals. IPs belonging to known malicious bot networks are already blocked via the Threat List. The rules below add an extra action layer on top of this automatic detection.

Basic Rules

# Block all requests with a high bot score
IF   waf.bot_score ge 80
THEN block

# CAPTCHA for medium bot score
IF   waf.bot_score ge 60
THEN captcha

# Allow search/SEO bots, block all other bots
IF   waf.client.bot == on
AND  waf.client.bot_category not_in search, seo, monitoring
THEN block

With these selections in the console:

• Block high bot score — Field Bot Score (0–100) · Operator greater or equal · Value 80 → Action Block
• CAPTCHA for medium bot score — Field Bot Score (0–100) · Operator greater or equal · Value 60 → Action CAPTCHA
• Block other bots — Field Known Bot · Operator equals · Value Yes AND Field Bot Category · Operator not in list · Value search, seo, monitoring → Action Block

waf.bot_score returns a confidence score from 0 to 100 for each request; the closer to 100, the more certain the bot detection. Blocking requests at 80 and above keeps the risk of false positives on real users to a minimum.

For mid-range scores (60–79), prefer CAPTCHA. Some automation tools leave weak signatures; CAPTCHA validates traffic in this grey area without blocking legitimate users.

The waf.client.bot_category not_in search, seo, monitoring rule explicitly excludes known search engine crawlers and monitoring services from blocking. This ensures your site’s indexing and uptime tracking are not disrupted.

Advanced Variant

# Block AI crawlers on /api paths but allow on other paths
IF   waf.client.bot_category == ai_crawler
AND  http.request.base_uri has_prefix /api
THEN block

# Redirect non-bot requests from hosting IPs to CAPTCHA
IF   waf.client.ip_type == hosting
AND  waf.client.bot == off
AND  waf.bot_score ge 40
THEN captcha

With these selections in the console:

• Block AI crawlers on the API — Field Bot Category · Operator equals · Value ai_crawler AND Field Path (without query) · Operator starts with · Value /api → Action Block
• CAPTCHA for suspicious hosting traffic — Field IP Infrastructure Type · Operator equals · Value hosting AND Field Known Bot · Operator equals · Value No AND Field Bot Score (0–100) · Operator greater or equal · Value 40 → Action CAPTCHA

AI crawlers often target API endpoints to harvest raw data. This rule blocks AI-class bots only under the /api prefix, while continuing to allow the same category of bots on static content or blog pages.

The second rule catches traffic from data center or cloud hosting IPs that carries no explicit bot signature but appears moderately suspicious. A real user can still solve the CAPTCHA; automated tools generally cannot.

Tips

• Always allow legitimate bots with waf.client.bot_category in search, seo to protect your search ranking
• Test with log action first, then switch to block after observing thresholds
• You can allowlist Googlebot by name: waf.client.bot_name == googlebot → allow