AIWAF

A self-learning Web Application Firewall for Python web applications. Framework-agnostic core with optional Django, Flask, and FastAPI adapters.

AIWAF provides context-aware protection with rate limiting, anomaly detection, honeypots, UUID tamper protection, smart keyword learning, file-extension probing detection, exempt path/IP awareness, and scheduled retraining.

Latest Enhancements

• Smart keyword filtering to avoid blocking legitimate paths like /profile/
• Granular reset controls for blacklist, keywords, and exemptions
• Context-aware learning that prioritizes suspicious traffic over normal routes
• Enhanced keyword controls via AIWAF_ALLOWED_PATH_KEYWORDS and AIWAF_EXEMPT_KEYWORDS
• Comprehensive HTTP method validation in honeypot logic
• Enhanced honeypot timing with page expiry/reload flow
• Header validation with quality scoring and bot-pattern detection

---

Quick Installation

pip install aiwaf

Optional framework extras:

pip install "aiwaf[django]"
pip install "aiwaf[flask]"
pip install "aiwaf[fastapi]"
pip install "aiwaf[rust]"

Rust extra installs aiwaf-rust, which accelerates selected paths and is required for persisted AI model inference from JSON IsolationForest artifacts.

Important:

• Use the adapter package for your framework (aiwaf.django, aiwaf.flask, or aiwaf.fast).
• For Django setup and command details, see INSTALLATION.md and REPO_GUIDE_DJANGO.md.

---

System Requirements

• Python 3.8+
• CPU-only operation (no GPU required)
• Small deployments: ~1 vCPU and ~512 MB RAM
• Moderate deployments: 2 to 4 vCPU and 2 to 4 GB RAM recommended
• For production, schedule detect/train jobs and rotate logs

---

Package Structure

aiwaf/
  core/                         # framework-agnostic helpers, training, storage abstractions
  core/geolock/ipinfo_lite.mmdb # bundled GeoIP database
  django/                       # Django adapter (middleware, models, trainer, commands)
  flask/                        # Flask adapter (integration class, middleware, CLI helpers)
  fast/                         # FastAPI adapter (middleware, decorators, CLI helpers)

Framework entry points:

# Django
import aiwaf.django as aiwaf

# Flask
import aiwaf.flask as aiwaf

# FastAPI
import aiwaf.fast as aiwaf

---

Features

• IP blocklist

  • blocks known suspicious sources quickly
  • supports runtime updates through adapter storage

• Rate limiting

  • sliding-window request control (AIWAF_RATE_WINDOW, AIWAF_RATE_MAX)
  • flood threshold support (AIWAF_RATE_FLOOD) for aggressive abuse

• AI anomaly detection

  • IsolationForest-based behavioral detection
  • model training updates as traffic grows
  • persisted runtime models use JSON-only artifacts; Rust IsolationForest is required for saved-model inference

• Dynamic keyword learning

  • learns suspicious path terms from attack-like traffic
  • excludes exempt/allowed terms to reduce false positives

• File-extension probing detection

  • detects repeated probes for extensions like .php, .asp, .jsp

• Header validation

  • missing required-header detection
  • suspicious user-agent and header-combination checks
  • header quality scoring
  • static-asset exemption support

• Enhanced honeypot timing

  • GET to POST timing checks via AIWAF_MIN_FORM_TIME
  • page-age validation via AIWAF_MAX_PAGE_TIME
  • method-misuse checks (for example POST to read-only endpoints)

• UUID tamper protection

  • score-based UUID abuse detection
  • malformed UUIDs add high score and can block immediately
  • valid UUID requests that repeatedly end in 404 increase score
  • score decays on successful UUID requests
  • blocks when per-IP UUID score crosses threshold

• GeoIP support

  • optional country-level allow/block behavior
  • local bundled MMDB support by default

• Built-in logging path

  • adapter-level request logging can feed training when primary access logs are unavailable

---

Header Validation Details

What it detects:

• missing core browser-like headers
• low-diversity header sets typical of simple bots
• suspicious or automation-focused user agents
• unrealistic header combinations

What it allows:

• normal browser traffic with complete headers
• well-identified clients and known legitimate bots
• static file requests when exempt patterns are configured

Useful test pattern:

# often low-quality header profile
curl http://your-app.example/

# compare against normal browser traffic

---

Exemptions and Safe Routing

AIWAF supports:

• exempt paths (AIWAF_EXEMPT_PATHS)
• exempt IPs (adapter-managed allowlists)
• exempt keywords (AIWAF_EXEMPT_KEYWORDS)
• allowed route keywords (AIWAF_ALLOWED_PATH_KEYWORDS)

Effects of exemption:

• excluded from keyword learning
• bypass of selected blocking paths
• reduced false positives on trusted operational routes (webhooks, health, static assets)

Decorator-based exemptions:

• Django adapter and Flask adapter both expose exemption decorators in their adapter modules.

---

Training and Retraining

Training pipeline:

1. Read configured access logs or adapter logger output
2. Detect suspicious patterns (including heavy 404 probe behavior)
3. Train/update IsolationForest when AI thresholds are met
4. Refresh dynamic keywords from suspicious traffic
5. Remove exempt/allowed noise from learned keyword set

Thresholds:

• AIWAF_MIN_AI_LOGS default 10,000 for full AI training
• AIWAF_MIN_TRAIN_LOGS default 50 for keyword-focused fallback
• AIWAF_FORCE_AI_TRAINING can override AI threshold gating

Daily retraining is recommended for active internet-facing workloads.

Model persistence is intentionally JSON-only. AIWAF does not load Python object model artifacts (pickle, joblib, or skops) because those formats can execute code during deserialization. Scikit-learn models may be used during a training run for immediate analysis, but they are not persisted. To persist and reload runtime AI models, enable the Rust backend so aiwaf_rust.IsolationForest can save/load JSON state. The old bundled model.pkl artifacts have been removed; retrain to generate a model.json artifact.

---

Configuration (AIWAF_*)

AIWAF uses flat AIWAF_* settings/config keys. Some knobs are adapter-specific; core controls are shared.

Required in most deployments:

AIWAF_ACCESS_LOG = "/var/log/nginx/access.log"

Core defaults (examples):

AIWAF_DISABLE_AI = False
AIWAF_MIN_AI_LOGS = 10000
AIWAF_MIN_TRAIN_LOGS = 50
AIWAF_FORCE_AI_TRAINING = False
AIWAF_AI_CONTAMINATION = 0.05

AIWAF_RATE_WINDOW = 10
AIWAF_RATE_MAX = 20
AIWAF_RATE_FLOOD = 10
AIWAF_WINDOW_SECONDS = 60

AIWAF_MIN_FORM_TIME = 1.0
AIWAF_MAX_PAGE_TIME = 240
AIWAF_FILE_EXTENSIONS = [".php", ".asp", ".jsp"]

AIWAF_UUID_SCORE_ENABLED = True
AIWAF_UUID_SCORE_WINDOW_SECONDS = 60
AIWAF_UUID_SCORE_BLOCK_THRESHOLD = 5
AIWAF_UUID_SCORE_MALFORMED_WEIGHT = 5
AIWAF_UUID_SCORE_NOT_FOUND_WEIGHT = 1
AIWAF_UUID_SCORE_SUCCESS_DECAY = 2

AIWAF_ALLOWED_PATH_KEYWORDS = ["profile", "user", "account", "dashboard"]
AIWAF_EXEMPT_KEYWORDS = ["api", "webhook", "health", "static", "media"]
AIWAF_EXEMPT_PATHS = ["/favicon.ico", "/robots.txt", "/static/", "/health/"]

Model storage:

AIWAF_MODEL_PATH = "aiwaf/resources/model.json"
AIWAF_MODEL_STORAGE = "file"          # file | db | cache
AIWAF_MODEL_CACHE_KEY = "aiwaf:model"
AIWAF_MODEL_CACHE_TIMEOUT = None
AIWAF_MODEL_STORAGE_FALLBACK = True

Only JSON-serializable model artifacts are saved. Python object model artifacts are rejected by design.

Header controls:

AIWAF_REQUIRED_HEADERS = None         # list or method->list mapping
AIWAF_HEADER_QUALITY_MIN_SCORE = 3

GeoIP:

AIWAF_GEO_BLOCK_ENABLED = False
AIWAF_GEOIP_DB_PATH = "aiwaf/core/geolock/ipinfo_lite.mmdb"
AIWAF_GEO_BLOCK_COUNTRIES = ["CN", "RU"]
AIWAF_GEO_ALLOW_COUNTRIES = []
AIWAF_GEO_CACHE_SECONDS = 3600
AIWAF_GEO_CACHE_PREFIX = "aiwaf:geo:"

Rust acceleration:

AIWAF_USE_RUST = False
AIWAF_RUST_ISOLATION_FOREST = True

When enabled, AIWAF attempts Rust-backed helpers and falls back to Python automatically. Persisted AI model loading requires a JSON artifact; the Rust IsolationForest backend provides the supported JSON model state. Pickle-based model.pkl artifacts are no longer shipped or loaded.

Legacy compatibility:

• if you still use nested AIWAF_SETTINGS, AIWAF maps common keys into flat AIWAF_* values at startup.

---

Middleware Setup

Order matters in all adapters. Put protection middleware early and logging middleware near the end.

Unified all / auto Selection

AIWAF now supports a centralized "enable everything with smart defaults" mode across adapters.

• FastAPI and Flask: pass middlewares=["all"] (or ["auto"])
• Django: use "aiwaf.django.middleware.all" in MIDDLEWARE

Auto behavior:

• logging middleware is enabled when AIWAF_ACCESS_LOG is missing/empty
• logging middleware is disabled when AIWAF_ACCESS_LOG is configured
• geo middleware is enabled when any of these are true:
  • explicit geo enable flag is on
  • static geo block list has countries
  • dynamic geo block store/table has countries

Django example order:

MIDDLEWARE = [
    "aiwaf.django.middleware.JsonExceptionMiddleware",
    "aiwaf.django.middleware.GeoBlockMiddleware",
    "aiwaf.django.middleware.IPAndKeywordBlockMiddleware",
    "aiwaf.django.middleware.RateLimitMiddleware",
    "aiwaf.django.middleware.AIAnomalyMiddleware",
    "aiwaf.django.middleware.HoneypotTimingMiddleware",
    "aiwaf.django.middleware.UUIDTamperMiddleware",
    "aiwaf.django.middleware.HeaderValidationMiddleware",
    "aiwaf.django.middleware_logger.AIWAFLoggerMiddleware",
]

If JSON API clients need JSON 403 bodies, keep JsonExceptionMiddleware near the top.

Django alias example:

MIDDLEWARE = [
    "django.middleware.security.SecurityMiddleware",
    "aiwaf.django.middleware.all",
]

FastAPI quick integration:

from fastapi import FastAPI
from aiwaf.fast import AIWAF

app = FastAPI()

aiwaf = AIWAF(
    app,
    middlewares=["all"],
    storage={"backend": "memory"},
    header_validation={"enabled": True, "quality_threshold": 3},
    rate_limiting={"enabled": True, "window_seconds": 10, "max_requests": 20},
    logging_middleware={"enabled": True, "log_dir": "aiwaf_logs", "log_format": "json"},
)

Flask quick integration:

from flask import Flask
from aiwaf.flask import AIWAF

app = Flask(__name__)
aiwaf = AIWAF(app, middlewares=["all"])

---

Operations

Django adapter examples:

python manage.py detect_and_train
python manage.py regenerate_model
python manage.py aiwaf_reset --keywords --confirm
python manage.py add_ipexemption 203.0.113.10 --reason "trusted integration"
python manage.py add_pathexemption /api/webhooks/ --reason "partner callbacks"
python manage.py aiwaf_logging --status
python manage.py geo_block_country list
python manage.py geo_block_country add US
python manage.py geo_block_country remove US

Flask adapter:

• use aiwaf.flask.AIWAF for middleware registration
• use aiwaf.flask.cli.AIWAFManager for CSV-backed operational tasks

FastAPI adapter:

• use aiwaf.fast.AIWAF for middleware registration
• use aiwaf fast ... or aiwaf-fast ... for CLI operations

Django Command Reference

Common management commands:

python manage.py detect_and_train
python manage.py regenerate_model
python manage.py aiwaf_reset --confirm
python manage.py aiwaf_reset --blacklist --confirm
python manage.py aiwaf_reset --keywords --confirm
python manage.py aiwaf_reset --exemptions --confirm
python manage.py add_ipexemption <ip> --reason "optional reason"
python manage.py add_pathexemption /path/prefix/ --reason "optio