Require repeated evidence before blocking
Escalate an IP only after it continues to show abusive behavior while already challenged; pause enforcement when log coverage is incomplete.
Back to projectsProduction since October 2026
Backend Engineering / Security / Observability
Abuse Detector
Automated scraper and bot detection for KSP's e-commerce site, with temporary Cloudflare enforcement, an operator dashboard, and an AI-assisted review loop.
Production detection workflow
From request logs to temporary enforcement
Incremental traffic analysis combines behavioral evidence, safety gates, and operator feedback before changing Cloudflare lists.
Aggregate request evidence
Read Cloudflare request logs every 15 minutes and retain hourly per-IP aggregates.
- Loki / LogQL
- Python 3.12
Score and review
Combine behavioral signals and cross-IP fingerprints with log-health checks and local-LLM review.
- SQLite / Postgres
- Local LLM
Enforce and audit
Maintain temporary challenge and block lists with operator overrides and emergency release controls.
- Cloudflare Lists and Rules APIs
- HTTPS dashboard
The challenge
High-volume automated catalog reads created abusive traffic without corresponding cart, checkout, or purchase activity.
The approach
Built a Python service that incrementally aggregates Cloudflare request logs from Loki into hourly per-IP state, scores traffic behavior and fingerprints, and manages temporary challenge and block lists with escalation based on repeated evidence.
My role
Designed and built the detection service, Cloudflare integration, operator dashboard, and local-LLM review and evaluation workflow.
Engineering decisions
Keep operator decisions auditable
Record decisions and allow overrides, retain emergency release controls, and replay answered AI cases when evaluating a prompt or model.
Scope & delivery
- Incremental hourly aggregates avoid repeatedly scanning a full day of request logs.
- Behavioral scoring and cross-IP scraper signatures identify automated catalog extraction and rotating address pools.
- Health-gated enforcement pauses when log coverage is incomplete; repeated evidence is required before escalation from challenge to block.
- Audited allow overrides, network exemptions, a kill switch, and emergency release controls support safe operations.
- HTTPS operator dashboard shows candidates, explanations, and effect reports, with logged review actions.
- Local-LLM review uses operator feedback as memory and tracks accuracy, with replayable evaluations for prompt and model changes.