Add centralized IP reputation tracking system

Created a comprehensive IP reputation system that tracks IPs across all
toolkit scripts with tags/attack types, scores, and detailed analytics.

NEW FILES:
- lib/ip-reputation.sh: Core reputation library with optimized database
  * Fast lookup using pipe-delimited file format
  * Attack type tagging system (bitmask: SQL, XSS, RCE, Bot, Scanner, etc.)
  * Reputation scoring (0-100) based on hits and attack severity
  * GeoIP country lookup integration
  * Automatic cleanup of old entries
  * Thread-safe with file locking

- modules/security/ip-reputation-manager.sh: Interactive management tool
  * Query individual IPs with full details
  * View top malicious/active IPs
  * Database statistics and analytics
  * Manual IP flagging/whitelisting
  * Import IPs from logs
  * Export to readable reports
  * Live monitoring mode

INTEGRATION:
All security and analysis scripts now use the centralized reputation system:

- modules/website/500-error-tracker.sh:
  * Tracks IPs generating 500 errors
  * Tags bots/scanners with BOT/SCANNER flags
  * Background processing for performance

- modules/security/live-attack-monitor.sh:
  * Maps attack types to reputation flags
  * Tracks SSH bruteforce, SQL injection, XSS, DDoS, etc.
  * Real-time reputation updates

- modules/website/website-error-analyzer.sh:
  * Tags filtered bots in error analysis
  * Builds IP reputation from website errors

- launcher.sh:
  * Added IP Reputation Manager to Bot & Traffic Analysis menu
  * Menu option 4 in Security > Analysis > Bot & Traffic Analysis

KEY FEATURES:
✓ Centralized IP tracking across ALL scripts
✓ Multi-tag system (IP can have multiple attack types)
✓ Reputation scores increase with more tags/attacks
✓ Country tracking via GeoIP
✓ Optimized for high-volume traffic (attacks with 1000s of IPs)
✓ Fast lookups even during DDoS
✓ Background processing doesn't slow down analysis
✓ Database cleanup/maintenance tools
✓ Export for reports and sharing

BENEFITS:
- Single source of truth for IP reputation
- Scripts share intelligence (bot detected in one script = flagged for all)
- Track IPs across time and multiple attack vectors
- Identify repeat offenders with multiple attack types
- Make blocking decisions based on comprehensive data
- Performance optimized with file locking and background updates
This commit is contained in:
cschantz
2025-11-05 18:45:55 -05:00
parent f128219f05
commit 4ff98033f2
6 changed files with 969 additions and 9 deletions
+14 -1
View File
@@ -9,6 +9,7 @@
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
source "$SCRIPT_DIR/lib/common-functions.sh"
source "$SCRIPT_DIR/lib/ip-reputation.sh"
# Ensure color variables are set
DIM='\033[2m'
@@ -111,6 +112,8 @@ while IFS= read -r log; do
line_lower="${line,,}"
if [[ "$line_lower" =~ (bot|crawler|spider|scanner|monitor|cloud|amazon|google|microsoft|azure) ]]; then
((filtered_bots++))
# Track bot IP with BOT flag
flag_ip_attack "$ip" "BOT" 0 "500 error - filtered as bot/scanner" >/dev/null 2>&1 &
continue
fi
fi
@@ -119,14 +122,20 @@ while IFS= read -r log; do
line_lower="${line,,}"
if [[ "$line_lower" =~ (bot|crawler|spider|scraper|scanner|check|monitor|uptime|pingdom|newrelic|datadog|nagios|zabbix|prtg|gomez|keynote|catchpoint|dotcom-monitor|site24x7|uptimerobot|statuscake|nodequery|hetrixtools|freshping|uptrendscom|siteuptime|montastic|updown\.io|apex|alertsite|webmon|wormly) ]]; then
((filtered_bots++))
# Track monitoring/uptime bot
flag_ip_attack "$ip" "BOT" 0 "Monitoring/uptime bot" >/dev/null 2>&1 &
continue
fi
if [[ "$line_lower" =~ (semrush|ahrefs|moz|majestic|serpstat|screaming|screamingfrog|sitebulb|linkchecker|validator|scanner|security|acunetix|nessus|openvas|burp|nikto|skipfish|w3af|sqlmap|metasploit|nmap|masscan|zmap|shodan|censys|binaryedge) ]]; then
((filtered_bots++))
# Track scanner with higher score
flag_ip_attack "$ip" "SCANNER" 0 "Security scanner detected" >/dev/null 2>&1 &
continue
fi
if [[ "$line_lower" =~ (curl|wget|python|perl|ruby|java|go-http|libwww|axios|node-fetch|http\.client|httpie|postman|insomnia|apachehttp|okhttp|httpclient) ]]; then
((filtered_bots++))
# Track programmatic access
flag_ip_attack "$ip" "BOT" 0 "HTTP library/tool" >/dev/null 2>&1 &
continue
fi
@@ -146,7 +155,11 @@ while IFS= read -r log; do
((domain_count["$domain"]++))
((total_500s++))
# Track IP in reputation database (500 error = slight increase in score)
# Most 500s are due to server issues, not attacks, so low score increase
increment_ip_hits "$ip" 1 >/dev/null 2>&1 &
# Save for analysis
echo "$domain|$user|$status|$url|$timestamp|$ip" >> "$ERRORS_500"
fi