cschantz/Linux-Server-Management-Toolkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
2ad6658f49092da802962d5cc8e0f8ae8529f154
Linux-Server-Management-Too…/tools/analyze-historical-attacks.sh
T
cschantz 2ad6658f49 Rewrite historical attack analyzer to show per-IP summaries 
Issue:
User reported: "it seems to just list all possible hits"
- Old format listed every individual attack hit
- No grouping or organization by IP
- Hard to understand what each IP actually did
- No reputation context

User Request:
"show an IP, saying what it did, saying how many times it did it,
and what its reputation is"

Solution:
Completely rewrote output format to group by IP with summaries:

New Output Format:
================================================================================
ATTACKING IPs - DETAILED BREAKDOWN
================================================================================

[1] 192.168.1.100
    Attacks: 15 | Avg Score: 87 | Threat Level: CRITICAL
    Attack Types: WEBSHELL(8), SQLI(5), XSS(2)
    Reputation: AbuseIPDB 85% confidence (142 reports) | China
    Sample Targets:
      - /wp-admin/alfa-rex.php
      - /admin.php?id=1' union select...
      - /upload.php?file=../../../../etc/passwd

[2] 45.83.66.23
    Attacks: 8 | Avg Score: 92 | Threat Level: CRITICAL
    Attack Types: CMD(5), TRAVERSAL(3)
    Sample Targets:
      - /cgi-bin/admin.cgi?cmd=cat%20/etc/passwd
      - /../../../etc/shadow

Changes Made:

1. Added IP-level tracking (lines 151-153):
   - IP_ATTACK_DETAILS: Store all attack types per IP
   - IP_ATTACK_COUNT: Count total attacks per IP
   - IP_SAMPLE_URLS: Store first 3 sample URLs per IP

2. Track data during scan (lines 240-260):
   - Aggregate attack types per IP
   - Keep sample URLs for context
   - Count occurrences of each attack type

3. New output section (lines 284-352):
   - Sort IPs by cumulative threat score (worst first)
   - Calculate average score per IP
   - Count attack type occurrences: "SQLI(5), XSS(2)"
   - Show reputation from AbuseIPDB (if available)
   - Display sample target URLs for context
   - Limit to top 50 attacking IPs

4. Improved summary stats (lines 360-381):
   - Added "Unique attacking IPs" count
   - Condensed attack type summary to top 10
   - Removed redundant "Top Signatures" section

5. Source IP reputation library (line 30):
   - Optional: loads get_threat_intelligence() if available
   - Gracefully skips reputation if not available

Benefits:
✅ Clean per-IP summary (not a flood of individual hits)
✅ Shows what each IP did and how many times
✅ Includes reputation context from AbuseIPDB
✅ Sample URLs provide attack pattern examples
✅ Sorted by threat level (worst attackers first)
✅ Much easier to understand and act on
2025-12-13 02:40:34 -05:00

409 lines
13 KiB
Bash
Executable File
Raw Blame History

#!/bin/bash
#
# Historical Attack Log Analyzer
# Scans past Apache/Nginx logs for attack patterns using ET Open signatures
#
# Usage: bash analyze-historical-attacks.sh [options]
#
# Options:
# -d DAYS Analyze logs from last N days (default: 7)
# -l LOGFILE Analyze specific log file
# -o OUTPUT Output report file (default: /tmp/attack-analysis-TIMESTAMP.txt)
# -t THRESHOLD Minimum threat score to report (default: 50)
# -v Verbose mode (show all attacks)
# -h Show help
# Get script directory
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/.."
# Source required libraries
source "$SCRIPT_DIR/lib/attack-signatures.sh" 2>/dev/null || {
echo "ERROR: attack-signatures.sh not found"
exit 1
}
source "$SCRIPT_DIR/lib/http-attack-analyzer.sh" 2>/dev/null || {
echo "ERROR: http-attack-analyzer.sh not found"
exit 1
}
# Try to source IP reputation library (optional)
source "$SCRIPT_DIR/lib/ip-reputation.sh" 2>/dev/null
# Colors
RED='\033[0;31m'
YELLOW='\033[1;33m'
GREEN='\033[0;32m'
BLUE='\033[0;34m'
CYAN='\033[0;36m'
BOLD='\033[1m'
NC='\033[0m'
# Default options
DAYS=7
LOG_FILE=""
OUTPUT_FILE="/tmp/attack-analysis-$(date +%Y%m%d_%H%M%S).txt"
THRESHOLD=50
VERBOSE=0
# Parse command line arguments
while getopts "d:l:o:t:vh" opt; do
case $opt in
d) DAYS="$OPTARG" ;;
l) LOG_FILE="$OPTARG" ;;
o) OUTPUT_FILE="$OPTARG" ;;
t) THRESHOLD="$OPTARG" ;;
v) VERBOSE=1 ;;
h)
cat << EOF
Historical Attack Log Analyzer
Scans past Apache/Nginx logs for attack patterns using ET Open signatures
Usage: $0 [options]
Options:
-d DAYS Analyze logs from last N days (default: 7)
-l LOGFILE Analyze specific log file
-o OUTPUT Output report file (default: /tmp/attack-analysis-TIMESTAMP.txt)
-t THRESHOLD Minimum threat score to report (default: 50)
-v Verbose mode (show all attacks)
-h Show this help
Examples:
# Analyze last 7 days
$0
# Analyze last 30 days
$0 -d 30
# Analyze specific log file
$0 -l /var/log/apache2/access.log
# Show all attacks (including low severity)
$0 -t 0 -v
# Save report to custom location
$0 -o /root/attack-report.txt
EOF
exit 0
;;
\?)
echo "Invalid option: -$OPTARG" >&2
exit 1
;;
esac
done
echo "================================================================================
"
echo -e "${CYAN}${BOLD}Historical Attack Log Analyzer${NC}"
echo "Powered by Emerging Threats Open Ruleset"
echo "================================================================================
"
# Find log files to analyze
LOG_FILES=()
if [ -n "$LOG_FILE" ]; then
# Specific log file provided
if [ ! -f "$LOG_FILE" ]; then
echo -e "${RED}ERROR: Log file not found: $LOG_FILE${NC}"
exit 1
fi
LOG_FILES=("$LOG_FILE")
echo -e "${GREEN}✓${NC} Analyzing specific file: $LOG_FILE"
else
# Auto-detect log files
echo -e "${BLUE}[*]${NC} Searching for Apache/Nginx log files..."
# Common log locations
SEARCH_PATHS=(
"/var/log/apache2"
"/var/log/httpd"
"/usr/local/apache/logs"
"/var/log/nginx"
"/usr/local/apache/domlogs"
)
for path in "${SEARCH_PATHS[@]}"; do
if [ -d "$path" ]; then
# Find access logs modified in last N days
while IFS= read -r log; do
LOG_FILES+=("$log")
done < <(find "$path" -type f \( -name "access*.log*" -o -name "access_log*" -o -name "*.com" -o -name "*.net" -o -name "*.org" \) -mtime -"$DAYS" 2>/dev/null)
fi
done
if [ ${#LOG_FILES[@]} -eq 0 ]; then
echo -e "${RED}ERROR: No log files found in last $DAYS days${NC}"
exit 1
fi
echo -e "${GREEN}✓${NC} Found ${#LOG_FILES[@]} log files from last $DAYS days"
fi
# Initialize counters
TOTAL_LINES=0
TOTAL_ATTACKS=0
CRITICAL_ATTACKS=0
HIGH_ATTACKS=0
MEDIUM_ATTACKS=0
declare -A ATTACK_TYPES
declare -A TOP_ATTACKERS
declare -A SIGNATURE_HITS
declare -A IP_ATTACK_DETAILS # Store detailed attack info per IP
declare -A IP_ATTACK_COUNT # Count attacks per IP
declare -A IP_SAMPLE_URLS # Sample URLs per IP
# Progress indicator
show_progress() {
count=$1
total=$2
percent=$((count * 100 / total))
echo -ne "\r${BLUE}[*]${NC} Processing: $count/$total lines ($percent%) "
}
# Start analysis
echo ""
echo -e "${BLUE}[*]${NC} Starting analysis (Threshold: $THRESHOLD)..."
echo ""
{
# Write report header
echo "================================================================================
"
echo "HISTORICAL ATTACK ANALYSIS REPORT"
echo "Generated: $(date)"
echo "Period: Last $DAYS days"
echo "Threshold: $THRESHOLD"
echo "================================================================================
"
echo ""
# Analyze each log file
for log_file in "${LOG_FILES[@]}"; do
echo "[*] Analyzing: $log_file"
# Handle compressed logs
if [[ "$log_file" =~ \.gz$ ]]; then
CAT_CMD="zcat"
elif [[ "$log_file" =~ \.bz2$ ]]; then
CAT_CMD="bzcat"
else
CAT_CMD="cat"
fi
file_attacks=0
line_count=0
while IFS= read -r line; do
line_count=$((line_count + 1))
TOTAL_LINES=$((TOTAL_LINES + 1))
# Show progress every 1000 lines
if [ $((line_count % 1000)) -eq 0 ]; then
show_progress "$TOTAL_LINES" "unknown"
fi
# Analyze line
result=$(analyze_http_log_line "$line" 2>/dev/null)
threat_score="${result%%||*}"
if [ "$threat_score" -ge "$THRESHOLD" ]; then
temp="${result#*||}"
attack_types="${temp%%||*}"
temp="${temp#*||}"
signatures="${temp%%||*}"
temp="${temp#*||}"
ip="${temp%%||*}"
uri="${temp#*||}"
# Count attacks
TOTAL_ATTACKS=$((TOTAL_ATTACKS + 1))
file_attacks=$((file_attacks + 1))
# Categorize by severity
if [ "$threat_score" -ge 85 ]; then
CRITICAL_ATTACKS=$((CRITICAL_ATTACKS + 1))
elif [ "$threat_score" -ge 70 ]; then
HIGH_ATTACKS=$((HIGH_ATTACKS + 1))
elif [ "$threat_score" -ge 50 ]; then
MEDIUM_ATTACKS=$((MEDIUM_ATTACKS + 1))
fi
# Track attack types
IFS=',' read -ra types <<< "$attack_types"
for type in "${types[@]}"; do
ATTACK_TYPES["$type"]=$((${ATTACK_TYPES[$type]:-0} + 1))
done
# Track top attackers (cumulative score)
TOP_ATTACKERS["$ip"]=$((${TOP_ATTACKERS[$ip]:-0} + threat_score))
# Track attack count per IP
IP_ATTACK_COUNT["$ip"]=$((${IP_ATTACK_COUNT[$ip]:-0} + 1))
# Store attack type details per IP
current_types="${IP_ATTACK_DETAILS[$ip]}"
if [ -z "$current_types" ]; then
IP_ATTACK_DETAILS["$ip"]="$attack_types"
else
IP_ATTACK_DETAILS["$ip"]="$current_types,$attack_types"
fi
# Store sample URL (keep first 3)
current_urls="${IP_SAMPLE_URLS[$ip]}"
url_count=$(echo "$current_urls" | grep -o "||" | wc -l)
if [ "$url_count" -lt 3 ]; then
if [ -z "$current_urls" ]; then
IP_SAMPLE_URLS["$ip"]="${uri:0:100}"
else
IP_SAMPLE_URLS["$ip"]="$current_urls||${uri:0:100}"
fi
fi
# Track signatures
IFS=',' read -ra sigs <<< "$signatures"
for sig in "${sigs[@]}"; do
SIGNATURE_HITS["$sig"]=$((${SIGNATURE_HITS[$sig]:-0} + 1))
done
fi
done < <($CAT_CMD "$log_file" 2>/dev/null)
echo " → Found $file_attacks attacks" >> "$OUTPUT_FILE"
done
echo "" >> "$OUTPUT_FILE"
echo "================================================================================
" >> "$OUTPUT_FILE"
echo "ATTACKING IPs - DETAILED BREAKDOWN" >> "$OUTPUT_FILE"
echo "================================================================================
" >> "$OUTPUT_FILE"
echo "" >> "$OUTPUT_FILE"
# Sort IPs by cumulative threat score and display
# Create sorted list first to avoid subshell issues
sorted_ips=$(for ip in "${!TOP_ATTACKERS[@]}"; do
echo "${TOP_ATTACKERS[$ip]}:$ip"
done | sort -t: -k1 -nr | head -50)
ip_count=0
while IFS=: read -r cumulative_score ip; do
ip_count=$((ip_count + 1))
attack_count="${IP_ATTACK_COUNT[$ip]:-0}"
all_attack_types="${IP_ATTACK_DETAILS[$ip]}"
sample_urls="${IP_SAMPLE_URLS[$ip]}"
# Count occurrences of each attack type
declare -A type_counts
IFS=',' read -ra attacks <<< "$all_attack_types"
for attack in "${attacks[@]}"; do
[ -n "$attack" ] && type_counts["$attack"]=$((${type_counts[$attack]:-0} + 1))
done
# Format attack summary
attack_summary=""
for type in "${!type_counts[@]}"; do
if [ -z "$attack_summary" ]; then
attack_summary="$type(${type_counts[$type]})"
else
attack_summary="$attack_summary, $type(${type_counts[$type]})"
fi
done
unset type_counts
# Determine threat level
avg_score=$((cumulative_score / attack_count))
if [ "$avg_score" -ge 85 ]; then
level="CRITICAL"
elif [ "$avg_score" -ge 70 ]; then
level="HIGH"
else
level="MEDIUM"
fi
# Print IP summary
echo "[$ip_count] $ip" >> "$OUTPUT_FILE"
printf " Attacks: %d | Avg Score: %d | Threat Level: %s\n" "$attack_count" "$avg_score" "$level" >> "$OUTPUT_FILE"
echo " Attack Types: $attack_summary" >> "$OUTPUT_FILE"
# Get reputation (if available)
if type get_threat_intelligence &>/dev/null; then
threat_intel=$(get_threat_intelligence "$ip" 2>/dev/null)
if [ -n "$threat_intel" ]; then
IFS='|' read -r abuse_conf abuse_rpts country isp geo timing whitelisted <<< "$threat_intel"
if [ "${abuse_conf:-0}" -gt 0 ]; then
printf " Reputation: AbuseIPDB %d%% confidence (%d reports) | %s\n" "${abuse_conf:-0}" "${abuse_rpts:-0}" "${country:-Unknown}" >> "$OUTPUT_FILE"
fi
fi
fi
# Show sample URLs
if [ -n "$sample_urls" ]; then
echo " Sample Targets:" >> "$OUTPUT_FILE"
IFS='||' read -ra urls <<< "$sample_urls"
for url in "${urls[@]}"; do
echo " - $url" >> "$OUTPUT_FILE"
done
fi
echo "" >> "$OUTPUT_FILE"
done <<< "$sorted_ips"
echo "================================================================================
" >> "$OUTPUT_FILE"
echo "SUMMARY STATISTICS" >> "$OUTPUT_FILE"
echo "================================================================================
" >> "$OUTPUT_FILE"
echo "" >> "$OUTPUT_FILE"
echo "Total lines processed: $TOTAL_LINES" >> "$OUTPUT_FILE"
echo "Total attacks detected: $TOTAL_ATTACKS" >> "$OUTPUT_FILE"
echo "Unique attacking IPs: ${#TOP_ATTACKERS[@]}" >> "$OUTPUT_FILE"
echo "" >> "$OUTPUT_FILE"
echo "Attack Severity:" >> "$OUTPUT_FILE"
echo " - Critical (≥85): $CRITICAL_ATTACKS" >> "$OUTPUT_FILE"
echo " - High (70-84): $HIGH_ATTACKS" >> "$OUTPUT_FILE"
echo " - Medium (50-69): $MEDIUM_ATTACKS" >> "$OUTPUT_FILE"
echo "" >> "$OUTPUT_FILE"
# Top Attack Types
echo "Top Attack Types:" >> "$OUTPUT_FILE"
for type in "${!ATTACK_TYPES[@]}"; do
echo "$type:${ATTACK_TYPES[$type]}"
done | sort -t: -k2 -nr | head -10 | while IFS=: read -r type count; do
printf " %-20s %5d attacks\n" "$type" "$count" >> "$OUTPUT_FILE"
done
echo "" >> "$OUTPUT_FILE"
echo "================================================================================
" >> "$OUTPUT_FILE"
echo "END OF REPORT" >> "$OUTPUT_FILE"
echo "================================================================================
" >> "$OUTPUT_FILE"
} > "$OUTPUT_FILE"
# Clear progress line
echo -ne "\r\033[K"
# Display summary to terminal
echo ""
echo -e "${GREEN}✓${NC} Analysis complete!"
echo ""
echo "Summary:"
echo " Lines processed: $TOTAL_LINES"
echo " Attacks detected: $TOTAL_ATTACKS"
echo " - Critical (≥85): $CRITICAL_ATTACKS"
echo " - High (70-84): $HIGH_ATTACKS"
echo " - Medium (50-69): $MEDIUM_ATTACKS"
echo ""
echo -e "${GREEN}✓${NC} Full report saved to: $OUTPUT_FILE"
echo ""
# Offer to view report
read -p "View report now? [y/N]: " view_report
if [[ "$view_report" =~ ^[Yy]$ ]]; then
less "$OUTPUT_FILE"
fi
Reference in New Issue View Git Blame Copy Permalink