5654392b8c1813d2646b2453036aa669f674af3e
2 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 cschantz
|
5654392b8c |
Enhance CT_LIMIT optimizer with per-site intelligence - analyzes ALL sites
USER REQUEST: "you have to confirm it will check for all of the sites? as it effects them all" PROBLEM: CT_LIMIT affects ALL sites on server, but optimizer only looked at aggregate traffic, not individual site requirements SOLUTION: Added comprehensive per-site analysis using sysref database NEW CAPABILITIES: 1. AUTO-DISCOVERS ALL SITES - Reads sysref database (auto-generated at launcher startup) - Gets all domains, document roots, and log paths - Confirms: "Per-Site Analysis (All X Sites Checked)" 2. DETECTS SITE TYPE FOR EACH DOMAIN - WordPress (checks WP database entries) - Ecommerce (WooCommerce, Magento indicators) - Framework (Composer/vendor detection) - Dynamic (50+ PHP files) - Moderate (5-50 PHP files) - Static (minimal PHP) 3. CALCULATES SITE COMPLEXITY SCORE (1-10) Factors: - WordPress: +3 base + (plugins/5) - Ecommerce: +5 (shopping cart needs many connections) - Framework/Dynamic: +2 - Ajax-heavy (20+ .js files): +2 - Result: Higher score = needs more CT_LIMIT headroom 4. ANALYZES TRAFFIC PER DOMAIN - Max concurrent connections per site - Unique IPs per site - Total requests per site - Separated from aggregate analysis 5. FACTORS COMPLEXITY INTO RECOMMENDATIONS - Average complexity across all sites - Complexity buffer added to recommendations - Ecommerce sites get +15/+10 buffer - Formula: CT_LIMIT = max_legit + buffer + complexity_factor 6. DISPLAYS PER-SITE BREAKDOWN ``` Per-Site Analysis (All 3 Sites Checked): DOMAIN TYPE CMPLX MAX_CONN UNIQ_IPs ──────────────────────────────────────────────────────────────────── example.com wordpress 7 45 128 shop.example.com ecommerce 9 82 245 static.example.com static 1 8 34 ⚠️ 2 high-complexity sites detected (WordPress/Ecommerce/Framework - need higher CT_LIMIT) ``` EXAMPLE RECOMMENDATION ADJUSTMENT: BEFORE (no site analysis): - BALANCED: CT_LIMIT = 65 AFTER (with 2 WordPress sites, 1 ecommerce): - Average complexity: 7 - Complexity buffer: 7 * 2 = 14 - Ecommerce bonus: +10 - BALANCED: CT_LIMIT = 89 - Reason: "Accounts for WordPress admin/Ajax + ecommerce checkout" INTELLIGENCE: ✅ Knows WordPress admin needs more connections ✅ Knows ecommerce checkout = simultaneous AJAX calls ✅ Knows static sites need minimal limits ✅ Knows Ajax-heavy sites (React/Vue) need headroom ✅ Accounts for plugin count (more plugins = more connections) CONFIRMATION FOR USER: Report clearly shows: "Per-Site Analysis (All X Sites Checked)" Where X = actual number of sites discovered from sysref database SAFETY: - If sysref.db doesn't exist, builds it automatically - Skips aliases (only analyzes primary domains) - Skips unknown/system domains - Only analyzes sites with actual log files FUNCTIONS ADDED: - detect_site_type() - WordPress/ecommerce/framework detection - calculate_site_complexity() - 1-10 score based on site needs - analyze_per_site_traffic() - Per-domain traffic breakdown - Enhanced generate_recommendation() - Factors in complexity FILES MODIFIED: - modules/security/optimize-ct-limit.sh - Added reference-db.sh sourcing (line 19) - Added detect_site_type() (lines 54-92) - Added calculate_site_complexity() (lines 94-136) - Added analyze_per_site_traffic() (lines 138-183) - Enhanced generate_recommendation() (lines 368-408, 449-465) - Added per-site analysis call in main() (line 625) RESULT: ✅ Confirms ALL sites checked ✅ Tailors CT_LIMIT to actual site portfolio ✅ Prevents blocking legitimate WordPress/ecommerce traffic ✅ Shows exactly which sites drive the requirement |
||
|
cschantz
|
dbb4322cd2 |
Add intelligent CT_LIMIT optimizer - analyzes traffic to recommend optimal limit
PROBLEM: Live monitor showed static CT_LIMIT="100" recommendation - No analysis of actual site traffic - No consideration of legitimate high-connection users - Could block CDNs, bots, or legitimate traffic spikes - No way to know what's safe for the specific server SOLUTION: Created comprehensive CT_LIMIT optimizer script NEW SCRIPT: modules/security/optimize-ct-limit.sh WHAT IT DOES: 1. Analyzes Apache logs (last 24 hours by default) - Parses all domain logs in /var/log/apache2/domlogs/ - Tracks max concurrent connections per IP per domain - Identifies user agents and behavior patterns 2. Classifies IP behavior using bot-signatures.sh - Legitimate bots (Googlebot, Bingbot, etc.) - AI crawlers (GPT, Claude, etc.) - CDNs (Cloudflare, Akamai, etc.) - Normal users vs high-traffic users - Potential scrapers 3. Analyzes current active connections - Uses ss or netstat to check real-time connections - Identifies current highest connection counts 4. Calculates statistics - 95th percentile of legitimate user connections - 99th percentile for headroom - Max concurrent from single legitimate IP - Separates bot/CDN traffic from user traffic 5. Provides 3 recommendations: a) CONSERVATIVE (max_legit + 20) - For high-traffic sites b) BALANCED (max_legit + 10) - Recommended for most ⭐ c) AGGRESSIVE (max_legit + 5) - Only during active attack 6. Whitelist recommendations - Identifies bots/CDNs exceeding recommended limit - Suggests specific IPs to whitelist in CSF - Prevents blocking Googlebot, monitoring services, etc. 7. One-command application - Backs up csf.conf automatically - Updates CT_LIMIT to recommended value - Enables SYNFLOOD protection - Restarts CSF - Provides monitoring command EXAMPLE OUTPUT: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Connection Analysis Summary: Total unique IPs analyzed: 1,247 Legitimate users: 1,180 Bots/CDNs/Crawlers: 67 Legitimate User Connection Patterns: Max concurrent from single IP: 45 95th percentile: 12 concurrent connections 99th percentile: 28 concurrent connections Current Active Connections: Highest right now: 8 connections from 1.2.3.4 Current CSF Configuration: CT_LIMIT = 150 📊 RECOMMENDED CT_LIMIT VALUES ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1. CONSERVATIVE: CT_LIMIT = 65 • Allows headroom for traffic spikes • Won't block legitimate users 2. BALANCED: CT_LIMIT = 55 ⭐ • Based on 99th percentile + buffer • Blocks most attack traffic 3. AGGRESSIVE: CT_LIMIT = 50 • Maximum DDoS protection • May affect some legitimate users ⚠️ WHITELIST RECOMMENDATIONS Found bots/crawlers with high connection counts: • 66.249.72.38 (Googlebot) 82 connections • 40.77.167.88 (Bingbot) 65 connections • 157.55.39.183 (UptimeRobot) 48 connections To whitelist: csf -a <IP> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ INTEGRATION WITH LIVE MONITOR: - Press 'c' during live monitoring to run optimizer - Recommendation updates based on detected DDoS/SYN floods - Quick Actions panel shows: "Press 'c' to run CT_LIMIT optimizer" - Help screen updated with 'c' key USAGE: 1. Standalone: modules/security/optimize-ct-limit.sh 2. From live monitor: Press 'c' during monitoring 3. With custom period: optimize-ct-limit.sh 48 (48 hours) SAFETY: - Automatic backup of csf.conf before changes - Minimum thresholds (50/80/100) prevent too-aggressive limits - Option to apply or just view recommendations - Full report saved to /tmp for review INTELLIGENCE: - Uses actual traffic data, not guesses - Accounts for legitimate high-connection sources - Prevents blocking search engines and monitoring - Adapts to each server's unique traffic patterns FILES MODIFIED: - modules/security/optimize-ct-limit.sh (NEW - 650 lines) - modules/security/live-attack-monitor.sh - Added 'c' key handler (line 1019-1024) - Updated Quick Actions recommendation (line 438) - Updated help screen (line 1045) - Updated footer keys (line 457) |