cschantz/Linux-Server-Management-Toolkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
134 Commits 2 Branches 0 Tags
8bf1b96c2f9a6269a9e6901c46807429e0072c48
Commit Graph

2 Commits

Author SHA1 Message Date
cschantz 8bf1b96c2f Enhance CT_LIMIT optimizer with per-site intelligence - analyzes ALL sites 
USER REQUEST: "you have to confirm it will check for all of the sites?
as it effects them all"

PROBLEM: CT_LIMIT affects ALL sites on server, but optimizer only looked
at aggregate traffic, not individual site requirements

SOLUTION: Added comprehensive per-site analysis using sysref database

NEW CAPABILITIES:

1. AUTO-DISCOVERS ALL SITES
   - Reads sysref database (auto-generated at launcher startup)
   - Gets all domains, document roots, and log paths
   - Confirms: "Per-Site Analysis (All X Sites Checked)"

2. DETECTS SITE TYPE FOR EACH DOMAIN
   - WordPress (checks WP database entries)
   - Ecommerce (WooCommerce, Magento indicators)
   - Framework (Composer/vendor detection)
   - Dynamic (50+ PHP files)
   - Moderate (5-50 PHP files)
   - Static (minimal PHP)

3. CALCULATES SITE COMPLEXITY SCORE (1-10)
   Factors:
   - WordPress: +3 base + (plugins/5)
   - Ecommerce: +5 (shopping cart needs many connections)
   - Framework/Dynamic: +2
   - Ajax-heavy (20+ .js files): +2
   - Result: Higher score = needs more CT_LIMIT headroom

4. ANALYZES TRAFFIC PER DOMAIN
   - Max concurrent connections per site
   - Unique IPs per site
   - Total requests per site
   - Separated from aggregate analysis

5. FACTORS COMPLEXITY INTO RECOMMENDATIONS
   - Average complexity across all sites
   - Complexity buffer added to recommendations
   - Ecommerce sites get +15/+10 buffer
   - Formula: CT_LIMIT = max_legit + buffer + complexity_factor

6. DISPLAYS PER-SITE BREAKDOWN
   ```
   Per-Site Analysis (All 3 Sites Checked):
   DOMAIN                         TYPE         CMPLX  MAX_CONN  UNIQ_IPs
   ────────────────────────────────────────────────────────────────────
   example.com                    wordpress        7        45       128
   shop.example.com               ecommerce        9        82       245
   static.example.com             static           1         8        34

   ⚠️  2 high-complexity sites detected
      (WordPress/Ecommerce/Framework - need higher CT_LIMIT)
   ```

EXAMPLE RECOMMENDATION ADJUSTMENT:

BEFORE (no site analysis):
  - BALANCED: CT_LIMIT = 65

AFTER (with 2 WordPress sites, 1 ecommerce):
  - Average complexity: 7
  - Complexity buffer: 7 * 2 = 14
  - Ecommerce bonus: +10
  - BALANCED: CT_LIMIT = 89
  - Reason: "Accounts for WordPress admin/Ajax + ecommerce checkout"

INTELLIGENCE:

✅ Knows WordPress admin needs more connections
✅ Knows ecommerce checkout = simultaneous AJAX calls
✅ Knows static sites need minimal limits
✅ Knows Ajax-heavy sites (React/Vue) need headroom
✅ Accounts for plugin count (more plugins = more connections)

CONFIRMATION FOR USER:

Report clearly shows:
"Per-Site Analysis (All X Sites Checked)"

Where X = actual number of sites discovered from sysref database

SAFETY:

- If sysref.db doesn't exist, builds it automatically
- Skips aliases (only analyzes primary domains)
- Skips unknown/system domains
- Only analyzes sites with actual log files

FUNCTIONS ADDED:

- detect_site_type() - WordPress/ecommerce/framework detection
- calculate_site_complexity() - 1-10 score based on site needs
- analyze_per_site_traffic() - Per-domain traffic breakdown
- Enhanced generate_recommendation() - Factors in complexity

FILES MODIFIED:

- modules/security/optimize-ct-limit.sh
  - Added reference-db.sh sourcing (line 19)
  - Added detect_site_type() (lines 54-92)
  - Added calculate_site_complexity() (lines 94-136)
  - Added analyze_per_site_traffic() (lines 138-183)
  - Enhanced generate_recommendation() (lines 368-408, 449-465)
  - Added per-site analysis call in main() (line 625)

RESULT:

✅ Confirms ALL sites checked
✅ Tailors CT_LIMIT to actual site portfolio
✅ Prevents blocking legitimate WordPress/ecommerce traffic
✅ Shows exactly which sites drive the requirement

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-14 15:30:55 -05:00
cschantz b33c57086f Add intelligent CT_LIMIT optimizer - analyzes traffic to recommend optimal limit 
PROBLEM: Live monitor showed static CT_LIMIT="100" recommendation
- No analysis of actual site traffic
- No consideration of legitimate high-connection users
- Could block CDNs, bots, or legitimate traffic spikes
- No way to know what's safe for the specific server

SOLUTION: Created comprehensive CT_LIMIT optimizer script

NEW SCRIPT: modules/security/optimize-ct-limit.sh

WHAT IT DOES:
1. Analyzes Apache logs (last 24 hours by default)
   - Parses all domain logs in /var/log/apache2/domlogs/
   - Tracks max concurrent connections per IP per domain
   - Identifies user agents and behavior patterns

2. Classifies IP behavior using bot-signatures.sh
   - Legitimate bots (Googlebot, Bingbot, etc.)
   - AI crawlers (GPT, Claude, etc.)
   - CDNs (Cloudflare, Akamai, etc.)
   - Normal users vs high-traffic users
   - Potential scrapers

3. Analyzes current active connections
   - Uses ss or netstat to check real-time connections
   - Identifies current highest connection counts

4. Calculates statistics
   - 95th percentile of legitimate user connections
   - 99th percentile for headroom
   - Max concurrent from single legitimate IP
   - Separates bot/CDN traffic from user traffic

5. Provides 3 recommendations:
   a) CONSERVATIVE (max_legit + 20) - For high-traffic sites
   b) BALANCED (max_legit + 10) - Recommended for most ⭐
   c) AGGRESSIVE (max_legit + 5) - Only during active attack

6. Whitelist recommendations
   - Identifies bots/CDNs exceeding recommended limit
   - Suggests specific IPs to whitelist in CSF
   - Prevents blocking Googlebot, monitoring services, etc.

7. One-command application
   - Backs up csf.conf automatically
   - Updates CT_LIMIT to recommended value
   - Enables SYNFLOOD protection
   - Restarts CSF
   - Provides monitoring command

EXAMPLE OUTPUT:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Connection Analysis Summary:
  Total unique IPs analyzed: 1,247
  Legitimate users: 1,180
  Bots/CDNs/Crawlers: 67

Legitimate User Connection Patterns:
  Max concurrent from single IP: 45
  95th percentile: 12 concurrent connections
  99th percentile: 28 concurrent connections

Current Active Connections:
  Highest right now: 8 connections from 1.2.3.4

Current CSF Configuration:
  CT_LIMIT = 150

📊 RECOMMENDED CT_LIMIT VALUES
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

1. CONSERVATIVE: CT_LIMIT = 65
   • Allows headroom for traffic spikes
   • Won't block legitimate users

2. BALANCED: CT_LIMIT = 55 ⭐
   • Based on 99th percentile + buffer
   • Blocks most attack traffic

3. AGGRESSIVE: CT_LIMIT = 50
   • Maximum DDoS protection
   • May affect some legitimate users

⚠️  WHITELIST RECOMMENDATIONS
Found bots/crawlers with high connection counts:
  • 66.249.72.38   (Googlebot)         82 connections
  • 40.77.167.88   (Bingbot)           65 connections
  • 157.55.39.183  (UptimeRobot)       48 connections

To whitelist: csf -a <IP>
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

INTEGRATION WITH LIVE MONITOR:
- Press 'c' during live monitoring to run optimizer
- Recommendation updates based on detected DDoS/SYN floods
- Quick Actions panel shows: "Press 'c' to run CT_LIMIT optimizer"
- Help screen updated with 'c' key

USAGE:
1. Standalone: modules/security/optimize-ct-limit.sh
2. From live monitor: Press 'c' during monitoring
3. With custom period: optimize-ct-limit.sh 48  (48 hours)

SAFETY:
- Automatic backup of csf.conf before changes
- Minimum thresholds (50/80/100) prevent too-aggressive limits
- Option to apply or just view recommendations
- Full report saved to /tmp for review

INTELLIGENCE:
- Uses actual traffic data, not guesses
- Accounts for legitimate high-connection sources
- Prevents blocking search engines and monitoring
- Adapts to each server's unique traffic patterns

FILES MODIFIED:
- modules/security/optimize-ct-limit.sh (NEW - 650 lines)
- modules/security/live-attack-monitor.sh
  - Added 'c' key handler (line 1019-1024)
  - Updated Quick Actions recommendation (line 438)
  - Updated help screen (line 1045)
  - Updated footer keys (line 457)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-14 15:26:31 -05:00