Add comprehensive PHP & Server Optimizer planning document

FEATURE PLANNING: PHP-FPM and server-wide optimization system OVERVIEW: Intelligent analyzer that scans all domains, detects PHP configs, analyzes usage patterns, and provides one-click optimization with automatic backups and safety checks. LEVERAGES EXISTING INFRASTRUCTURE: - user-manager.sh: Domain/user detection (70% of work done) - system-detect.sh: Control panel detection - optimize-ct-limit.sh: Traffic analysis model - get_user_log_files(): Log location mapping CORE CAPABILITIES: 1. Detect all PHP-FPM pool configs per domain 2. Find php.ini hierarchy (.user.ini, local, global) 3. Analyze memory usage, traffic patterns, error logs 4. Calculate optimal pm.max_children, memory_limit, opcache 5. Detect issues: max_children reached, memory exhausted, slow requests 6. Provide actionable recommendations with safety checks 7. One-click apply with automatic backups IMPLEMENTATION PHASES: - Phase 1: lib/php-detector.sh (detection logic) - Phase 2: lib/php-analyzer.sh (analysis engine) - Phase 3: modules/performance/php-optimizer.sh (main script) - Phase 4: Integration with live-attack-monitor TRACKED METRICS: - pm.max_children, pm.start_servers, pm.min/max_spare_servers - memory_limit, max_execution_time, upload_max_filesize - opcache settings, hit rates, memory consumption - Process counts, memory usage, CPU patterns - Error rates, slow request logs NEXT: Expand metrics tracking and begin Phase 1 implementation
2025-12-02 19:34:04 -05:00
parent e76a00f591
commit 56f84a6db4
1 changed files with 429 additions and 0 deletions
@@ -0,0 +1,429 @@
+# PHP & Server Optimizer - Comprehensive Planning Document
+
+## Overview
+Intelligent PHP-FPM, memory, and resource optimizer that analyzes per-domain usage patterns and provides actionable recommendations with one-click fixes.
+
+## What We Already Have (Foundation)
+✅ **user-manager.sh** - Complete user/domain detection for cPanel, Plesk, InterWorx
+✅ **system-detect.sh** - Control panel, PHP version, web server detection
+✅ **optimize-ct-limit.sh** - Traffic pattern analysis model (can reuse approach)
+✅ **Domain home directories already tracked** via get_user_info()
+✅ **Log file detection** via get_user_log_files()
+
+## Architecture
+
+### Module Name
+`/root/server-toolkit/modules/performance/php-optimizer.sh`
+
+### Core Components
+
+#### 1. **Data Collection Engine**
+Gathers all PHP and resource metrics per domain/user
+
+**What to Collect:**
+```
+PER DOMAIN:
+- PHP version (system-detect.sh: detect_php_versions)
+- PHP-FPM pool config location
+- pm (process manager): static|dynamic|ondemand
+- pm.max_children (current value)
+- pm.start_servers
+- pm.min_spare_servers
+- pm.max_spare_servers
+- pm.max_requests
+- memory_limit (php.ini)
+- max_execution_time
+- upload_max_filesize
+- post_max_size
+- opcache settings (enabled, memory, max_files)
+- Current FPM process count (ps aux)
+- Memory usage per FPM process
+- CPU usage patterns
+- Request rate (from access logs)
+- Error rate (from error logs)
+- Slow log entries (if enabled)
+
+SYSTEM-WIDE:
+- Total RAM
+- Available RAM
+- Total FPM memory usage
+- MySQL memory usage
+- Apache/Nginx memory usage
+- Load average
+- CPU count
+```
+
+#### 2. **Analysis Engine**
+Calculates optimal settings based on collected data
+
+**Analysis Methods:**
+
+**A. Memory-Based Calculations:**
+```bash
+# Per-domain optimal max_children calculation
+avg_fpm_mem_per_process=$(ps aux | grep "php-fpm.*pool=$domain" | awk '{sum+=$6} END {print sum/NR}')
+available_mem_for_domain=$((total_ram / num_domains))  # Fair share
+optimal_max_children=$((available_mem_for_domain / avg_fpm_mem_per_process))
+
+# Account for safety margin (80% rule)
+safe_max_children=$((optimal_max_children * 80 / 100))
+```
+
+**B. Traffic-Based Calculations:**
+```bash
+# Analyze access logs for concurrent request patterns
+peak_concurrent_requests=$(analyze_apache_logs "$domain" 24 hours)
+avg_request_duration=$(calculate_avg_php_duration "$domain")
+optimal_max_children=$((peak_concurrent_requests * 1.5))  # 50% buffer
+```
+
+**C. Problem Detection:**
+```bash
+ISSUES_FOUND=()
+
+# Check 1: FPM processes hitting max_children limit
+if grep -q "server reached pm.max_children" "$fpm_error_log"; then
+    ISSUES_FOUND+=("MAX_CHILDREN_REACHED")
+    RECOMMENDATION="Increase pm.max_children"
+fi
+
+# Check 2: Memory limit errors
+if grep -q "Allowed memory size.*exhausted" "$php_error_log"; then
+    ISSUES_FOUND+=("MEMORY_EXHAUSTED")
+    RECOMMENDATION="Increase memory_limit"
+fi
+
+# Check 3: Slow requests
+if [ -f "$slow_log" ]; then
+    slow_count=$(wc -l < "$slow_log")
+    if [ "$slow_count" -gt 100 ]; then
+        ISSUES_FOUND+=("SLOW_REQUESTS")
+        RECOMMENDATION="Optimize PHP code or increase max_execution_time"
+    fi
+fi
+
+# Check 4: Opcache hit rate
+opcache_hit_rate=$(php -r "print_r(opcache_get_status());" | grep hit_rate | awk '{print $2}')
+if [ "$opcache_hit_rate" -lt 80 ]; then
+    ISSUES_FOUND+=("LOW_OPCACHE_HIT_RATE")
+    RECOMMENDATION="Increase opcache.memory_consumption"
+fi
+```
+
+#### 3. **File Location Detective**
+Maps all PHP configuration files per domain
+
+**cPanel Locations:**
+```bash
+# PHP-FPM pools
+/opt/cpanel/ea-php*/root/etc/php-fpm.d/$username.conf
+/var/cpanel/userdata/$username/$domain
+
+# PHP.ini locations
+/opt/cpanel/ea-php*/root/etc/php.d/
+~/.php/
+/home/$username/.php/
+/home/$username/public_html/.user.ini
+```
+
+**Plesk Locations:**
+```bash
+# PHP-FPM pools
+/etc/php-fpm.d/plesk-php*-fpm/$domain.conf
+
+# PHP.ini
+/var/www/vhosts/system/$domain/etc/php.ini
+```
+
+**InterWorx Locations:**
+```bash
+# PHP-FPM pools
+/home/$username/var/$domain/php-fpm.conf
+
+# PHP.ini
+/home/$username/var/$domain/etc/php.ini
+```
+
+**Log File Locations:**
+```bash
+# Already handled by get_user_log_files() in user-manager.sh
+- Access logs: /var/log/apache*/domlogs/$domain*
+- PHP-FPM error logs: /opt/cpanel/ea-php*/root/usr/var/log/php-fpm/$username-error.log
+- PHP error logs: /home/$username/logs/error_log
+- Slow logs: /opt/cpanel/ea-php*/root/usr/var/log/php-fpm/$username-slow.log
+```
+
+#### 4. **Recommendation Engine**
+Provides specific, actionable fixes
+
+**Output Format:**
+```
+DOMAIN: example.com (user: examplec, PHP 8.2)
+STATUS: ⚠️  NEEDS OPTIMIZATION
+
+CURRENT CONFIGURATION:
+  ├─ pm.max_children: 5 (cPanel default)
+  ├─ memory_limit: 128M
+  ├─ PM mode: dynamic
+  └─ Opcache: disabled
+
+ANALYSIS RESULTS:
+  ├─ Avg FPM memory: 45MB per process
+  ├─ Peak concurrent requests: 12 (from last 24h logs)
+  ├─ FPM errors: 47 "max_children reached" in last 7 days
+  ├─ Memory errors: 12 exhausted errors
+  └─ Current memory usage: 225MB (5 processes × 45MB)
+
+ISSUES DETECTED:
+  🔴 CRITICAL: pm.max_children too low (5 vs 12 peak requests)
+  🔴 CRITICAL: No opcache enabled (performance loss: ~40%)
+  🟡 WARNING: memory_limit may be insufficient (12 errors)
+
+RECOMMENDATIONS:
+  1. Increase pm.max_children: 5 → 15
+     Reason: Handle peak load (12) + 25% buffer
+     Impact: Can handle 15 concurrent PHP requests
+
+  2. Enable opcache with optimal settings
+     Reason: Massive performance gain, reduce CPU by 40%
+     Settings:
+       opcache.enable=1
+       opcache.memory_consumption=128
+       opcache.max_accelerated_files=10000
+
+  3. Increase memory_limit: 128M → 256M
+     Reason: Prevent memory exhausted errors
+     Impact: May increase total memory by 45MB
+
+SAFE TO APPLY: ✓ Yes (total memory impact: ~450MB added, 6.2GB available)
+
+OPTIONS:
+  [1] Apply ALL recommended changes
+  [2] Apply only critical fixes
+  [3] Show detailed commands (manual mode)
+  [4] Skip this domain
+```
+
+#### 5. **Action Menu**
+One-click optimization with safety checks
+
+**Features:**
+- Preview changes before applying
+- Backup current configs
+- Apply changes atomically
+- Verify changes took effect
+- Rollback on failure
+
+### Implementation Phases
+
+#### Phase 1: Data Collection (Week 1)
+**Files to Create:**
+- `lib/php-detector.sh` - Detect all PHP configs per domain
+- `lib/php-analyzer.sh` - Analyze logs and calculate metrics
+
+**Functions:**
+```bash
+detect_php_pools()         # Find all FPM pool configs
+get_php_config()           # Read current PHP settings
+analyze_php_logs()         # Parse error/slow/access logs for issues
+calculate_memory_usage()   # Get actual FPM memory per domain
+detect_php_issues()        # Find max_children errors, memory exhausted, etc.
+```
+
+#### Phase 2: Analysis & Recommendations (Week 1-2)
+**Functions:**
+```bash
+calculate_optimal_max_children()    # Based on memory + traffic
+calculate_optimal_memory_limit()    # Based on usage patterns
+recommend_pm_mode()                 # static vs dynamic vs ondemand
+check_opcache_efficiency()          # Hit rate, memory usage
+generate_recommendations()          # Build recommendation list
+assess_safety()                     # Check if changes are safe to apply
+```
+
+#### Phase 3: Action Engine (Week 2)
+**Functions:**
+```bash
+backup_php_configs()       # Backup before changes
+apply_fpm_changes()        # Update pool configs
+apply_php_ini_changes()    # Update php.ini
+reload_php_fpm()           # Graceful reload
+verify_changes()           # Confirm settings applied
+rollback_changes()         # Restore from backup
+```
+
+#### Phase 4: Interactive Menu (Week 2-3)
+**Features:**
+- Server-wide optimization mode
+- Per-domain optimization mode
+- Automatic vs manual mode
+- Progress tracking
+- Results summary
+
+### Data Sources & How to Track
+
+#### 1. **Domain Discovery**
+```bash
+# Already have this!
+source /root/server-toolkit/lib/user-manager.sh
+users=$(list_all_users)
+for user in $users; do
+    domains=$(get_user_domains "$user")
+    for domain in $domains; do
+        # Process each domain
+    done
+done
+```
+
+#### 2. **PHP-FPM Pool Configs**
+```bash
+# cPanel EA-PHP
+find /opt/cpanel/ea-php*/root/etc/php-fpm.d/ -name "*.conf" -type f
+
+# Plesk
+find /etc/php-fpm.d/ -name "*.conf" -type f 2>/dev/null
+
+# InterWorx
+find /home/*/var/*/php-fpm.conf -type f 2>/dev/null
+```
+
+#### 3. **PHP Error Logs**
+```bash
+# Use existing function!
+error_logs=$(get_user_log_files "$user" "error")
+```
+
+#### 4. **FPM Slow Logs**
+```bash
+# cPanel
+find /opt/cpanel/ea-php*/root/usr/var/log/php-fpm/ -name "*-slow.log"
+```
+
+#### 5. **Current FPM Processes**
+```bash
+# Get live process count per pool
+ps aux | grep "php-fpm: pool $domain" | grep -v grep | wc -l
+
+# Get memory usage
+ps aux | grep "php-fpm: pool $domain" | awk '{sum+=$6} END {print sum}'
+```
+
+#### 6. **Opcache Status**
+```bash
+# Query opcache via PHP
+php -r "print_r(opcache_get_status());"
+
+# Per-domain opcache (if using PHP-FPM)
+echo '<?php print_r(opcache_get_status()); ?>' | \
+  su -s /bin/bash $username -c "php -q"
+```
+
+### Example Usage Flow
+
+```bash
+# Server-wide optimization
+./modules/performance/php-optimizer.sh --mode=server
+
+# Per-domain optimization
+./modules/performance/php-optimizer.sh --domain=example.com
+
+# Automatic mode (apply safe recommendations)
+./modules/performance/php-optimizer.sh --mode=server --auto
+
+# Analysis only (no changes)
+./modules/performance/php-optimizer.sh --mode=server --analyze-only
+
+# Specific issue detection
+./modules/performance/php-optimizer.sh --check=max_children
+```
+
+### Safety Features
+
+1. **Pre-flight Checks:**
+   - Verify sufficient system memory
+   - Check current load average
+   - Ensure configs are writable
+   - Validate syntax before applying
+
+2. **Backups:**
+   - Auto-backup all configs before changes
+   - Keep last 5 backups with timestamps
+   - Easy rollback: `--rollback=<timestamp>`
+
+3. **Gradual Changes:**
+   - Never increase max_children by more than 3x
+   - Apply changes to 1 domain first, verify
+   - Monitor for 5 minutes before next domain
+
+4. **Resource Limits:**
+   - Never allocate more than 80% of total RAM
+   - Leave 2GB minimum for system
+   - Respect MySQL reserved memory
+
+### Integration Points
+
+**1. Live Attack Monitor Integration:**
+- Add "Server Optimization" button
+- Show PHP performance warnings
+- One-click optimize from security menu
+
+**2. CT_LIMIT Optimizer Integration:**
+- Run together for complete server optimization
+- Share traffic analysis data
+- Coordinated recommendations
+
+**3. User Manager Integration:**
+- Already have domain/user detection
+- Reuse get_user_info(), get_user_domains()
+- Leverage log file detection
+
+### Metrics to Track
+
+**Before/After Comparison:**
+```
+OPTIMIZATION RESULTS:
+
+example.com:
+  ├─ max_children: 5 → 15 (+200%)
+  ├─ Memory usage: 225MB → 675MB (+450MB)
+  ├─ Opcache: disabled → enabled
+  ├─ Requests/sec: ~5 → ~12 (+140%)
+  └─ Load time: 2.5s → 0.8s (-68%)
+
+System Impact:
+  ├─ Total FPM memory: 2.1GB → 3.8GB
+  ├─ Load average: 2.5 → 1.8 (-28%)
+  └─ Available RAM: 8GB → 6.5GB
+```
+
+### Future Enhancements
+
+1. **Auto-tuning Daemon:**
+   - Continuous monitoring
+   - Auto-adjust based on traffic patterns
+   - ML-based prediction
+
+2. **Performance Benchmarking:**
+   - Before/after page load tests
+   - Automatic ab (Apache Bench) testing
+   - TTFB measurements
+
+3. **Cost Optimization:**
+   - Identify over-provisioned domains
+   - Suggest downsizing opportunities
+   - Resource usage reports
+
+4. **Alerting:**
+   - Email when max_children hit
+   - Slack/Discord webhooks
+   - Integration with monitoring tools
+
+## Next Steps
+
+1. ✅ Review this plan
+2. Create lib/php-detector.sh (detection logic)
+3. Create lib/php-analyzer.sh (analysis logic)
+4. Create modules/performance/php-optimizer.sh (main script)
+5. Test on small server first
+6. Add to live-attack-monitor menu
+7. Full testing on production