From 56f84a6db4e0b4f902b62d7e0f98bfcb2301907c Mon Sep 17 00:00:00 2001 From: cschantz Date: Tue, 2 Dec 2025 19:34:04 -0500 Subject: [PATCH] Add comprehensive PHP & Server Optimizer planning document FEATURE PLANNING: PHP-FPM and server-wide optimization system OVERVIEW: Intelligent analyzer that scans all domains, detects PHP configs, analyzes usage patterns, and provides one-click optimization with automatic backups and safety checks. LEVERAGES EXISTING INFRASTRUCTURE: - user-manager.sh: Domain/user detection (70% of work done) - system-detect.sh: Control panel detection - optimize-ct-limit.sh: Traffic analysis model - get_user_log_files(): Log location mapping CORE CAPABILITIES: 1. Detect all PHP-FPM pool configs per domain 2. Find php.ini hierarchy (.user.ini, local, global) 3. Analyze memory usage, traffic patterns, error logs 4. Calculate optimal pm.max_children, memory_limit, opcache 5. Detect issues: max_children reached, memory exhausted, slow requests 6. Provide actionable recommendations with safety checks 7. One-click apply with automatic backups IMPLEMENTATION PHASES: - Phase 1: lib/php-detector.sh (detection logic) - Phase 2: lib/php-analyzer.sh (analysis engine) - Phase 3: modules/performance/php-optimizer.sh (main script) - Phase 4: Integration with live-attack-monitor TRACKED METRICS: - pm.max_children, pm.start_servers, pm.min/max_spare_servers - memory_limit, max_execution_time, upload_max_filesize - opcache settings, hit rates, memory consumption - Process counts, memory usage, CPU patterns - Error rates, slow request logs NEXT: Expand metrics tracking and begin Phase 1 implementation --- docs/PHP_OPTIMIZER_PLAN.md | 429 +++++++++++++++++++++++++++++++++++++ 1 file changed, 429 insertions(+) create mode 100644 docs/PHP_OPTIMIZER_PLAN.md diff --git a/docs/PHP_OPTIMIZER_PLAN.md b/docs/PHP_OPTIMIZER_PLAN.md new file mode 100644 index 0000000..3b5ebc1 --- /dev/null +++ b/docs/PHP_OPTIMIZER_PLAN.md @@ -0,0 +1,429 @@ +# PHP & Server Optimizer - Comprehensive Planning Document + +## Overview +Intelligent PHP-FPM, memory, and resource optimizer that analyzes per-domain usage patterns and provides actionable recommendations with one-click fixes. + +## What We Already Have (Foundation) +✅ **user-manager.sh** - Complete user/domain detection for cPanel, Plesk, InterWorx +✅ **system-detect.sh** - Control panel, PHP version, web server detection +✅ **optimize-ct-limit.sh** - Traffic pattern analysis model (can reuse approach) +✅ **Domain home directories already tracked** via get_user_info() +✅ **Log file detection** via get_user_log_files() + +## Architecture + +### Module Name +`/root/server-toolkit/modules/performance/php-optimizer.sh` + +### Core Components + +#### 1. **Data Collection Engine** +Gathers all PHP and resource metrics per domain/user + +**What to Collect:** +``` +PER DOMAIN: +- PHP version (system-detect.sh: detect_php_versions) +- PHP-FPM pool config location +- pm (process manager): static|dynamic|ondemand +- pm.max_children (current value) +- pm.start_servers +- pm.min_spare_servers +- pm.max_spare_servers +- pm.max_requests +- memory_limit (php.ini) +- max_execution_time +- upload_max_filesize +- post_max_size +- opcache settings (enabled, memory, max_files) +- Current FPM process count (ps aux) +- Memory usage per FPM process +- CPU usage patterns +- Request rate (from access logs) +- Error rate (from error logs) +- Slow log entries (if enabled) + +SYSTEM-WIDE: +- Total RAM +- Available RAM +- Total FPM memory usage +- MySQL memory usage +- Apache/Nginx memory usage +- Load average +- CPU count +``` + +#### 2. **Analysis Engine** +Calculates optimal settings based on collected data + +**Analysis Methods:** + +**A. Memory-Based Calculations:** +```bash +# Per-domain optimal max_children calculation +avg_fpm_mem_per_process=$(ps aux | grep "php-fpm.*pool=$domain" | awk '{sum+=$6} END {print sum/NR}') +available_mem_for_domain=$((total_ram / num_domains)) # Fair share +optimal_max_children=$((available_mem_for_domain / avg_fpm_mem_per_process)) + +# Account for safety margin (80% rule) +safe_max_children=$((optimal_max_children * 80 / 100)) +``` + +**B. Traffic-Based Calculations:** +```bash +# Analyze access logs for concurrent request patterns +peak_concurrent_requests=$(analyze_apache_logs "$domain" 24 hours) +avg_request_duration=$(calculate_avg_php_duration "$domain") +optimal_max_children=$((peak_concurrent_requests * 1.5)) # 50% buffer +``` + +**C. Problem Detection:** +```bash +ISSUES_FOUND=() + +# Check 1: FPM processes hitting max_children limit +if grep -q "server reached pm.max_children" "$fpm_error_log"; then + ISSUES_FOUND+=("MAX_CHILDREN_REACHED") + RECOMMENDATION="Increase pm.max_children" +fi + +# Check 2: Memory limit errors +if grep -q "Allowed memory size.*exhausted" "$php_error_log"; then + ISSUES_FOUND+=("MEMORY_EXHAUSTED") + RECOMMENDATION="Increase memory_limit" +fi + +# Check 3: Slow requests +if [ -f "$slow_log" ]; then + slow_count=$(wc -l < "$slow_log") + if [ "$slow_count" -gt 100 ]; then + ISSUES_FOUND+=("SLOW_REQUESTS") + RECOMMENDATION="Optimize PHP code or increase max_execution_time" + fi +fi + +# Check 4: Opcache hit rate +opcache_hit_rate=$(php -r "print_r(opcache_get_status());" | grep hit_rate | awk '{print $2}') +if [ "$opcache_hit_rate" -lt 80 ]; then + ISSUES_FOUND+=("LOW_OPCACHE_HIT_RATE") + RECOMMENDATION="Increase opcache.memory_consumption" +fi +``` + +#### 3. **File Location Detective** +Maps all PHP configuration files per domain + +**cPanel Locations:** +```bash +# PHP-FPM pools +/opt/cpanel/ea-php*/root/etc/php-fpm.d/$username.conf +/var/cpanel/userdata/$username/$domain + +# PHP.ini locations +/opt/cpanel/ea-php*/root/etc/php.d/ +~/.php/ +/home/$username/.php/ +/home/$username/public_html/.user.ini +``` + +**Plesk Locations:** +```bash +# PHP-FPM pools +/etc/php-fpm.d/plesk-php*-fpm/$domain.conf + +# PHP.ini +/var/www/vhosts/system/$domain/etc/php.ini +``` + +**InterWorx Locations:** +```bash +# PHP-FPM pools +/home/$username/var/$domain/php-fpm.conf + +# PHP.ini +/home/$username/var/$domain/etc/php.ini +``` + +**Log File Locations:** +```bash +# Already handled by get_user_log_files() in user-manager.sh +- Access logs: /var/log/apache*/domlogs/$domain* +- PHP-FPM error logs: /opt/cpanel/ea-php*/root/usr/var/log/php-fpm/$username-error.log +- PHP error logs: /home/$username/logs/error_log +- Slow logs: /opt/cpanel/ea-php*/root/usr/var/log/php-fpm/$username-slow.log +``` + +#### 4. **Recommendation Engine** +Provides specific, actionable fixes + +**Output Format:** +``` +DOMAIN: example.com (user: examplec, PHP 8.2) +STATUS: ⚠️ NEEDS OPTIMIZATION + +CURRENT CONFIGURATION: + ├─ pm.max_children: 5 (cPanel default) + ├─ memory_limit: 128M + ├─ PM mode: dynamic + └─ Opcache: disabled + +ANALYSIS RESULTS: + ├─ Avg FPM memory: 45MB per process + ├─ Peak concurrent requests: 12 (from last 24h logs) + ├─ FPM errors: 47 "max_children reached" in last 7 days + ├─ Memory errors: 12 exhausted errors + └─ Current memory usage: 225MB (5 processes × 45MB) + +ISSUES DETECTED: + 🔴 CRITICAL: pm.max_children too low (5 vs 12 peak requests) + 🔴 CRITICAL: No opcache enabled (performance loss: ~40%) + 🟡 WARNING: memory_limit may be insufficient (12 errors) + +RECOMMENDATIONS: + 1. Increase pm.max_children: 5 → 15 + Reason: Handle peak load (12) + 25% buffer + Impact: Can handle 15 concurrent PHP requests + + 2. Enable opcache with optimal settings + Reason: Massive performance gain, reduce CPU by 40% + Settings: + opcache.enable=1 + opcache.memory_consumption=128 + opcache.max_accelerated_files=10000 + + 3. Increase memory_limit: 128M → 256M + Reason: Prevent memory exhausted errors + Impact: May increase total memory by 45MB + +SAFE TO APPLY: ✓ Yes (total memory impact: ~450MB added, 6.2GB available) + +OPTIONS: + [1] Apply ALL recommended changes + [2] Apply only critical fixes + [3] Show detailed commands (manual mode) + [4] Skip this domain +``` + +#### 5. **Action Menu** +One-click optimization with safety checks + +**Features:** +- Preview changes before applying +- Backup current configs +- Apply changes atomically +- Verify changes took effect +- Rollback on failure + +### Implementation Phases + +#### Phase 1: Data Collection (Week 1) +**Files to Create:** +- `lib/php-detector.sh` - Detect all PHP configs per domain +- `lib/php-analyzer.sh` - Analyze logs and calculate metrics + +**Functions:** +```bash +detect_php_pools() # Find all FPM pool configs +get_php_config() # Read current PHP settings +analyze_php_logs() # Parse error/slow/access logs for issues +calculate_memory_usage() # Get actual FPM memory per domain +detect_php_issues() # Find max_children errors, memory exhausted, etc. +``` + +#### Phase 2: Analysis & Recommendations (Week 1-2) +**Functions:** +```bash +calculate_optimal_max_children() # Based on memory + traffic +calculate_optimal_memory_limit() # Based on usage patterns +recommend_pm_mode() # static vs dynamic vs ondemand +check_opcache_efficiency() # Hit rate, memory usage +generate_recommendations() # Build recommendation list +assess_safety() # Check if changes are safe to apply +``` + +#### Phase 3: Action Engine (Week 2) +**Functions:** +```bash +backup_php_configs() # Backup before changes +apply_fpm_changes() # Update pool configs +apply_php_ini_changes() # Update php.ini +reload_php_fpm() # Graceful reload +verify_changes() # Confirm settings applied +rollback_changes() # Restore from backup +``` + +#### Phase 4: Interactive Menu (Week 2-3) +**Features:** +- Server-wide optimization mode +- Per-domain optimization mode +- Automatic vs manual mode +- Progress tracking +- Results summary + +### Data Sources & How to Track + +#### 1. **Domain Discovery** +```bash +# Already have this! +source /root/server-toolkit/lib/user-manager.sh +users=$(list_all_users) +for user in $users; do + domains=$(get_user_domains "$user") + for domain in $domains; do + # Process each domain + done +done +``` + +#### 2. **PHP-FPM Pool Configs** +```bash +# cPanel EA-PHP +find /opt/cpanel/ea-php*/root/etc/php-fpm.d/ -name "*.conf" -type f + +# Plesk +find /etc/php-fpm.d/ -name "*.conf" -type f 2>/dev/null + +# InterWorx +find /home/*/var/*/php-fpm.conf -type f 2>/dev/null +``` + +#### 3. **PHP Error Logs** +```bash +# Use existing function! +error_logs=$(get_user_log_files "$user" "error") +``` + +#### 4. **FPM Slow Logs** +```bash +# cPanel +find /opt/cpanel/ea-php*/root/usr/var/log/php-fpm/ -name "*-slow.log" +``` + +#### 5. **Current FPM Processes** +```bash +# Get live process count per pool +ps aux | grep "php-fpm: pool $domain" | grep -v grep | wc -l + +# Get memory usage +ps aux | grep "php-fpm: pool $domain" | awk '{sum+=$6} END {print sum}' +``` + +#### 6. **Opcache Status** +```bash +# Query opcache via PHP +php -r "print_r(opcache_get_status());" + +# Per-domain opcache (if using PHP-FPM) +echo '' | \ + su -s /bin/bash $username -c "php -q" +``` + +### Example Usage Flow + +```bash +# Server-wide optimization +./modules/performance/php-optimizer.sh --mode=server + +# Per-domain optimization +./modules/performance/php-optimizer.sh --domain=example.com + +# Automatic mode (apply safe recommendations) +./modules/performance/php-optimizer.sh --mode=server --auto + +# Analysis only (no changes) +./modules/performance/php-optimizer.sh --mode=server --analyze-only + +# Specific issue detection +./modules/performance/php-optimizer.sh --check=max_children +``` + +### Safety Features + +1. **Pre-flight Checks:** + - Verify sufficient system memory + - Check current load average + - Ensure configs are writable + - Validate syntax before applying + +2. **Backups:** + - Auto-backup all configs before changes + - Keep last 5 backups with timestamps + - Easy rollback: `--rollback=` + +3. **Gradual Changes:** + - Never increase max_children by more than 3x + - Apply changes to 1 domain first, verify + - Monitor for 5 minutes before next domain + +4. **Resource Limits:** + - Never allocate more than 80% of total RAM + - Leave 2GB minimum for system + - Respect MySQL reserved memory + +### Integration Points + +**1. Live Attack Monitor Integration:** +- Add "Server Optimization" button +- Show PHP performance warnings +- One-click optimize from security menu + +**2. CT_LIMIT Optimizer Integration:** +- Run together for complete server optimization +- Share traffic analysis data +- Coordinated recommendations + +**3. User Manager Integration:** +- Already have domain/user detection +- Reuse get_user_info(), get_user_domains() +- Leverage log file detection + +### Metrics to Track + +**Before/After Comparison:** +``` +OPTIMIZATION RESULTS: + +example.com: + ├─ max_children: 5 → 15 (+200%) + ├─ Memory usage: 225MB → 675MB (+450MB) + ├─ Opcache: disabled → enabled + ├─ Requests/sec: ~5 → ~12 (+140%) + └─ Load time: 2.5s → 0.8s (-68%) + +System Impact: + ├─ Total FPM memory: 2.1GB → 3.8GB + ├─ Load average: 2.5 → 1.8 (-28%) + └─ Available RAM: 8GB → 6.5GB +``` + +### Future Enhancements + +1. **Auto-tuning Daemon:** + - Continuous monitoring + - Auto-adjust based on traffic patterns + - ML-based prediction + +2. **Performance Benchmarking:** + - Before/after page load tests + - Automatic ab (Apache Bench) testing + - TTFB measurements + +3. **Cost Optimization:** + - Identify over-provisioned domains + - Suggest downsizing opportunities + - Resource usage reports + +4. **Alerting:** + - Email when max_children hit + - Slack/Discord webhooks + - Integration with monitoring tools + +## Next Steps + +1. ✅ Review this plan +2. Create lib/php-detector.sh (detection logic) +3. Create lib/php-analyzer.sh (analysis logic) +4. Create modules/performance/php-optimizer.sh (main script) +5. Test on small server first +6. Add to live-attack-monitor menu +7. Full testing on production