Add comprehensive PHP metrics tracking documentation

DOCUMENTATION: Complete guide to PHP configuration hierarchy and metrics CRITICAL ADDITIONS: 1. PHP Config Hierarchy (.user.ini > pool php.ini > global) 2. How to determine which config takes effect 3. 70+ PHP settings to track with explanations COMPREHENSIVE METRICS COVERAGE: **Memory Settings:** - memory_limit, upload_max_filesize, post_max_size - max_input_vars, realpath_cache_size - Detection: memory exhausted errors, upload failures **PHP-FPM Pool Settings (MOST CRITICAL!):** - pm (static/dynamic/ondemand modes) - pm.max_children, pm.start_servers, pm.min/max_spare_servers - pm.max_requests, pm.process_idle_timeout - request_terminate_timeout, request_slowlog_timeout - Detection: max_children reached errors, slow logs **OPcache (MASSIVE PERFORMANCE!):** - opcache.enable, opcache.memory_consumption - opcache.max_accelerated_files - opcache.jit, opcache.jit_buffer_size (PHP 8+) - Hit rate calculation, cache effectiveness **Execution & Timeout:** - max_execution_time, max_input_time - default_socket_timeout - Detection: timeout errors **Session Management:** - session.save_handler (files/redis/memcached) - session.gc_maxlifetime - Performance impact analysis **Security Settings:** - disable_functions, open_basedir - display_errors (MUST be Off in production!) - allow_url_include prevention **APCu Cache:** - apc.shm_size, apc.ttl - User cache tracking **Detection Commands:** - Find all php.ini files affecting domain - Get effective settings hierarchy - Check opcache hit rates - Find max_children errors - Track slow requests - Calculate memory per process **Per-Domain Metrics Matrix:** Complete YAML template showing all tracked metrics, live stats, issue detection, and recommendations This documentation enables intelligent optimization with precise detection and actionable recommendations!
2025-12-02 19:37:24 -05:00
parent 66c01296c5
commit 1afe7c476a
1 changed files with 469 additions and 0 deletions
@@ -0,0 +1,469 @@
+# Comprehensive PHP Metrics Tracking Guide
+
+## PHP Configuration Hierarchy & Detection
+
+### Configuration File Priority (Highest to Lowest)
+Understanding which config takes effect is critical for accurate optimization.
+
+```
+1. .user.ini (per-directory, PHP-FPM only)
+   Location: /home/user/public_html/.user.ini
+   Scope: Specific directory and subdirectories
+   Reloads: Automatically every user_ini.cache_ttl seconds (default 300)
+
+2. .htaccess (Apache with mod_php only, NOT PHP-FPM!)
+   Location: /home/user/public_html/.htaccess
+   Scope: Directory-specific
+   Note: Does NOT work with PHP-FPM!
+
+3. php.ini (per-pool, cPanel EA-PHP)
+   Location: /opt/cpanel/ea-php*/root/etc/php.ini
+   Scope: All domains using that PHP version
+
+4. Additional .ini files (per-pool)
+   Location: /opt/cpanel/ea-php*/root/etc/php.d/*.ini
+   Scope: Per PHP version, loaded alphabetically
+
+5. Global php.ini
+   Location: /etc/php.ini (legacy)
+   Scope: System-wide fallback
+```
+
+### How to Determine Effective Settings
+
+**Method 1: Query via PHP (Most Accurate)**
+```bash
+# Get effective value for specific domain
+echo '<?php echo ini_get("memory_limit"); ?>' | \
+  su -s /bin/bash $username -c "php -q -d open_basedir="
+
+# Get ALL effective settings
+php -r 'print_r(ini_get_all());' > /tmp/php_all_settings.txt
+
+# Per-domain via web request (if domain is accessible)
+curl -s "http://$domain/phpinfo.php" | grep -A1 "memory_limit"
+```
+
+**Method 2: Parse Configuration Files**
+```bash
+# Find ALL possible config files affecting a domain
+find_php_configs() {
+    local domain="$1"
+    local user="$2"
+    local php_version="$3"  # e.g., "ea-php82"
+
+    # Priority order
+    echo "=== Config Hierarchy for $domain ==="
+
+    # 1. .user.ini
+    local user_ini="/home/$user/public_html/.user.ini"
+    if [ -f "$user_ini" ]; then
+        echo "1. .user.ini: $user_ini (HIGHEST PRIORITY)"
+        grep -E "memory_limit|max_execution_time|upload_max_filesize" "$user_ini"
+    fi
+
+    # 2. Pool-specific php.ini
+    local pool_ini="/opt/cpanel/$php_version/root/etc/php.ini"
+    if [ -f "$pool_ini" ]; then
+        echo "2. Pool php.ini: $pool_ini"
+        grep -E "memory_limit|max_execution_time|upload_max_filesize" "$pool_ini"
+    fi
+
+    # 3. Additional .ini files
+    local ini_dir="/opt/cpanel/$php_version/root/etc/php.d"
+    if [ -d "$ini_dir" ]; then
+        echo "3. Additional .ini files: $ini_dir/*.ini"
+        grep -h -E "memory_limit|max_execution_time|upload_max_filesize" "$ini_dir"/*.ini 2>/dev/null
+    fi
+}
+```
+
+## Complete PHP Metrics to Track
+
+### 1. **Memory Settings** (Critical for Performance)
+
+```ini
+# Basic Memory
+memory_limit = 256M                    # Per-script memory limit
+                                       # Track: Current value, recommended, % of total RAM
+
+# Upload Limits (Related to Memory)
+upload_max_filesize = 64M              # Max single file upload
+post_max_size = 128M                   # Max POST data (should be >= upload_max_filesize)
+max_input_vars = 1000                  # Max input variables (forms with many fields)
+max_input_nesting_level = 64           # Max array nesting depth
+max_input_time = 60                    # Max time parsing input data
+
+# Realpath Cache (Memory for path resolution)
+realpath_cache_size = 4096K            # Cache size for realpath() calls
+realpath_cache_ttl = 120               # TTL in seconds
+```
+
+**Why Track:**
+- `memory_limit` too low → "Allowed memory size exhausted" errors
+- `post_max_size < upload_max_filesize` → Upload failures
+- `realpath_cache_size` too small → File I/O slowdowns
+
+**Detection:**
+```bash
+# Find memory exhausted errors
+grep -r "Allowed memory size.*exhausted" /home/$user/*/logs/error_log
+
+# Find upload failures
+grep -r "POST Content-Length.*exceeds" /home/$user/*/logs/error_log
+```
+
+### 2. **Execution & Timeout Settings**
+
+```ini
+# Script Execution
+max_execution_time = 30                # Max script runtime (seconds)
+max_input_time = 60                    # Max time for input parsing
+default_socket_timeout = 60            # Default socket timeout
+
+# CGI-specific
+cgi.force_redirect = 1
+cgi.fix_pathinfo = 0                   # Security: prevent path injection
+```
+
+**Why Track:**
+- `max_execution_time` too low → Scripts timeout on slow operations
+- Long-running cron jobs need higher limits
+
+**Detection:**
+```bash
+# Find timeout errors
+grep -r "Maximum execution time.*exceeded" /home/$user/*/logs/error_log
+```
+
+### 3. **PHP-FPM Pool Settings** (Most Critical for Optimization!)
+
+```ini
+# Process Manager Type
+pm = dynamic                           # static | dynamic | ondemand
+                                       # static: Fixed number of children
+                                       # dynamic: Scales between min/max
+                                       # ondemand: Spawns on-demand (saves memory)
+
+# Process Limits (DYNAMIC mode)
+pm.max_children = 50                   # Max simultaneous processes
+pm.start_servers = 5                   # Processes started at boot
+pm.min_spare_servers = 5               # Minimum idle processes
+pm.max_spare_servers = 35              # Maximum idle processes
+
+# Process Limits (STATIC mode)
+pm.max_children = 50                   # Fixed number of processes
+
+# Process Limits (ONDEMAND mode)
+pm.max_children = 50                   # Max processes
+pm.process_idle_timeout = 10s          # Kill idle process after X seconds
+
+# Process Recycling
+pm.max_requests = 500                  # Respawn after X requests (prevent memory leaks)
+
+# Status & Monitoring
+pm.status_path = /fpm-status           # Status page URL
+ping.path = /fpm-ping                  # Health check URL
+ping.response = pong
+
+# Timeouts
+request_terminate_timeout = 30s        # Kill request after X seconds (0 = disabled)
+request_slowlog_timeout = 5s           # Log slow requests taking > X seconds
+
+# Logging
+slowlog = /var/log/php-fpm/$pool-slow.log
+catch_workers_output = yes             # Capture stdout/stderr
+php_admin_value[error_log] = /var/log/php-fpm/$pool-error.log
+```
+
+**Why Track (CRITICAL!):**
+- `pm.max_children` too low → "server reached pm.max_children" errors → requests queue/fail
+- `pm.max_children` too high → OOM kills, server crashes
+- `pm = static` wastes memory on low-traffic sites
+- `pm = ondemand` adds latency (process spawn time)
+- `pm.max_requests = 0` → memory leaks never cleared
+
+**Detection:**
+```bash
+# Find max_children errors (CRITICAL)
+grep "server reached pm.max_children" /opt/cpanel/ea-php*/root/usr/var/log/php-fpm/*error.log
+
+# Find slow requests
+tail -100 /opt/cpanel/ea-php*/root/usr/var/log/php-fpm/*slow.log
+
+# Current process count vs limit
+current=$(ps aux | grep "php-fpm: pool $domain" | grep -v grep | wc -l)
+max=$(grep "pm.max_children" /opt/cpanel/ea-php*/root/etc/php-fpm.d/$user.conf | cut -d'=' -f2)
+echo "Current: $current / Max: $max"
+```
+
+### 4. **OPcache Settings** (Massive Performance Impact!)
+
+```ini
+[opcache]
+; Enable/Disable
+opcache.enable = 1                     # Enable opcache
+opcache.enable_cli = 0                 # Disable for CLI (causes issues)
+
+; Memory Settings
+opcache.memory_consumption = 128       # MB for opcache (CRITICAL!)
+opcache.interned_strings_buffer = 8    # MB for string interning
+opcache.max_accelerated_files = 10000  # Max cached files (set to > total PHP files)
+
+; Validation & Updates
+opcache.revalidate_freq = 2            # Check file changes every X seconds (0 = always check)
+opcache.validate_timestamps = 1        # Check if files changed (0 = never check, production)
+opcache.fast_shutdown = 1              # Faster shutdown
+
+; Advanced
+opcache.enable_file_override = 1       # Optimize file_exists(), is_file()
+opcache.optimization_level = 0x7FFFBFFF
+opcache.save_comments = 1              # Required for some frameworks (Doctrine, Symfony)
+opcache.load_comments = 1
+
+; JIT (PHP 8.0+)
+opcache.jit = tracing                  # off | function | tracing
+opcache.jit_buffer_size = 100M         # JIT compilation buffer
+```
+
+**Why Track (HUGE PERFORMANCE!):**
+- Opcache disabled → 40-70% slower, 300% more CPU
+- `opcache.memory_consumption` too small → Cache thrashing
+- `opcache.max_accelerated_files` too low → Not all files cached
+- Hit rate < 90% → Increase memory or max files
+
+**Detection:**
+```bash
+# Get opcache status (MOST IMPORTANT METRICS!)
+php -r "print_r(opcache_get_status());" | grep -E "opcache_enabled|memory_usage|opcache_statistics|num_cached_scripts|hits|misses|blacklist_misses"
+
+# Calculate hit rate
+stats=$(php -r '$s=opcache_get_status(); echo $s["opcache_statistics"]["hits"].",".$s["opcache_statistics"]["misses"];')
+hits=$(echo $stats | cut -d',' -f1)
+misses=$(echo $stats | cut -d',' -f2)
+total=$((hits + misses))
+hit_rate=$((hits * 100 / total))
+echo "Opcache Hit Rate: ${hit_rate}%"
+
+# If hit rate < 90% → Need more memory or max_files!
+```
+
+### 5. **Session Settings**
+
+```ini
+session.save_handler = files           # files | memcached | redis
+session.save_path = "/var/lib/php/session"
+session.gc_maxlifetime = 1440          # Session timeout (seconds)
+session.gc_probability = 1
+session.gc_divisor = 1000              # GC runs 1/1000 requests
+session.cookie_lifetime = 0            # Session cookie expires on browser close
+```
+
+**Why Track:**
+- `session.save_path` full disk → Session writes fail
+- Using `files` on high-traffic → I/O bottleneck (use Redis!)
+
+### 6. **Error Handling & Logging**
+
+```ini
+error_reporting = E_ALL & ~E_DEPRECATED & ~E_STRICT
+display_errors = Off                   # CRITICAL: Must be Off in production!
+display_startup_errors = Off
+log_errors = On                        # Log to file
+error_log = /home/$user/logs/php_error.log
+ignore_repeated_errors = Off
+ignore_repeated_source = Off
+report_memleaks = On
+```
+
+**Why Track:**
+- `display_errors = On` in production → Security risk (exposes paths)
+- No `error_log` set → Errors go to Apache log (harder to track)
+
+### 7. **Security Settings**
+
+```ini
+; Disable Dangerous Functions
+disable_functions = exec,passthru,shell_exec,system,proc_open,popen,curl_exec,curl_multi_exec,parse_ini_file,show_source
+
+; Open Basedir (Restrict File Access)
+open_basedir = /home/$user:/tmp       # Prevent directory traversal
+
+; File Uploads
+file_uploads = On
+upload_tmp_dir = /tmp                  # Temp upload directory
+
+; Misc Security
+expose_php = Off                       # Hide PHP version in headers
+allow_url_fopen = On                   # Allow remote file access (needed for many apps)
+allow_url_include = Off                # CRITICAL: Prevent remote code execution
+```
+
+### 8. **APCu Cache** (User Cache, separate from OPcache)
+
+```ini
+[apcu]
+apc.enabled = 1
+apc.shm_size = 32M                     # Shared memory size
+apc.ttl = 7200                         # Time to live
+apc.gc_ttl = 3600                      # Garbage collection TTL
+apc.enable_cli = 0
+```
+
+**Why Track:**
+- WordPress object cache, WooCommerce, etc. use APCu
+- Low hit rate → Increase shm_size
+
+### 9. **MySQL/Database Settings** (php.ini side)
+
+```ini
+mysqli.max_persistent = -1             # Max persistent connections (-1 = unlimited)
+mysqli.max_links = -1                  # Max total connections
+mysqli.default_socket = /var/lib/mysql/mysql.sock
+pdo_mysql.default_socket = /var/lib/mysql/mysql.sock
+```
+
+### 10. **Zend Extensions**
+
+```ini
+zend_extension=opcache.so
+zend_extension=ioncube_loader_lin_8.2.so  # If using IonCube
+```
+
+## Complete Metrics Tracking List
+
+### Per-Domain Tracking Matrix
+
+```yaml
+domain: example.com
+user: examplec
+php_version: ea-php82
+
+config_hierarchy:
+  1_user_ini: /home/examplec/public_html/.user.ini
+  2_pool_ini: /opt/cpanel/ea-php82/root/etc/php.ini
+  3_pool_d: /opt/cpanel/ea-php82/root/etc/php.d/
+  4_global: /etc/php.ini
+
+effective_settings:
+  # Memory
+  memory_limit: 256M
+  upload_max_filesize: 64M
+  post_max_size: 128M
+  max_input_vars: 1000
+  realpath_cache_size: 4096K
+
+  # Execution
+  max_execution_time: 30
+  max_input_time: 60
+  request_terminate_timeout: 30
+
+  # PHP-FPM Pool
+  pm: dynamic
+  pm.max_children: 50
+  pm.start_servers: 5
+  pm.min_spare_servers: 5
+  pm.max_spare_servers: 35
+  pm.max_requests: 500
+  pm.process_idle_timeout: 10s
+
+  # OPcache
+  opcache.enable: 1
+  opcache.memory_consumption: 128M
+  opcache.max_accelerated_files: 10000
+  opcache.jit: tracing
+  opcache.jit_buffer_size: 100M
+
+  # Sessions
+  session.save_handler: redis
+  session.save_path: "tcp://127.0.0.1:6379"
+
+  # Security
+  display_errors: Off
+  open_basedir: /home/examplec:/tmp
+  disable_functions: exec,passthru,shell_exec
+
+live_metrics:
+  # Process Stats
+  current_processes: 12
+  avg_memory_per_process: 45MB
+  total_memory_usage: 540MB
+  cpu_usage: 15%
+
+  # OPcache Stats
+  opcache_hit_rate: 95.3%
+  opcache_memory_used: 87MB / 128MB
+  opcache_cached_scripts: 2847 / 10000
+  opcache_wasted_memory: 2.1MB
+
+  # Traffic Stats (last 24h)
+  peak_concurrent_requests: 18
+  avg_requests_per_minute: 45
+  total_requests: 64,800
+
+  # Error Stats (last 7 days)
+  memory_exhausted: 0
+  max_execution_time: 3
+  max_children_reached: 47  # CRITICAL!
+  slow_requests: 12
+
+issues_detected:
+  - type: CRITICAL
+    code: MAX_CHILDREN_REACHED
+    count: 47
+    message: "pm.max_children limit hit 47 times in 7 days"
+    recommendation: "Increase from 50 to 75"
+
+  - type: WARNING
+    code: SLOW_REQUESTS
+    count: 12
+    message: "12 requests took > 5 seconds"
+    recommendation: "Review slow log, optimize code"
+
+recommendations:
+  - priority: HIGH
+    setting: pm.max_children
+    current: 50
+    recommended: 75
+    reason: "Peak concurrent (18) + buffer (50%) + safety margin"
+    impact: "Handle 75 concurrent PHP requests vs 50"
+    memory_impact: +1.1GB
+
+  - priority: MEDIUM
+    setting: opcache.max_accelerated_files
+    current: 10000
+    recommended: 15000
+    reason: "Currently caching 2847 files, room for growth"
+    impact: "Better cache coverage as site grows"
+```
+
+## Detection Commands Cheat Sheet
+
+```bash
+# Find ALL php.ini files affecting a domain
+find /opt/cpanel/ea-php*/root/etc/ -name "php.ini"
+find /home/$user/public_html -name ".user.ini"
+
+# Find FPM pool config
+grep -r "pool.*$domain" /opt/cpanel/ea-php*/root/etc/php-fpm.d/
+
+# Get effective settings for domain
+su -s /bin/bash $user -c "php -r 'phpinfo();'" | grep -A1 "memory_limit"
+
+# Check opcache status
+php -r "var_dump(opcache_get_status());"
+
+# Find max_children errors
+grep -r "max_children" /opt/cpanel/ea-php*/root/usr/var/log/php-fpm/
+
+# Find slow requests
+find /opt/cpanel/ea-php*/root/usr/var/log/php-fpm/ -name "*slow.log" -exec tail -50 {} \;
+
+# Count current FPM processes
+ps aux | grep "php-fpm: pool $domain" | wc -l
+
+# Memory per process
+ps aux | grep "php-fpm: pool $domain" | awk '{sum+=$6} END {print sum/NR " KB avg per process"}'
+```
+
+This comprehensive tracking will allow us to build an intelligent optimizer that knows EXACTLY what to fix!