Linux-Server-Management-Toolkit/docs/PHP_METRICS_COMPREHENSIVE.md

# Comprehensive PHP Metrics Tracking Guide

## PHP Configuration Hierarchy & Detection

### Configuration File Priority (Highest to Lowest)
Understanding which config takes effect is critical for accurate optimization.

```
1. .user.ini (per-directory, PHP-FPM only)
   Location: /home/user/public_html/.user.ini
   Scope: Specific directory and subdirectories
   Reloads: Automatically every user_ini.cache_ttl seconds (default 300)

2. .htaccess (Apache with mod_php only, NOT PHP-FPM!)
   Location: /home/user/public_html/.htaccess
   Scope: Directory-specific
   Note: Does NOT work with PHP-FPM!

3. php.ini (per-pool, cPanel EA-PHP)
   Location: /opt/cpanel/ea-php*/root/etc/php.ini
   Scope: All domains using that PHP version

4. Additional .ini files (per-pool)
   Location: /opt/cpanel/ea-php*/root/etc/php.d/*.ini
   Scope: Per PHP version, loaded alphabetically

5. Global php.ini
   Location: /etc/php.ini (legacy)
   Scope: System-wide fallback
```

### How to Determine Effective Settings

**Method 1: Query via PHP (Most Accurate)**
```bash
# Get effective value for specific domain
echo '<?php echo ini_get("memory_limit"); ?>' | \
  su -s /bin/bash $username -c "php -q -d open_basedir="

# Get ALL effective settings
php -r 'print_r(ini_get_all());' > /tmp/php_all_settings.txt

# Per-domain via web request (if domain is accessible)
curl -s "http://$domain/phpinfo.php" | grep -A1 "memory_limit"
```

**Method 2: Parse Configuration Files**
```bash
# Find ALL possible config files affecting a domain
find_php_configs() {
    local domain="$1"
    local user="$2"
    local php_version="$3"  # e.g., "ea-php82"

    # Priority order
    echo "=== Config Hierarchy for $domain ==="

    # 1. .user.ini
    local user_ini="/home/$user/public_html/.user.ini"
    if [ -f "$user_ini" ]; then
        echo "1. .user.ini: $user_ini (HIGHEST PRIORITY)"
        grep -E "memory_limit|max_execution_time|upload_max_filesize" "$user_ini"
    fi

    # 2. Pool-specific php.ini
    local pool_ini="/opt/cpanel/$php_version/root/etc/php.ini"
    if [ -f "$pool_ini" ]; then
        echo "2. Pool php.ini: $pool_ini"
        grep -E "memory_limit|max_execution_time|upload_max_filesize" "$pool_ini"
    fi

    # 3. Additional .ini files
    local ini_dir="/opt/cpanel/$php_version/root/etc/php.d"
    if [ -d "$ini_dir" ]; then
        echo "3. Additional .ini files: $ini_dir/*.ini"
        grep -h -E "memory_limit|max_execution_time|upload_max_filesize" "$ini_dir"/*.ini 2>/dev/null
    fi
}
```

## Complete PHP Metrics to Track

### 1. **Memory Settings** (Critical for Performance)

```ini
# Basic Memory
memory_limit = 256M                    # Per-script memory limit
                                       # Track: Current value, recommended, % of total RAM

# Upload Limits (Related to Memory)
upload_max_filesize = 64M              # Max single file upload
post_max_size = 128M                   # Max POST data (should be >= upload_max_filesize)
max_input_vars = 1000                  # Max input variables (forms with many fields)
max_input_nesting_level = 64           # Max array nesting depth
max_input_time = 60                    # Max time parsing input data

# Realpath Cache (Memory for path resolution)
realpath_cache_size = 4096K            # Cache size for realpath() calls
realpath_cache_ttl = 120               # TTL in seconds
```

**Why Track:**
- `memory_limit` too low → "Allowed memory size exhausted" errors
- `post_max_size < upload_max_filesize` → Upload failures
- `realpath_cache_size` too small → File I/O slowdowns

**Detection:**
```bash
# Find memory exhausted errors
grep -r "Allowed memory size.*exhausted" /home/$user/*/logs/error_log

# Find upload failures
grep -r "POST Content-Length.*exceeds" /home/$user/*/logs/error_log
```

### 2. **Execution & Timeout Settings**

```ini
# Script Execution
max_execution_time = 30                # Max script runtime (seconds)
max_input_time = 60                    # Max time for input parsing
default_socket_timeout = 60            # Default socket timeout

# CGI-specific
cgi.force_redirect = 1
cgi.fix_pathinfo = 0                   # Security: prevent path injection
```

**Why Track:**
- `max_execution_time` too low → Scripts timeout on slow operations
- Long-running cron jobs need higher limits

**Detection:**
```bash
# Find timeout errors
grep -r "Maximum execution time.*exceeded" /home/$user/*/logs/error_log
```

### 3. **PHP-FPM Pool Settings** (Most Critical for Optimization!)

```ini
# Process Manager Type
pm = dynamic                           # static | dynamic | ondemand
                                       # static: Fixed number of children
                                       # dynamic: Scales between min/max
                                       # ondemand: Spawns on-demand (saves memory)

# Process Limits (DYNAMIC mode)
pm.max_children = 50                   # Max simultaneous processes
pm.start_servers = 5                   # Processes started at boot
pm.min_spare_servers = 5               # Minimum idle processes
pm.max_spare_servers = 35              # Maximum idle processes

# Process Limits (STATIC mode)
pm.max_children = 50                   # Fixed number of processes

# Process Limits (ONDEMAND mode)
pm.max_children = 50                   # Max processes
pm.process_idle_timeout = 10s          # Kill idle process after X seconds

# Process Recycling
pm.max_requests = 500                  # Respawn after X requests (prevent memory leaks)

# Status & Monitoring
pm.status_path = /fpm-status           # Status page URL
ping.path = /fpm-ping                  # Health check URL
ping.response = pong

# Timeouts
request_terminate_timeout = 30s        # Kill request after X seconds (0 = disabled)
request_slowlog_timeout = 5s           # Log slow requests taking > X seconds

# Logging
slowlog = /var/log/php-fpm/$pool-slow.log
catch_workers_output = yes             # Capture stdout/stderr
php_admin_value[error_log] = /var/log/php-fpm/$pool-error.log
```

**Why Track (CRITICAL!):**
- `pm.max_children` too low → "server reached pm.max_children" errors → requests queue/fail
- `pm.max_children` too high → OOM kills, server crashes
- `pm = static` wastes memory on low-traffic sites
- `pm = ondemand` adds latency (process spawn time)
- `pm.max_requests = 0` → memory leaks never cleared

**Detection:**
```bash
# Find max_children errors (CRITICAL)
grep "server reached pm.max_children" /opt/cpanel/ea-php*/root/usr/var/log/php-fpm/*error.log

# Find slow requests
tail -100 /opt/cpanel/ea-php*/root/usr/var/log/php-fpm/*slow.log

# Current process count vs limit
current=$(ps aux | grep "php-fpm: pool $domain" | grep -v grep | wc -l)
max=$(grep "pm.max_children" /opt/cpanel/ea-php*/root/etc/php-fpm.d/$user.conf | cut -d'=' -f2)
echo "Current: $current / Max: $max"
```

### 4. **OPcache Settings** (Massive Performance Impact!)

```ini
[opcache]
; Enable/Disable
opcache.enable = 1                     # Enable opcache
opcache.enable_cli = 0                 # Disable for CLI (causes issues)

; Memory Settings
opcache.memory_consumption = 128       # MB for opcache (CRITICAL!)
opcache.interned_strings_buffer = 8    # MB for string interning
opcache.max_accelerated_files = 10000  # Max cached files (set to > total PHP files)

; Validation & Updates
opcache.revalidate_freq = 2            # Check file changes every X seconds (0 = always check)
opcache.validate_timestamps = 1        # Check if files changed (0 = never check, production)
opcache.fast_shutdown = 1              # Faster shutdown

; Advanced
opcache.enable_file_override = 1       # Optimize file_exists(), is_file()
opcache.optimization_level = 0x7FFFBFFF
opcache.save_comments = 1              # Required for some frameworks (Doctrine, Symfony)
opcache.load_comments = 1

; JIT (PHP 8.0+)
opcache.jit = tracing                  # off | function | tracing
opcache.jit_buffer_size = 100M         # JIT compilation buffer
```

**Why Track (HUGE PERFORMANCE!):**
- Opcache disabled → 40-70% slower, 300% more CPU
- `opcache.memory_consumption` too small → Cache thrashing
- `opcache.max_accelerated_files` too low → Not all files cached
- Hit rate < 90% → Increase memory or max files

**Detection:**
```bash
# Get opcache status (MOST IMPORTANT METRICS!)
php -r "print_r(opcache_get_status());" | grep -E "opcache_enabled|memory_usage|opcache_statistics|num_cached_scripts|hits|misses|blacklist_misses"

# Calculate hit rate
stats=$(php -r '$s=opcache_get_status(); echo $s["opcache_statistics"]["hits"].",".$s["opcache_statistics"]["misses"];')
hits=$(echo $stats | cut -d',' -f1)
misses=$(echo $stats | cut -d',' -f2)
total=$((hits + misses))
hit_rate=$((hits * 100 / total))
echo "Opcache Hit Rate: ${hit_rate}%"

# If hit rate < 90% → Need more memory or max_files!
```

### 5. **Session Settings**

```ini
session.save_handler = files           # files | memcached | redis
session.save_path = "/var/lib/php/session"
session.gc_maxlifetime = 1440          # Session timeout (seconds)
session.gc_probability = 1
session.gc_divisor = 1000              # GC runs 1/1000 requests
session.cookie_lifetime = 0            # Session cookie expires on browser close
```

**Why Track:**
- `session.save_path` full disk → Session writes fail
- Using `files` on high-traffic → I/O bottleneck (use Redis!)

### 6. **Error Handling & Logging**

```ini
error_reporting = E_ALL & ~E_DEPRECATED & ~E_STRICT
display_errors = Off                   # CRITICAL: Must be Off in production!
display_startup_errors = Off
log_errors = On                        # Log to file
error_log = /home/$user/logs/php_error.log
ignore_repeated_errors = Off
ignore_repeated_source = Off
report_memleaks = On
```

**Why Track:**
- `display_errors = On` in production → Security risk (exposes paths)
- No `error_log` set → Errors go to Apache log (harder to track)

### 7. **Security Settings**

```ini
; Disable Dangerous Functions
disable_functions = exec,passthru,shell_exec,system,proc_open,popen,curl_exec,curl_multi_exec,parse_ini_file,show_source

; Open Basedir (Restrict File Access)
open_basedir = /home/$user:/tmp       # Prevent directory traversal

; File Uploads
file_uploads = On
upload_tmp_dir = /tmp                  # Temp upload directory

; Misc Security
expose_php = Off                       # Hide PHP version in headers
allow_url_fopen = On                   # Allow remote file access (needed for many apps)
allow_url_include = Off                # CRITICAL: Prevent remote code execution
```

### 8. **APCu Cache** (User Cache, separate from OPcache)

```ini
[apcu]
apc.enabled = 1
apc.shm_size = 32M                     # Shared memory size
apc.ttl = 7200                         # Time to live
apc.gc_ttl = 3600                      # Garbage collection TTL
apc.enable_cli = 0
```

**Why Track:**
- WordPress object cache, WooCommerce, etc. use APCu
- Low hit rate → Increase shm_size

### 9. **MySQL/Database Settings** (php.ini side)

```ini
mysqli.max_persistent = -1             # Max persistent connections (-1 = unlimited)
mysqli.max_links = -1                  # Max total connections
mysqli.default_socket = /var/lib/mysql/mysql.sock
pdo_mysql.default_socket = /var/lib/mysql/mysql.sock
```

### 10. **Zend Extensions**

```ini
zend_extension=opcache.so
zend_extension=ioncube_loader_lin_8.2.so  # If using IonCube
```

## Complete Metrics Tracking List

### Per-Domain Tracking Matrix

```yaml
domain: example.com
user: examplec
php_version: ea-php82

config_hierarchy:
  1_user_ini: /home/examplec/public_html/.user.ini
  2_pool_ini: /opt/cpanel/ea-php82/root/etc/php.ini
  3_pool_d: /opt/cpanel/ea-php82/root/etc/php.d/
  4_global: /etc/php.ini

effective_settings:
  # Memory
  memory_limit: 256M
  upload_max_filesize: 64M
  post_max_size: 128M
  max_input_vars: 1000
  realpath_cache_size: 4096K

  # Execution
  max_execution_time: 30
  max_input_time: 60
  request_terminate_timeout: 30

  # PHP-FPM Pool
  pm: dynamic
  pm.max_children: 50
  pm.start_servers: 5
  pm.min_spare_servers: 5
  pm.max_spare_servers: 35
  pm.max_requests: 500
  pm.process_idle_timeout: 10s

  # OPcache
  opcache.enable: 1
  opcache.memory_consumption: 128M
  opcache.max_accelerated_files: 10000
  opcache.jit: tracing
  opcache.jit_buffer_size: 100M

  # Sessions
  session.save_handler: redis
  session.save_path: "tcp://127.0.0.1:6379"

  # Security
  display_errors: Off
  open_basedir: /home/examplec:/tmp
  disable_functions: exec,passthru,shell_exec

live_metrics:
  # Process Stats
  current_processes: 12
  avg_memory_per_process: 45MB
  total_memory_usage: 540MB
  cpu_usage: 15%

  # OPcache Stats
  opcache_hit_rate: 95.3%
  opcache_memory_used: 87MB / 128MB
  opcache_cached_scripts: 2847 / 10000
  opcache_wasted_memory: 2.1MB

  # Traffic Stats (last 24h)
  peak_concurrent_requests: 18
  avg_requests_per_minute: 45
  total_requests: 64,800

  # Error Stats (last 7 days)
  memory_exhausted: 0
  max_execution_time: 3
  max_children_reached: 47  # CRITICAL!
  slow_requests: 12

issues_detected:
  - type: CRITICAL
    code: MAX_CHILDREN_REACHED
    count: 47
    message: "pm.max_children limit hit 47 times in 7 days"
    recommendation: "Increase from 50 to 75"

  - type: WARNING
    code: SLOW_REQUESTS
    count: 12
    message: "12 requests took > 5 seconds"
    recommendation: "Review slow log, optimize code"

recommendations:
  - priority: HIGH
    setting: pm.max_children
    current: 50
    recommended: 75
    reason: "Peak concurrent (18) + buffer (50%) + safety margin"
    impact: "Handle 75 concurrent PHP requests vs 50"
    memory_impact: +1.1GB

  - priority: MEDIUM
    setting: opcache.max_accelerated_files
    current: 10000
    recommended: 15000
    reason: "Currently caching 2847 files, room for growth"
    impact: "Better cache coverage as site grows"
```

## Detection Commands Cheat Sheet

```bash
# Find ALL php.ini files affecting a domain
find /opt/cpanel/ea-php*/root/etc/ -name "php.ini"
find /home/$user/public_html -name ".user.ini"

# Find FPM pool config
grep -r "pool.*$domain" /opt/cpanel/ea-php*/root/etc/php-fpm.d/

# Get effective settings for domain
su -s /bin/bash $user -c "php -r 'phpinfo();'" | grep -A1 "memory_limit"

# Check opcache status
php -r "var_dump(opcache_get_status());"

# Find max_children errors
grep -r "max_children" /opt/cpanel/ea-php*/root/usr/var/log/php-fpm/

# Find slow requests
find /opt/cpanel/ea-php*/root/usr/var/log/php-fpm/ -name "*slow.log" -exec tail -50 {} \;

# Count current FPM processes
ps aux | grep "php-fpm: pool $domain" | wc -l

# Memory per process
ps aux | grep "php-fpm: pool $domain" | awk '{sum+=$6} END {print sum/NR " KB avg per process"}'
```

This comprehensive tracking will allow us to build an intelligent optimizer that knows EXACTLY what to fix!