cschantz c8ebe4b0f0 Phase 2: Advanced analytics for loadwatch-analyzer - predictive and trend analysis
PHASE 2 ENHANCEMENTS (5 new features):

1. LOAD TREND DIRECTION ANALYSIS
   - Analyzes 1min vs 5min vs 15min load averages
   - Detects RISING (problem worsening), FALLING (resolving), or STABLE
   - Provides snapshot counts for each trend type
   - Critical for understanding if issue is active or resolving

2. CONNECTION STATE BREAKDOWN
   - Parses network connection states from logs
   - Aggregates by state (ESTABLISHED, SYN_RECV, CLOSE_WAIT, TIME_WAIT, etc)
   - Shows average and total counts per state
   - Detects:
     * SYN flood attacks (high SYN_RECV)
     * Connection leaks (high CLOSE_WAIT)
     * Excessive TIME_WAIT (may need tuning)

3. MEMORY GROWTH VELOCITY TRACKING
   - Calculates rate of memory consumption change
   - Tracks MiB/hour growth or decline
   - Predicts time until OOM if memory is declining
   - Proactive alert: "Memory declining - OOM predicted in X hours"
   - Shows whether memory is stable, increasing, or declining

4. R-STATE PROCESS COUNT
   - Counts runnable (R-state) processes waiting for CPU
   - Better CPU pressure metric than load average alone
   - R-state > CPU cores = CPU contention
   - Detects:
     * Severe CPU pressure (R-state > 10)
     * Moderate contention (R-state > 5)
     * Normal range (R-state <= 5)

5. MYSQL THREAD ANOMALY DETECTION
   - Parses summary line mysql[current/expected] format
   - Alerts when current > 3x expected threads
   - Shows anomaly delta (extra threads)
   - Detects connection storms and thread explosions
   - Tracks httpd process count for correlation

REPORT SECTIONS ADDED:
- MySQL Thread Anomaly alerts in Critical Alerts section
- Memory Growth Velocity in Memory Analysis section
- Load Trend Direction in CPU & Load Analysis section
- CPU Pressure Analysis (R-state) - new dedicated section
- Network Connection Analysis - new dedicated section

PARSING ENHANCEMENTS:
- Enhanced summary line parsing for mysql[X/Y] format
- R-state process counting from top output
- Network state aggregation from network stats section
- Httpd count tracking for trending

ANALYSIS IMPROVEMENTS:
- Predictive OOM warnings based on memory velocity
- Trend-based load analysis (not just absolute values)
- State-specific network connection warnings
- CPU pressure quantification via R-state

IMPACT:
- Shifts from reactive (what happened) to predictive (what will happen)
- Provides trend analysis for problem resolution tracking
- Detects attacks and leaks from connection state patterns
- Better CPU pressure understanding via R-state metrics
- MySQL connection storm early warning system

All features tested and validated on production logs.
2025-11-20 21:50:16 -05:00
2025-11-03 18:14:51 -05:00

Linux Server Management Toolkit

Comprehensive multi-panel server management suite supporting cPanel, InterWorx, Plesk, and standalone Apache with modular architecture and intelligent security features.

📦 Directory Structure

server-toolkit/
├── launcher.sh                          # Main menu system
├── README.md                            # This file
│
├── modules/                             # Modular scripts organized by category
│   │
│   ├── diagnostics/                     # 🔍 System Diagnostics
│   │   ├── system-health-check.sh      # Comprehensive health analysis
│   │   └── loadwatch-analyzer.sh       # System health from loadwatch monitoring logs
│   │
│   ├── security/                        # 🛡️ Security & Threat Analysis
│   │   ├── bot-analyzer.sh             # Full bot/threat analysis
│   │   ├── live-attack-monitor.sh      # Real-time attack monitoring dashboard
│   │   ├── ssh-attack-monitor.sh       # SSH brute force detection
│   │   ├── web-traffic-monitor.sh      # Web traffic monitoring
│   │   ├── firewall-activity-monitor.sh # CSF/iptables monitoring
│   │   ├── enable-cphulk.sh            # cPHulk enablement with CSF whitelist import
│   │   ├── ip-reputation-manager.sh    # Centralized IP reputation tracking
│   │   └── tail-*.sh                   # Various log monitoring scripts
│   │
│   ├── backup/                          # 💾 Backup & Recovery (Acronis Cyber Protect)
│   │   ├── acronis-backup-manager.sh   # Main backup management menu
│   │   ├── acronis-install.sh          # Install Acronis agent
│   │   ├── acronis-update.sh           # Update Acronis agent
│   │   ├── acronis-uninstall.sh        # Uninstall Acronis agent
│   │   ├── acronis-register.sh         # Register agent with cloud
│   │   ├── acronis-configure.sh        # Configure agent settings
│   │   ├── acronis-agent-status.sh     # Comprehensive agent status check
│   │   ├── acronis-trigger-backup.sh   # Trigger manual backups with optimizations
│   │   ├── acronis-backup-status.sh    # Check backup job status
│   │   ├── acronis-list-backups.sh     # List all backups
│   │   ├── acronis-plan-manager.sh     # Manage protection plans
│   │   ├── acronis-schedule-viewer.sh  # View backup schedules
│   │   ├── acronis-restore.sh          # Restore from backup
│   │   ├── acronis-logs.sh             # View Acronis logs
│   │   └── acronis-troubleshoot.sh     # Troubleshoot common issues
│   │
│   ├── website/                         # 🌐 Website Diagnostics & Troubleshooting
│   │   ├── website-error-analyzer.sh   # Comprehensive website error analysis
│   │   └── 500-error-tracker.sh        # Track and analyze 500 errors
│   │
│   ├── diagnostics/                     # 🔍 System Diagnostics & Log Analysis
│   │   ├── system-health-check.sh      # Comprehensive health analysis
│   │   └── loadwatch-analyzer.sh       # System health monitoring from loadwatch logs
│   │
│   ├── performance/                     # 📊 Performance Analysis
│   │   ├── hardware-health-check.sh    # Hardware diagnostics
│   │   ├── mysql-query-analyzer.sh     # MySQL performance analysis
│   │   ├── network-bandwidth-analyzer.sh # Network analysis
│   │   └── (other performance modules)
│   │
│   └── maintenance/                     # 🧹 System Maintenance
│       └── cleanup-toolkit-data.sh     # Clean temporary toolkit data
│
├── lib/                                 # Shared libraries
│   ├── common-functions.sh             # Reusable functions
│   ├── system-detect.sh                # System type detection
│   ├── user-manager.sh                 # User account management
│   ├── mysql-analyzer.sh               # MySQL utilities
│   └── reference-db.sh                 # Cross-module intelligence sharing
│
├── config/                              # Configuration files
│   ├── settings.conf                   # Main configuration
│   ├── whitelist-ips.txt               # IP whitelist
│   └── whitelist-user-agents.txt       # User-Agent whitelist
│
└── tools/                               # Utility scripts
    ├── diagnostic-report.sh            # Generate system reports
    └── test-*.sh                       # Testing utilities

🚀 Quick Start

Installation & Running

One command - automatic cleanup:

curl -sL https://git.mull.lol/cschantz/Linux-Server-Management-Toolkit/archive/main.tar.gz | tar xz && source linux-server-management-toolkit/run.sh

When exiting (option 0), answer "yes" and cleanup happens automatically - no extra steps.

Or if already downloaded:

source /root/server-toolkit/run.sh

Key Features

🛡️ Security & Threat Analysis

  • 3-Mode Security Menu: Analysis / Actions / Live Monitoring
  • Live Attack Monitor: Real-time SOC dashboard with threat classification
  • Intelligent cPHulk Setup: Auto-imports CSF whitelists from all sources
  • IP Reputation Tracking: Centralized cross-module IP intelligence
  • Multi-Source Monitoring: SSH, Web, Firewall, cPHulk integration

💾 Backup & Recovery (Acronis Cyber Protect)

  • Complete Agent Management: Install, update, uninstall, register
  • Comprehensive Status Monitoring: Agent health, registration, cloud connectivity
  • Manual Backup Triggering: CLI-managed plans with performance optimizations
  • Backup Type Selection: Full, Incremental, Differential backups
  • Plan Management: View, enable/disable, delete protection plans
  • Restore Operations: Full restore capabilities from backups
  • Troubleshooting Tools: Log viewing and automated diagnostics

🌐 Website Diagnostics

  • Error Analysis: Comprehensive website error detection and troubleshooting
  • 500 Error Tracking: Detailed analysis of application errors
  • Log Integration: Apache, PHP-FPM, cPanel error log analysis
  • Smart Recommendations: Context-aware suggestions for fixing issues

🔍 System Diagnostics & Performance Monitoring

  • Comprehensive Health Checks: Hardware, services, security posture
  • Loadwatch Health Analyzer: Historical system health analysis from monitoring logs
    • Time-range analysis: 1h, 6h, 24h, 7d, 30d
    • Memory pressure detection and swap usage trending
    • CPU saturation analysis (idle, iowait, steal time)
    • Process issue detection (zombies, high CPU/MEM consumers)
    • MySQL performance monitoring
    • Actionable recommendations based on findings
  • Smart Recommendations: Context-aware suggestions based on findings
  • Multi-Panel Support: cPanel, InterWorx, Plesk, standalone Apache

📊 Session Intelligence

  • Reference Database: Cross-module data sharing (.sysref)
  • No Historical Tracking: Session-based intelligence only
  • "Download, Run, Fix, Delete": Designed for one-time troubleshooting

🎯 Usage Examples

Security Analysis with Live Monitoring

bash launcher.sh
# Select: Security & Threat Analysis
# Select: Live Monitoring & Alerts
# Select: Live Network Security Monitor

Enable cPHulk with CSF Whitelist

bash launcher.sh
# Select: Security & Threat Analysis
# Select: Security Actions & Fixes
# Select: Authentication Security
# Select: Enable cPHulk Protection

Acronis Backup Management

bash launcher.sh
# Select: Backup & Recovery
# Select: Check Agent Status (view health, registration, connectivity)
# Select: Trigger Manual Backup (with type selection and optimizations)
# Select: Manage Protection Plans

Website Error Analysis

bash launcher.sh
# Select: Website Diagnostics & Troubleshooting
# Select: Website Error Analyzer
# Choose a cPanel user account to analyze

System Health Check

bash launcher.sh
# Select: System Diagnostics
# Select: System Health Check

Loadwatch System Health Analysis

bash launcher.sh
# Select: Performance & Diagnostics
# Select: Loadwatch Health Analyzer
# Choose time range: 1h, 6h, 24h, 7d, or 30d

🔧 Configuration

Edit the configuration file:

nano /root/server-toolkit/config/settings.conf

🔒 Security Considerations

  • Run as root: Most modules require root access
  • Credentials stored safely: Git credentials in ~/.git-credentials (outside project)
  • No sensitive data in repo: .gitignore excludes keys, tokens, credentials
  • Test first: Try on non-production environments first

📊 Recent Updates (v2.2)

Multi-Control Panel Support (NEW!)

  • Full cPanel support (primary platform - production ready)
  • InterWorx support (validated on real servers - production ready)
  • Plesk support (validated on real servers - production ready)
  • Standalone Apache support (basic functionality)
  • 38/38 modules refactored for multi-panel architecture (100% complete)
  • Automated validation scripts for InterWorx and Plesk (13 and 15 tests)
  • All critical paths verified on production systems

System Detection & Abstraction

  • Automatic control panel detection (system-detect.sh)
  • Multi-panel user/domain management abstraction (user-manager.sh)
  • Dynamic log discovery for all panel types
  • Panel-specific path handling (docroots, logs, configs)
  • Zero hardcoded paths - all detection-based

Backup & Recovery

  • Complete Acronis Cyber Protect integration (16 management scripts)
  • Agent installation, registration, and update automation
  • Comprehensive status monitoring (health, registration, connectivity)
  • Manual backup triggering with performance optimizations
  • Protection plan management and scheduling

Website Diagnostics

  • Comprehensive website error analyzer (multi-panel)
  • 500 error tracking and troubleshooting (multi-panel)
  • Multi-log integration (Apache, PHP-FPM, all panels)
  • Smart error detection and recommendations

Security Enhancements

  • Bot analyzer with multi-panel log discovery
  • Live attack monitor supporting all control panels
  • Malware scanner with panel-aware docroot detection
  • Centralized IP reputation tracking
  • Real-time threat detection and classification

Core Infrastructure

  • Modular architecture with organized category structure
  • Reference database for cross-module intelligence
  • Comprehensive developer documentation (REFDB_FORMAT.txt)
  • Production validation complete for all major panels

🙏 Credits

Built for comprehensive cPanel/Linux server management with a focus on security and intelligent automation.


Version: 2.1.0 Repository: https://git.mull.lol/cschantz/Linux-Server-Management-Toolkit

S
Description
No description provided
Readme MIT 18 MiB
Languages
Shell 100%