Consolidate documentation into single reference file
Changes: - Updated REFDB_FORMAT.txt with all current information (2025-11-07) - Deleted 8 unnecessary/outdated .md files: - AUDIT-REPORT.md (old audit) - COMPREHENSIVE_AUDIT_REPORT.md (old audit) - DEVELOPMENT-GUIDELINES.md (merged into REFDB_FORMAT.txt) - PROJECT-STRUCTURE.md (outdated structure info) - SESSION_INTELLIGENCE.md (old design doc) - SETUP_GUIDE.md (old setup info) - TROUBLESHOOTING.md (info now in REFDB_FORMAT.txt) - WHATS_NEW.md (old changelog) Documentation Structure Now: - README.md: User-facing documentation (keep) - REFDB_FORMAT.txt: Developer/Claude reference (keep) REFDB_FORMAT.txt Updates: - Current status snapshot (2025-11-07) - WordPress cron manager documentation - Cancel button standards (mandatory) - Module template with cancel options - Git workflow guidelines - Recent commits log - Complete file structure map - Quick reference sections Benefits: - Single source of truth for development - No confusion between multiple docs - Easier to maintain and keep current - Clear separation: users read README, developers read REFDB_FORMAT https://claude.com/claude-code
This commit is contained in:
cschantz
-197
@@ -1,197 +0,0 @@
|
|||||||
# Server Toolkit - Audit Report
|
|
||||||
**Date:** 2025-10-31
|
|
||||||
**Status:** Production Ready (with notes)
|
|
||||||
|
|
||||||
## ✅ PASSING CHECKS
|
|
||||||
|
|
||||||
### Syntax Validation
|
|
||||||
All shell scripts pass `bash -n` syntax check:
|
|
||||||
- ✓ launcher.sh
|
|
||||||
- ✓ lib/common-functions.sh
|
|
||||||
- ✓ lib/system-detect.sh
|
|
||||||
- ✓ lib/user-manager.sh
|
|
||||||
- ✓ lib/reference-db.sh
|
|
||||||
- ✓ lib/mysql-analyzer.sh
|
|
||||||
- ✓ modules/security/bot-analyzer.sh
|
|
||||||
- ✓ modules/performance/mysql-query-analyzer.sh
|
|
||||||
- ✓ test-domain-detection.sh
|
|
||||||
- ✓ diagnostic-report.sh
|
|
||||||
|
|
||||||
### File Permissions
|
|
||||||
All scripts have correct execute permissions (755).
|
|
||||||
|
|
||||||
### Core Functionality
|
|
||||||
- ✓ Domain detection working
|
|
||||||
- ✓ User selection with arrow-key menu working
|
|
||||||
- ✓ Search functionality working
|
|
||||||
- ✓ Cleanup/Reset function working
|
|
||||||
- ✓ System detection working
|
|
||||||
- ✓ Bot analyzer working
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## ⚠️ INCOMPLETE MODULES
|
|
||||||
|
|
||||||
The following menu categories exist but have NO implemented scripts:
|
|
||||||
|
|
||||||
### 1. WordPress Management (Option 2)
|
|
||||||
**Menu shows 11 options, but ALL scripts missing:**
|
|
||||||
- wp-health-check.sh
|
|
||||||
- wp-cron-status.sh
|
|
||||||
- wp-cron-mass-fix.sh
|
|
||||||
- wp-cron-mass-create.sh
|
|
||||||
- wp-plugin-audit.sh
|
|
||||||
- wp-theme-audit.sh
|
|
||||||
- wp-mass-update.sh
|
|
||||||
- wp-malware-scan.sh
|
|
||||||
- wp-cleanup-spam.sh
|
|
||||||
- wp-mass-delete.sh
|
|
||||||
- wp-mass-backup.sh
|
|
||||||
|
|
||||||
**Impact:** Users clicking options 1-11 will see "Module not found" error.
|
|
||||||
|
|
||||||
### 2. Backup & Recovery (Option 4)
|
|
||||||
**Menu shows 7 options, all missing:**
|
|
||||||
- auto-backup.sh
|
|
||||||
- restore-backup.sh
|
|
||||||
- backup-mysql.sh
|
|
||||||
- backup-files.sh
|
|
||||||
- backup-config.sh
|
|
||||||
- backup-schedule.sh
|
|
||||||
- backup-verify.sh
|
|
||||||
|
|
||||||
### 3. Monitoring & Alerts (Option 5)
|
|
||||||
**Menu shows 5 options, all missing:**
|
|
||||||
- live-traffic.sh
|
|
||||||
- resource-monitor.sh
|
|
||||||
- error-log-watcher.sh
|
|
||||||
- alert-setup.sh
|
|
||||||
- uptime-monitor.sh
|
|
||||||
|
|
||||||
### 4. Troubleshooting & Diagnostics (Option 6)
|
|
||||||
**Menu shows 9 options, all missing:**
|
|
||||||
- error-hunter.sh
|
|
||||||
- slow-query-finder.sh
|
|
||||||
- disk-space-analyzer.sh
|
|
||||||
- permission-fixer.sh
|
|
||||||
- dns-tester.sh
|
|
||||||
- ssl-cert-checker.sh
|
|
||||||
- email-delivery-test.sh
|
|
||||||
- connection-tester.sh
|
|
||||||
- system-health.sh
|
|
||||||
|
|
||||||
### 5. Reporting & Analytics (Option 7)
|
|
||||||
**Menu shows 6 options, all missing:**
|
|
||||||
- server-report.sh
|
|
||||||
- security-audit.sh
|
|
||||||
- performance-report.sh
|
|
||||||
- usage-analytics.sh
|
|
||||||
- export-to-pdf.sh
|
|
||||||
- email-report.sh
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📋 RECOMMENDATIONS
|
|
||||||
|
|
||||||
### For Distribution NOW:
|
|
||||||
**Option A - Disable Incomplete Menus:**
|
|
||||||
Comment out or remove menu options 2, 4, 5, 6, 7 from launcher.sh.
|
|
||||||
Only show:
|
|
||||||
- Option 1: Security & Threat Analysis (WORKS - has bot-analyzer)
|
|
||||||
- Option 3: Performance (WORKS - has mysql-query-analyzer)
|
|
||||||
- Option 8: Cleanup/Reset (WORKS)
|
|
||||||
- Option 9: Configuration (WORKS)
|
|
||||||
|
|
||||||
### For Future Development:
|
|
||||||
1. Implement scripts one category at a time
|
|
||||||
2. Test each script before uncommenting menu option
|
|
||||||
3. Update WHATS_NEW.md when adding new modules
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🗂️ CLEAN FILE STRUCTURE
|
|
||||||
|
|
||||||
Current structure (cleaned):
|
|
||||||
```
|
|
||||||
server-toolkit/
|
|
||||||
├── launcher.sh ✓
|
|
||||||
├── diagnostic-report.sh ✓
|
|
||||||
├── test-domain-detection.sh ✓
|
|
||||||
├── README.md ✓
|
|
||||||
├── TROUBLESHOOTING.md ✓
|
|
||||||
├── SETUP_GUIDE.md ✓
|
|
||||||
├── WHATS_NEW.md ✓
|
|
||||||
├── REFDB_FORMAT.txt ✓
|
|
||||||
├── config/
|
|
||||||
│ ├── settings.conf ✓
|
|
||||||
│ ├── whitelist-ips.txt ✓
|
|
||||||
│ └── whitelist-user-agents.txt ✓
|
|
||||||
├── lib/
|
|
||||||
│ ├── common-functions.sh ✓
|
|
||||||
│ ├── system-detect.sh ✓
|
|
||||||
│ ├── user-manager.sh ✓
|
|
||||||
│ ├── reference-db.sh ✓
|
|
||||||
│ └── mysql-analyzer.sh ✓
|
|
||||||
└── modules/
|
|
||||||
├── security/
|
|
||||||
│ └── bot-analyzer.sh ✓ (WORKING)
|
|
||||||
├── performance/
|
|
||||||
│ └── mysql-query-analyzer.sh ✓ (WORKING)
|
|
||||||
├── wordpress/ (EMPTY - future)
|
|
||||||
├── backup/ (EMPTY - future)
|
|
||||||
├── monitoring/ (EMPTY - future)
|
|
||||||
├── troubleshooting/ (EMPTY - future)
|
|
||||||
└── reporting/ (EMPTY - future)
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## ✅ CLEANED FILES
|
|
||||||
|
|
||||||
Removed during audit:
|
|
||||||
- ❌ install.sh (unnecessary - users pull complete folder)
|
|
||||||
- ❌ .REFDB_FORMAT.txt (duplicate/outdated)
|
|
||||||
- ❌ .INTERACTIVE_MODE.txt (unknown old file)
|
|
||||||
- ❌ bot-analyzer.sh.backup (leftover from edits)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🎯 PRODUCTION READINESS
|
|
||||||
|
|
||||||
**Status: READY** for distribution with caveats:
|
|
||||||
|
|
||||||
### What Works Now (Production Ready):
|
|
||||||
1. ✅ Bot Analyzer (full-featured, tested)
|
|
||||||
2. ✅ MySQL Query Analyzer
|
|
||||||
3. ✅ Domain detection
|
|
||||||
4. ✅ User selection with search
|
|
||||||
5. ✅ Cleanup/Reset tools
|
|
||||||
6. ✅ Diagnostic reporting
|
|
||||||
|
|
||||||
### What to Do Before Public Release:
|
|
||||||
1. **Disable incomplete menu options** in launcher.sh (or clearly mark as "Coming Soon")
|
|
||||||
2. **Update README.md** to list only working features
|
|
||||||
3. **Add installation instructions** to README.md
|
|
||||||
|
|
||||||
### Suggested README.md Updates:
|
|
||||||
```markdown
|
|
||||||
## Current Features
|
|
||||||
- ✅ Bot & Botnet Analysis (comprehensive security scanning)
|
|
||||||
- ✅ MySQL Query Performance Analysis
|
|
||||||
- 🚧 WordPress Management (coming soon)
|
|
||||||
- 🚧 Backup & Recovery (coming soon)
|
|
||||||
- 🚧 Monitoring & Alerts (coming soon)
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📝 NEXT STEPS
|
|
||||||
|
|
||||||
1. Review incomplete menus in launcher.sh (lines 145-260)
|
|
||||||
2. Either:
|
|
||||||
- Comment out incomplete options
|
|
||||||
- OR add "(Coming Soon)" labels
|
|
||||||
3. Update README.md with current features only
|
|
||||||
4. Consider adding ROADMAP.md for planned features
|
|
||||||
|
|
||||||
**Bottom line:** The toolkit core is solid and production-ready. Just need to manage user expectations about incomplete features.
|
|
||||||
@@ -1,750 +0,0 @@
|
|||||||
# SERVER TOOLKIT - COMPREHENSIVE AUDIT REPORT
|
|
||||||
**Date:** 2025-11-01
|
|
||||||
**Auditor:** Claude (Sonnet 4.5)
|
|
||||||
**Audit Type:** Full Codebase Security, Functionality, and Data Integrity Review
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## EXECUTIVE SUMMARY
|
|
||||||
|
|
||||||
### Overall Health: **GOOD** ✓
|
|
||||||
- **Syntax:** All 13 shell scripts pass `bash -n` validation
|
|
||||||
- **Critical Bugs Found:** 2 (both fixed during audit)
|
|
||||||
- **Security Issues:** 0 critical, minor improvements recommended
|
|
||||||
- **Missing Features:** Several identified and documented
|
|
||||||
- **Data Integrity:** Reference database comprehensive, minor enhancements recommended
|
|
||||||
|
|
||||||
### Key Findings
|
|
||||||
1. ✅ **FIXED:** Missing `show_banner()` and `press_enter()` functions in common-functions.sh
|
|
||||||
2. ✅ **FIXED:** Cleanup function incomplete - missing new report file patterns
|
|
||||||
3. ⚠️ **ENHANCEMENT NEEDED:** Reference database could track network/hardware metrics
|
|
||||||
4. ✅ **VERIFIED:** System detection working correctly
|
|
||||||
5. ✅ **VERIFIED:** Cleanup/reset functionality now comprehensive
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. CODE STRUCTURE AUDIT
|
|
||||||
|
|
||||||
### Directory Organization: **EXCELLENT** ✓
|
|
||||||
```
|
|
||||||
/root/server-toolkit/
|
|
||||||
├── launcher.sh ✓ Main entry point
|
|
||||||
├── lib/ ✓ 5 library files
|
|
||||||
│ ├── common-functions.sh ✓ Shared utilities
|
|
||||||
│ ├── system-detect.sh ✓ Platform detection
|
|
||||||
│ ├── user-manager.sh ✓ User selection
|
|
||||||
│ ├── reference-db.sh ✓ Data caching
|
|
||||||
│ └── mysql-analyzer.sh ✓ MySQL utilities
|
|
||||||
├── modules/ ✓ Organized by category
|
|
||||||
│ ├── diagnostics/ ✓ 1 module (system-health-check.sh)
|
|
||||||
│ ├── performance/ ✓ 3 modules (mysql, network, hardware)
|
|
||||||
│ ├── security/ ✓ 1 module (bot-analyzer.sh)
|
|
||||||
│ └── [6 other categories] ⚠️ Placeholder directories
|
|
||||||
├── config/ ✓ Configuration files
|
|
||||||
├── tools/ ✓ Utility scripts
|
|
||||||
└── [Documentation] ✓ Comprehensive docs
|
|
||||||
```
|
|
||||||
|
|
||||||
### File Count
|
|
||||||
- **Total Scripts:** 13
|
|
||||||
- **Working Modules:** 5
|
|
||||||
- **Library Files:** 5
|
|
||||||
- **Config Files:** 3
|
|
||||||
- **Documentation:** 7 files
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2. SYNTAX AND CODE QUALITY
|
|
||||||
|
|
||||||
### Syntax Validation: **PASS** ✓
|
|
||||||
All scripts validated with `bash -n`:
|
|
||||||
```bash
|
|
||||||
✓ launcher.sh
|
|
||||||
✓ lib/common-functions.sh
|
|
||||||
✓ lib/system-detect.sh
|
|
||||||
✓ lib/user-manager.sh
|
|
||||||
✓ lib/reference-db.sh
|
|
||||||
✓ lib/mysql-analyzer.sh
|
|
||||||
✓ modules/diagnostics/system-health-check.sh
|
|
||||||
✓ modules/performance/mysql-query-analyzer.sh
|
|
||||||
✓ modules/performance/network-bandwidth-analyzer.sh
|
|
||||||
✓ modules/performance/hardware-health-check.sh
|
|
||||||
✓ modules/security/bot-analyzer.sh
|
|
||||||
✓ tools/test-domain-detection.sh
|
|
||||||
✓ tools/diagnostic-report.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### Code Standards
|
|
||||||
- ✅ Consistent bash strict mode (`set -eo pipefail`)
|
|
||||||
- ✅ Proper error handling with `|| true` on grep/find
|
|
||||||
- ✅ Safe variable substitution (`${var:-default}`)
|
|
||||||
- ✅ Proper arithmetic (`current=$((current + 1))`)
|
|
||||||
- ✅ No unsafe practices (eval, unescaped variables in SQL)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3. CRITICAL BUGS FOUND AND FIXED
|
|
||||||
|
|
||||||
### BUG #1: Missing Common Functions
|
|
||||||
**Severity:** HIGH
|
|
||||||
**Impact:** New modules (network-bandwidth-analyzer.sh, hardware-health-check.sh) would fail when calling `show_banner()` and `press_enter()`
|
|
||||||
**Location:** `lib/common-functions.sh`
|
|
||||||
|
|
||||||
**Problem:**
|
|
||||||
```bash
|
|
||||||
# These functions were called but not defined:
|
|
||||||
show_banner() # Called by new modules
|
|
||||||
press_enter() # Called by new modules
|
|
||||||
```
|
|
||||||
|
|
||||||
**Solution Applied:**
|
|
||||||
```bash
|
|
||||||
# Added to common-functions.sh:
|
|
||||||
press_enter() {
|
|
||||||
echo ""
|
|
||||||
read -p "Press Enter to continue..." _
|
|
||||||
}
|
|
||||||
|
|
||||||
show_banner() {
|
|
||||||
if [ -n "$1" ]; then
|
|
||||||
print_banner "$1"
|
|
||||||
else
|
|
||||||
print_banner "Server Toolkit"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Status:** ✅ FIXED
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### BUG #2: Incomplete Cleanup Function
|
|
||||||
**Severity:** MEDIUM
|
|
||||||
**Impact:** Cleanup/reset would not remove new report files, leaving orphaned data
|
|
||||||
**Location:** `launcher.sh:266-375`
|
|
||||||
|
|
||||||
**Problem:**
|
|
||||||
```bash
|
|
||||||
# Missing cleanup patterns for:
|
|
||||||
- /tmp/system_health_report_*
|
|
||||||
- /tmp/network_bandwidth_report_*
|
|
||||||
- /tmp/hardware_health_report_*
|
|
||||||
```
|
|
||||||
|
|
||||||
**Solution Applied:**
|
|
||||||
```bash
|
|
||||||
# Added to cleanup_all_data():
|
|
||||||
find /tmp -maxdepth 1 -name "system_health_report_*" -exec rm -f {} \;
|
|
||||||
find /tmp -maxdepth 1 -name "network_bandwidth_report_*" -exec rm -f {} \;
|
|
||||||
find /tmp -maxdepth 1 -name "hardware_health_report_*" -exec rm -f {} \;
|
|
||||||
```
|
|
||||||
|
|
||||||
**Status:** ✅ FIXED
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 4. CLEANUP/RESET FUNCTIONALITY AUDIT
|
|
||||||
|
|
||||||
### Comprehensive Coverage: **EXCELLENT** ✓
|
|
||||||
|
|
||||||
The cleanup function now removes:
|
|
||||||
1. ✅ System reference database (`.sysref`, `.sysref.timestamp`)
|
|
||||||
2. ✅ Temporary session directories (`/tmp/server-toolkit-*`)
|
|
||||||
3. ✅ Bot analyzer reports (`/tmp/bot_analysis_*`)
|
|
||||||
4. ✅ MySQL analysis reports (`/tmp/mysql_analysis_*`)
|
|
||||||
5. ✅ System health reports (`/tmp/system_health_report_*`) - **NEW**
|
|
||||||
6. ✅ Network bandwidth reports (`/tmp/network_bandwidth_report_*`) - **NEW**
|
|
||||||
7. ✅ Hardware health reports (`/tmp/hardware_health_report_*`) - **NEW**
|
|
||||||
8. ✅ Generic toolkit temp files (`/tmp/toolkit_*`)
|
|
||||||
9. ✅ All cache files (`/tmp/*.cache`, `/root/server-toolkit/*.cache`)
|
|
||||||
10. ✅ Environment variables (all `SYS_*` vars)
|
|
||||||
11. ✅ Function definitions (forces library reload)
|
|
||||||
12. ✅ Re-initialization with fresh detection
|
|
||||||
|
|
||||||
### What is Preserved (Correct): **VERIFIED** ✓
|
|
||||||
- ✅ Configuration files (`config/settings.conf`)
|
|
||||||
- ✅ User whitelists (`config/whitelist-ips.txt`, `config/whitelist-user-agents.txt`)
|
|
||||||
- ✅ Scripts themselves
|
|
||||||
- ✅ Server data (websites, databases, user files)
|
|
||||||
|
|
||||||
### Cleanup Completeness Score: **100%** ✓
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 5. REFERENCE DATABASE AUDIT
|
|
||||||
|
|
||||||
### Current Structure: **COMPREHENSIVE** ✓
|
|
||||||
|
|
||||||
**Tracked Data Types:**
|
|
||||||
1. ✅ **SYSTEM** - Control panel, OS, web server, database, PHP versions, hostname, CPU cores
|
|
||||||
2. ✅ **USERS** - Username, primary domain, DB count, domain count, disk usage, home directory
|
|
||||||
3. ✅ **DATABASES** - DB name, owner, domain, size, table count
|
|
||||||
4. ✅ **DOMAINS** - Domain, owner, document root, log path, PHP version, type, aliases
|
|
||||||
5. ✅ **WORDPRESS** - Domain, owner, path, DB name, DB user, version, plugin count, theme count
|
|
||||||
6. ✅ **LOGS** - Currently disabled (performance reasons)
|
|
||||||
7. ✅ **HEALTH_BASELINE** - System metrics, resource usage, service status, issue counts
|
|
||||||
|
|
||||||
### Health Baseline Metrics (Comprehensive): ✓
|
|
||||||
```
|
|
||||||
HEALTH|TIMESTAMP|datetime
|
|
||||||
HEALTH|MEMORY_TOTAL_MB|value|date
|
|
||||||
HEALTH|MEMORY_USED_PERCENT|value|date
|
|
||||||
HEALTH|CPU_LOAD_1MIN|value|date
|
|
||||||
HEALTH|CPU_CORES|value|date
|
|
||||||
HEALTH|DISK_USED_PERCENT|value|date
|
|
||||||
HEALTH|IOWAIT_PERCENT|value|date
|
|
||||||
HEALTH|EMAIL_QUEUE_SIZE|value|date
|
|
||||||
HEALTH|ZOMBIE_PROCESSES|value|date
|
|
||||||
HEALTH|HTTPD_STATUS|status|date
|
|
||||||
HEALTH|MYSQL_STATUS|status|date
|
|
||||||
HEALTH|FIREWALL_STATUS|status|date
|
|
||||||
HEALTH|CRITICAL_ISSUES|count|date
|
|
||||||
HEALTH|HIGH_ISSUES|count|date
|
|
||||||
HEALTH|MEDIUM_ISSUES|count|date
|
|
||||||
HEALTH|LOW_ISSUES|count|date
|
|
||||||
```
|
|
||||||
|
|
||||||
### Missing Data (Recommendations):
|
|
||||||
|
|
||||||
#### 🔍 NETWORK METRICS (Should be added)
|
|
||||||
```
|
|
||||||
HEALTH|NETWORK_INTERFACE|eth0|date
|
|
||||||
HEALTH|NETWORK_MTU|1500|date
|
|
||||||
HEALTH|NETWORK_RX_ERRORS|0|date
|
|
||||||
HEALTH|NETWORK_TX_ERRORS|0|date
|
|
||||||
HEALTH|NETWORK_RX_DROPPED|0|date
|
|
||||||
HEALTH|NETWORK_TX_DROPPED|0|date
|
|
||||||
HEALTH|TCP_RETRANS_PERCENT|12.89|date
|
|
||||||
HEALTH|PACKET_LOSS_PERCENT|0|date
|
|
||||||
```
|
|
||||||
|
|
||||||
**Rationale:** Network analyzer collects this data but doesn't store for trending
|
|
||||||
|
|
||||||
#### 🔍 HARDWARE METRICS (Should be added)
|
|
||||||
```
|
|
||||||
HEALTH|DISK_SMART_STATUS|PASSED|/dev/sda|date
|
|
||||||
HEALTH|DISK_REALLOCATED_SECTORS|0|/dev/sda|date
|
|
||||||
HEALTH|DISK_PENDING_SECTORS|0|/dev/sda|date
|
|
||||||
HEALTH|DISK_TEMPERATURE|35|/dev/sda|date
|
|
||||||
HEALTH|MEMORY_ECC_ERRORS|0|date
|
|
||||||
HEALTH|CPU_MCE_ERRORS|0|date
|
|
||||||
HEALTH|RAID_STATUS|optimal|date
|
|
||||||
```
|
|
||||||
|
|
||||||
**Rationale:** Hardware health check should save baseline for failure prediction
|
|
||||||
|
|
||||||
#### 🔍 SECURITY METRICS (Should be added)
|
|
||||||
```
|
|
||||||
HEALTH|SSH_FAILED_ATTEMPTS|10210|date
|
|
||||||
HEALTH|TOP_ATTACKER_IP|128.14.227.179|date
|
|
||||||
HEALTH|CPHULK_STATUS|enabled|date
|
|
||||||
HEALTH|CPHULK_BLOCKED_IPS|0|date
|
|
||||||
```
|
|
||||||
|
|
||||||
**Rationale:** Security baseline for attack trend analysis
|
|
||||||
|
|
||||||
#### 🔍 SERVICE RESPONSE TIMES (Optional - Advanced)
|
|
||||||
```
|
|
||||||
HEALTH|APACHE_RESPONSE_TIME_MS|150|date
|
|
||||||
HEALTH|MYSQL_RESPONSE_TIME_MS|25|date
|
|
||||||
HEALTH|DNS_RESPONSE_TIME_MS|10|date
|
|
||||||
```
|
|
||||||
|
|
||||||
**Rationale:** Performance baseline for degradation detection
|
|
||||||
|
|
||||||
### Cache Freshness: **OPTIMAL** ✓
|
|
||||||
- TTL: 1 hour (3600 seconds)
|
|
||||||
- Auto-rebuild on stale access
|
|
||||||
- Manual rebuild available
|
|
||||||
- Timestamp tracking working
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 6. MODULE FUNCTIONALITY AUDIT
|
|
||||||
|
|
||||||
### Working Modules (5/49 = 10%)
|
|
||||||
|
|
||||||
#### 1. System Health Check ✓ **EXCELLENT**
|
|
||||||
- **Location:** `modules/diagnostics/system-health-check.sh`
|
|
||||||
- **Phases:** 22 comprehensive analysis phases
|
|
||||||
- **Features:** Severity scoring, baseline tracking, cPHulkd integration
|
|
||||||
- **Recent Enhancements:** Hardware error proactivity, cPanel-specific recommendations
|
|
||||||
- **Issues:** None found
|
|
||||||
- **Score:** 10/10
|
|
||||||
|
|
||||||
#### 2. Bot Analyzer ✓ **EXCELLENT**
|
|
||||||
- **Location:** `modules/security/bot-analyzer.sh`
|
|
||||||
- **Features:** Threat scoring, CSF blocking, domain analysis, botnet detection
|
|
||||||
- **Issues:** None found
|
|
||||||
- **Score:** 10/10
|
|
||||||
|
|
||||||
#### 3. MySQL Query Analyzer ✓ **GOOD**
|
|
||||||
- **Location:** `modules/performance/mysql-query-analyzer.sh`
|
|
||||||
- **Features:** Slow query detection, live monitoring
|
|
||||||
- **Issues:** None found
|
|
||||||
- **Score:** 9/10
|
|
||||||
|
|
||||||
#### 4. Network & Bandwidth Analyzer ✓ **EXCELLENT** (NEW)
|
|
||||||
- **Location:** `modules/performance/network-bandwidth-analyzer.sh`
|
|
||||||
- **Features:** vnstat integration, per-domain traffic, connection analysis, MTU checks
|
|
||||||
- **Testing:** ✅ Validated during audit
|
|
||||||
- **Bugs Found:** 2 (fixed - missing functions)
|
|
||||||
- **Score:** 9/10 (deducted 1 for initial bugs)
|
|
||||||
|
|
||||||
#### 5. Hardware Health Check ✓ **EXCELLENT** (NEW)
|
|
||||||
- **Location:** `modules/performance/hardware-health-check.sh`
|
|
||||||
- **Features:** SMART disk health, memory ECC, CPU MCE, RAID status
|
|
||||||
- **Testing:** ✅ Syntax validated
|
|
||||||
- **Bugs Found:** 1 (fixed - missing functions)
|
|
||||||
- **Score:** 9/10 (deducted 1 for initial bugs)
|
|
||||||
|
|
||||||
### Not Implemented (44 modules)
|
|
||||||
See menu structure - all other menu options are placeholders
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 7. ERROR HANDLING AND EDGE CASES
|
|
||||||
|
|
||||||
### Error Handling Patterns: **EXCELLENT** ✓
|
|
||||||
|
|
||||||
**Grep Safety:**
|
|
||||||
```bash
|
|
||||||
# All grep commands properly handled:
|
|
||||||
result=$(grep "pattern" file 2>/dev/null || true)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Find Safety:**
|
|
||||||
```bash
|
|
||||||
# All find commands have error suppression:
|
|
||||||
files=$(find /path -name "*.txt" 2>/dev/null || true)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Arithmetic Safety:**
|
|
||||||
```bash
|
|
||||||
# All arithmetic uses safe patterns:
|
|
||||||
current=$((current + 1)) # NOT ((current++))
|
|
||||||
```
|
|
||||||
|
|
||||||
**Variable Safety:**
|
|
||||||
```bash
|
|
||||||
# All potentially unbound vars use defaults:
|
|
||||||
${var:-default}
|
|
||||||
${var:-}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Edge Cases Handled:
|
|
||||||
- ✅ No users on system
|
|
||||||
- ✅ No databases
|
|
||||||
- ✅ No domains
|
|
||||||
- ✅ No WordPress installations
|
|
||||||
- ✅ Missing system commands (smartctl, dmidecode, vnstat, sensors)
|
|
||||||
- ✅ Non-cPanel systems
|
|
||||||
- ✅ Empty log files
|
|
||||||
- ✅ Stale reference database
|
|
||||||
- ✅ First-time execution
|
|
||||||
- ✅ Interrupted execution (cleanup temp dirs)
|
|
||||||
|
|
||||||
### Edge Cases NOT Handled (Minor):
|
|
||||||
- ⚠️ Very large reference database (>100MB) - no size limiting
|
|
||||||
- ⚠️ Systems with >10,000 users - progress indicators may be slow
|
|
||||||
- ⚠️ Extremely large log files (>10GB) - analysis may timeout
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 8. SECURITY AUDIT
|
|
||||||
|
|
||||||
### Security Posture: **GOOD** ✓
|
|
||||||
|
|
||||||
**Secure Practices:**
|
|
||||||
- ✅ No `eval` usage
|
|
||||||
- ✅ No unquoted variables in command execution
|
|
||||||
- ✅ Proper MySQL query escaping (using `-e` flag, not string interpolation)
|
|
||||||
- ✅ Temp file creation uses `mktemp`
|
|
||||||
- ✅ No passwords stored in plain text
|
|
||||||
- ✅ No credentials in code
|
|
||||||
- ✅ Proper file permissions checks before operations
|
|
||||||
- ✅ Root requirement explicitly checked
|
|
||||||
|
|
||||||
**Potential Concerns (Minor):**
|
|
||||||
- ⚠️ Some temp files in `/tmp` not using `mktemp -d` (report files use predictable names)
|
|
||||||
- **Risk:** Low (reports contain public system info only)
|
|
||||||
- **Recommendation:** Consider using `mktemp` for all temp files
|
|
||||||
|
|
||||||
- ⚠️ CSF commands run without input validation
|
|
||||||
- **Risk:** Low (only called with controlled input from script)
|
|
||||||
- **Recommendation:** Add IP format validation before CSF calls
|
|
||||||
|
|
||||||
### Privilege Escalation: **SECURE** ✓
|
|
||||||
- ✅ Requires root (appropriate for system management)
|
|
||||||
- ✅ No unnecessary privilege dropping
|
|
||||||
- ✅ No unsafe sudo usage
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 9. SYSTEM DETECTION ACCURACY
|
|
||||||
|
|
||||||
### Detection Coverage: **COMPREHENSIVE** ✓
|
|
||||||
|
|
||||||
**Control Panels:**
|
|
||||||
- ✅ cPanel (tested)
|
|
||||||
- ✅ Plesk (code reviewed)
|
|
||||||
- ✅ InterWorx (code reviewed)
|
|
||||||
- ✅ None/Standalone (code reviewed)
|
|
||||||
|
|
||||||
**Operating Systems:**
|
|
||||||
- ✅ AlmaLinux (tested)
|
|
||||||
- ✅ CentOS, RHEL, Rocky, CloudLinux (code reviewed)
|
|
||||||
|
|
||||||
**Web Servers:**
|
|
||||||
- ✅ Apache (tested)
|
|
||||||
- ✅ Nginx, LiteSpeed, OpenLiteSpeed (code reviewed)
|
|
||||||
|
|
||||||
**Databases:**
|
|
||||||
- ✅ MariaDB (tested)
|
|
||||||
- ✅ MySQL (code reviewed)
|
|
||||||
- ✅ None (handled)
|
|
||||||
|
|
||||||
**PHP Detection:**
|
|
||||||
- ✅ Multiple versions (tested - found 8.0.30, 8.1.33, 8.2.29)
|
|
||||||
|
|
||||||
### Detection Accuracy: **100%** ✓
|
|
||||||
All detection on test system correct:
|
|
||||||
- Control Panel: cPanel 11.130.0.15 ✓
|
|
||||||
- OS: AlmaLinux 9.6 ✓
|
|
||||||
- Web Server: Apache 2.4.65 ✓
|
|
||||||
- Database: MariaDB 10.6.23 ✓
|
|
||||||
- Hostname: cloudvpstemplate.host.pickledperil.com ✓
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 10. MISSING FEATURES AND RECOMMENDATIONS
|
|
||||||
|
|
||||||
### High Priority Additions
|
|
||||||
|
|
||||||
#### 1. Network Metrics in Reference Database
|
|
||||||
**Why:** Network analyzer collects but doesn't persist data for trending
|
|
||||||
**Impact:** Cannot compare current vs historical network performance
|
|
||||||
**Implementation:** Add `save_network_baseline()` function to health check
|
|
||||||
**Effort:** Low (2-3 hours)
|
|
||||||
|
|
||||||
#### 2. Hardware Metrics in Reference Database
|
|
||||||
**Why:** Hardware health check should track SMART data over time
|
|
||||||
**Impact:** Cannot predict disk failures by tracking reallocated sector trends
|
|
||||||
**Implementation:** Add `save_hardware_baseline()` function to health check
|
|
||||||
**Effort:** Medium (4-6 hours)
|
|
||||||
|
|
||||||
#### 3. Security Metrics in Reference Database
|
|
||||||
**Why:** SSH attack trends not tracked
|
|
||||||
**Impact:** Cannot identify escalating attack patterns
|
|
||||||
**Implementation:** Add security metrics to health baseline
|
|
||||||
**Effort:** Low (2-3 hours)
|
|
||||||
|
|
||||||
#### 4. Reference Database Size Limiting
|
|
||||||
**Why:** No upper limit on database size
|
|
||||||
**Impact:** Could grow unbounded on very large systems
|
|
||||||
**Implementation:** Add rotation/pruning for old HEALTH entries
|
|
||||||
**Effort:** Medium (3-4 hours)
|
|
||||||
|
|
||||||
### Medium Priority Additions
|
|
||||||
|
|
||||||
#### 5. Better Error Messages for Missing Commands
|
|
||||||
**Why:** Some modules just say "not installed" without context
|
|
||||||
**Impact:** User may not understand which package to install
|
|
||||||
**Implementation:** Add package name hints (e.g., "smartctl not found - install smartmontools")
|
|
||||||
**Effort:** Low (1-2 hours)
|
|
||||||
|
|
||||||
#### 6. Progress Indicators for Long Operations
|
|
||||||
**Why:** Some operations (disk scanning) provide no feedback
|
|
||||||
**Impact:** User may think script hung
|
|
||||||
**Implementation:** Add progress indicators to hardware health check
|
|
||||||
**Effort:** Low (2 hours)
|
|
||||||
|
|
||||||
#### 7. Report Archiving
|
|
||||||
**Why:** Reports accumulate in /tmp indefinitely
|
|
||||||
**Impact:** /tmp bloat
|
|
||||||
**Implementation:** Archive old reports or auto-delete after 7 days
|
|
||||||
**Effort:** Low (2 hours)
|
|
||||||
|
|
||||||
### Low Priority (Nice to Have)
|
|
||||||
|
|
||||||
#### 8. Bandwidth Quota Tracking
|
|
||||||
**Why:** Network analyzer doesn't track against hosting limits
|
|
||||||
**Implementation:** Allow user to set monthly bandwidth cap, alert on approaching
|
|
||||||
**Effort:** Medium (4 hours)
|
|
||||||
|
|
||||||
#### 9. Email Notifications
|
|
||||||
**Why:** No alerting when critical issues found
|
|
||||||
**Implementation:** Email reports to admin when CRITICAL issues detected
|
|
||||||
**Effort:** Medium (6 hours)
|
|
||||||
|
|
||||||
#### 10. Comparison Reports
|
|
||||||
**Why:** Can't easily see "what changed since last scan"
|
|
||||||
**Implementation:** Diff between current and previous health report
|
|
||||||
**Effort:** High (8-10 hours)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 11. DATA PERSISTENCE AND INTEGRITY
|
|
||||||
|
|
||||||
### Reference Database Integrity: **EXCELLENT** ✓
|
|
||||||
|
|
||||||
**Data Consistency:**
|
|
||||||
- ✅ Pipe-delimited format consistent
|
|
||||||
- ✅ Field counts consistent per record type
|
|
||||||
- ✅ No corrupted entries found
|
|
||||||
- ✅ Proper escaping (no pipes in data fields)
|
|
||||||
|
|
||||||
**Update Mechanism:**
|
|
||||||
- ✅ Atomic writes (write to new file, then move)
|
|
||||||
- ✅ Timestamp tracking working
|
|
||||||
- ✅ TTL enforcement working
|
|
||||||
- ✅ Rebuild on corruption (auto-triggered)
|
|
||||||
|
|
||||||
**Cross-References:**
|
|
||||||
- ✅ User → Domains working
|
|
||||||
- ✅ User → Databases working
|
|
||||||
- ✅ Domain → WordPress working
|
|
||||||
- ✅ Database → Owner working
|
|
||||||
|
|
||||||
### Data Not Being Persisted (Should Be):
|
|
||||||
|
|
||||||
1. **Network Performance Trends**
|
|
||||||
- Current: Measured each run, not saved
|
|
||||||
- Should: Track TCP retransmission rate over time
|
|
||||||
- Benefit: Identify network degradation trends
|
|
||||||
|
|
||||||
2. **Hardware Health Trends**
|
|
||||||
- Current: SMART checked each run, not saved
|
|
||||||
- Should: Track reallocated sectors over time
|
|
||||||
- Benefit: Predict disk failure before it happens
|
|
||||||
|
|
||||||
3. **Attack Pattern History**
|
|
||||||
- Current: Bot analyzer shows current attacks
|
|
||||||
- Should: Track attack volume over time
|
|
||||||
- Benefit: Identify coordinated/escalating attacks
|
|
||||||
|
|
||||||
4. **Service Response Times**
|
|
||||||
- Current: Not measured
|
|
||||||
- Should: Track Apache/MySQL response times
|
|
||||||
- Benefit: Identify performance degradation
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 12. TESTING RECOMMENDATIONS
|
|
||||||
|
|
||||||
### Current Testing: **MINIMAL**
|
|
||||||
- Unit tests: None
|
|
||||||
- Integration tests: None
|
|
||||||
- Manual testing: Ad-hoc during development
|
|
||||||
|
|
||||||
### Recommended Testing Strategy:
|
|
||||||
|
|
||||||
#### 1. Smoke Tests (Quick Validation)
|
|
||||||
```bash
|
|
||||||
#!/bin/bash
|
|
||||||
# tests/smoke-test.sh
|
|
||||||
bash -n /root/server-toolkit/launcher.sh || exit 1
|
|
||||||
bash -n /root/server-toolkit/lib/*.sh || exit 1
|
|
||||||
bash -n /root/server-toolkit/modules/*/*.sh || exit 1
|
|
||||||
echo "✓ All syntax valid"
|
|
||||||
```
|
|
||||||
|
|
||||||
#### 2. Integration Tests
|
|
||||||
```bash
|
|
||||||
# Test cleanup
|
|
||||||
rm -f .sysref*
|
|
||||||
./launcher.sh # Should rebuild database
|
|
||||||
grep "^USER|" .sysref || exit 1
|
|
||||||
echo "✓ Database rebuild working"
|
|
||||||
|
|
||||||
# Test cleanup
|
|
||||||
./launcher.sh # Choose option 8 (cleanup)
|
|
||||||
[ ! -f .sysref ] || exit 1
|
|
||||||
echo "✓ Cleanup working"
|
|
||||||
```
|
|
||||||
|
|
||||||
#### 3. Module Tests
|
|
||||||
- Test each module in isolation
|
|
||||||
- Test with missing dependencies
|
|
||||||
- Test with edge cases (no users, no domains, etc.)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 13. PERFORMANCE ANALYSIS
|
|
||||||
|
|
||||||
### Reference Database Build Time: **EXCELLENT** ✓
|
|
||||||
- Current system: ~2-3 seconds
|
|
||||||
- 100 users: ~10-15 seconds (estimated)
|
|
||||||
- 1000 users: ~60-90 seconds (estimated)
|
|
||||||
|
|
||||||
### Module Performance:
|
|
||||||
- System Health Check: **5-10 seconds** ✓
|
|
||||||
- Bot Analyzer: **30-60 seconds** (depends on log size) ✓
|
|
||||||
- MySQL Query Analyzer: **10-20 seconds** ✓
|
|
||||||
- Network Analyzer: **5-10 seconds** ✓
|
|
||||||
- Hardware Health Check: **10-15 seconds** (with smartctl) ✓
|
|
||||||
|
|
||||||
### Bottlenecks Identified:
|
|
||||||
1. ⚠️ `du -sm` on large home directories (>100GB) - can be slow
|
|
||||||
- **Recommendation:** Add timeout or use `du --max-depth=1`
|
|
||||||
|
|
||||||
2. ⚠️ WordPress detection (`find -name wp-config.php`) on large systems
|
|
||||||
- **Recommendation:** Limit search depth or use locate database
|
|
||||||
|
|
||||||
3. ⚠️ SMART checks on many disks (>10 disks)
|
|
||||||
- **Recommendation:** Parallelize or add progress indicator
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 14. DOCUMENTATION AUDIT
|
|
||||||
|
|
||||||
### Documentation Quality: **EXCELLENT** ✓
|
|
||||||
|
|
||||||
**Files Present:**
|
|
||||||
- ✅ README.md - Comprehensive overview
|
|
||||||
- ✅ TROUBLESHOOTING.md - Common issues and fixes
|
|
||||||
- ✅ AUDIT-REPORT.md - Previous audit
|
|
||||||
- ✅ PROJECT-STRUCTURE.md - Architecture docs
|
|
||||||
- ✅ SETUP_GUIDE.md - Installation instructions
|
|
||||||
- ✅ REFDB_FORMAT.txt - Reference database specification (EXCELLENT)
|
|
||||||
- ✅ WHATS_NEW.md - Changelog
|
|
||||||
|
|
||||||
**Missing Documentation:**
|
|
||||||
- ⚠️ API documentation for library functions
|
|
||||||
- ⚠️ Module development guide
|
|
||||||
- ⚠️ Contributing guidelines
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 15. FINAL RECOMMENDATIONS
|
|
||||||
|
|
||||||
### Must Do (Before Production)
|
|
||||||
1. ✅ **DONE** - Fix missing `show_banner()` and `press_enter()` functions
|
|
||||||
2. ✅ **DONE** - Fix cleanup function to remove all report types
|
|
||||||
3. 🔄 **ADD** - Network metrics to reference database
|
|
||||||
4. 🔄 **ADD** - Hardware metrics to reference database
|
|
||||||
5. 🔄 **ADD** - Input validation for CSF IP addresses
|
|
||||||
|
|
||||||
### Should Do (Near Term)
|
|
||||||
6. 🔄 Add reference database size limiting/rotation
|
|
||||||
7. 🔄 Add package name hints for missing commands
|
|
||||||
8. 🔄 Add progress indicators to hardware health check
|
|
||||||
9. 🔄 Create smoke test suite
|
|
||||||
10. 🔄 Add report archiving/cleanup
|
|
||||||
|
|
||||||
### Nice to Have (Future)
|
|
||||||
11. Bandwidth quota tracking and alerting
|
|
||||||
12. Email notifications for critical issues
|
|
||||||
13. Comparison reports (diff between scans)
|
|
||||||
14. Unit test coverage
|
|
||||||
15. API documentation
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 16. AUDIT SUMMARY
|
|
||||||
|
|
||||||
### Scores
|
|
||||||
|
|
||||||
| Category | Score | Status |
|
|
||||||
|----------|-------|--------|
|
|
||||||
| Code Quality | 95/100 | ✅ Excellent |
|
|
||||||
| Security | 90/100 | ✅ Good |
|
|
||||||
| Functionality | 85/100 | ✅ Good |
|
|
||||||
| Error Handling | 95/100 | ✅ Excellent |
|
|
||||||
| Documentation | 90/100 | ✅ Excellent |
|
|
||||||
| Testing | 40/100 | ⚠️ Needs Improvement |
|
|
||||||
| Performance | 85/100 | ✅ Good |
|
|
||||||
| Data Integrity | 95/100 | ✅ Excellent |
|
|
||||||
|
|
||||||
### Overall Score: **89/100** - **EXCELLENT** ✅
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 17. WHAT WE'RE NOT TRACKING (BUT SHOULD BE)
|
|
||||||
|
|
||||||
### Reference Database Gaps
|
|
||||||
|
|
||||||
1. **Network Performance History**
|
|
||||||
- TCP retransmission rate trends
|
|
||||||
- Packet loss over time
|
|
||||||
- Interface errors trending
|
|
||||||
- Bandwidth usage per day/week/month
|
|
||||||
|
|
||||||
2. **Hardware Health Trends**
|
|
||||||
- SMART attribute changes (reallocated sectors increasing?)
|
|
||||||
- Disk temperature trends
|
|
||||||
- Memory error accumulation
|
|
||||||
- CPU error history
|
|
||||||
|
|
||||||
3. **Security Event History**
|
|
||||||
- SSH attack volume trends
|
|
||||||
- Blocked IP history
|
|
||||||
- Attack pattern changes
|
|
||||||
- Geographic attack sources
|
|
||||||
|
|
||||||
4. **Service Availability**
|
|
||||||
- Service downtime tracking
|
|
||||||
- Restart frequency
|
|
||||||
- Error log growth rate
|
|
||||||
|
|
||||||
5. **Resource Usage Trends**
|
|
||||||
- Disk usage growth rate (predict when full)
|
|
||||||
- Memory usage patterns
|
|
||||||
- CPU load trends
|
|
||||||
- Email queue size trends
|
|
||||||
|
|
||||||
### Implementation Priority
|
|
||||||
|
|
||||||
**High Priority:**
|
|
||||||
- Network: TCP retransmission, packet loss
|
|
||||||
- Hardware: SMART reallocated sectors, disk temperature
|
|
||||||
- Security: SSH attack counts
|
|
||||||
|
|
||||||
**Medium Priority:**
|
|
||||||
- Service: Downtime tracking
|
|
||||||
- Resource: Disk growth rate
|
|
||||||
|
|
||||||
**Low Priority:**
|
|
||||||
- Advanced trending and prediction
|
|
||||||
- Anomaly detection
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 18. CHANGELOG (Audit Actions)
|
|
||||||
|
|
||||||
### Fixed During Audit:
|
|
||||||
1. **2025-11-01 16:35** - Added `show_banner()` function to lib/common-functions.sh
|
|
||||||
2. **2025-11-01 16:35** - Added `press_enter()` function to lib/common-functions.sh
|
|
||||||
3. **2025-11-01 16:38** - Added system_health_report_* cleanup to launcher.sh
|
|
||||||
4. **2025-11-01 16:38** - Added network_bandwidth_report_* cleanup to launcher.sh
|
|
||||||
5. **2025-11-01 16:38** - Added hardware_health_report_* cleanup to launcher.sh
|
|
||||||
6. **2025-11-01 16:38** - Updated cleanup message to list all report types
|
|
||||||
|
|
||||||
### Validated During Audit:
|
|
||||||
- ✅ All 13 scripts pass syntax validation
|
|
||||||
- ✅ System detection accurate (cPanel, AlmaLinux, Apache, MariaDB)
|
|
||||||
- ✅ Reference database format correct and complete
|
|
||||||
- ✅ Cleanup function comprehensive
|
|
||||||
- ✅ Error handling robust
|
|
||||||
- ✅ Security practices sound
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## CONCLUSION
|
|
||||||
|
|
||||||
The Server Toolkit is in **excellent** condition with only minor enhancements recommended. The codebase is well-structured, properly documented, and follows bash best practices. The two bugs found during audit were minor and have been fixed.
|
|
||||||
|
|
||||||
The main area for improvement is **data persistence** - while the toolkit collects comprehensive data, not all of it is being saved for historical trending. Adding network, hardware, and security metrics to the reference database would enable powerful trend analysis and predictive maintenance.
|
|
||||||
|
|
||||||
**Recommended Next Steps:**
|
|
||||||
1. Review and approve the fixes made during this audit
|
|
||||||
2. Implement network metrics persistence
|
|
||||||
3. Implement hardware metrics persistence
|
|
||||||
4. Add basic smoke tests
|
|
||||||
5. Consider adding email alerting for critical issues
|
|
||||||
|
|
||||||
**Overall Assessment:** ✅ **PRODUCTION READY** with recommended enhancements
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**End of Audit Report**
|
|
||||||
@@ -1,606 +0,0 @@
|
|||||||
# Server Management Toolkit - Development Guidelines
|
|
||||||
|
|
||||||
**Version:** 2.1.0
|
|
||||||
**Last Updated:** 2025-11-07
|
|
||||||
|
|
||||||
This document provides uniform standards and quick reference for developing and maintaining the Server Management Toolkit.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Table of Contents
|
|
||||||
|
|
||||||
1. [Project Structure & File Map](#project-structure--file-map)
|
|
||||||
2. [Script Design Standards](#script-design-standards)
|
|
||||||
3. [User Experience Guidelines](#user-experience-guidelines)
|
|
||||||
4. [Shared Resources](#shared-resources)
|
|
||||||
5. [Testing Guidelines](#testing-guidelines)
|
|
||||||
6. [Git Workflow](#git-workflow)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Project Structure & File Map
|
|
||||||
|
|
||||||
### Directory Layout
|
|
||||||
|
|
||||||
```
|
|
||||||
server-toolkit/
|
|
||||||
├── launcher.sh # Main entry point - menu system
|
|
||||||
├── README.md # User documentation
|
|
||||||
├── DEVELOPMENT-GUIDELINES.md # This file (developer reference)
|
|
||||||
│
|
|
||||||
├── lib/ # Shared libraries (NEVER modify without testing all scripts)
|
|
||||||
│ ├── common-functions.sh # Core utilities (print_*, press_enter, etc.)
|
|
||||||
│ ├── system-detect.sh # OS/cPanel detection (SYS_* variables)
|
|
||||||
│ ├── user-manager.sh # cPanel user functions (select_user_interactive, etc.)
|
|
||||||
│ ├── mysql-analyzer.sh # Database analysis utilities
|
|
||||||
│ ├── ip-reputation.sh # Centralized IP reputation tracking
|
|
||||||
│ └── reference-db.sh # Session-based cross-module intelligence (.sysref)
|
|
||||||
│
|
|
||||||
├── config/ # Configuration files
|
|
||||||
│ ├── settings.conf # Main configuration
|
|
||||||
│ ├── whitelist-ips.txt # IP whitelist
|
|
||||||
│ └── whitelist-user-agents.txt # User-Agent whitelist
|
|
||||||
│
|
|
||||||
├── modules/ # Modular scripts by category
|
|
||||||
│ ├── security/ # Security & threat analysis
|
|
||||||
│ │ ├── bot-analyzer.sh
|
|
||||||
│ │ ├── live-attack-monitor.sh
|
|
||||||
│ │ ├── enable-cphulk.sh
|
|
||||||
│ │ └── ip-reputation-manager.sh
|
|
||||||
│ │
|
|
||||||
│ ├── website/ # Website diagnostics
|
|
||||||
│ │ ├── website-error-analyzer.sh
|
|
||||||
│ │ ├── 500-error-tracker.sh
|
|
||||||
│ │ ├── wordpress-menu.sh # WordPress submenu
|
|
||||||
│ │ └── wordpress/ # WordPress-specific tools
|
|
||||||
│ │ └── wordpress-cron-manager.sh
|
|
||||||
│ │
|
|
||||||
│ ├── backup/ # Acronis Cyber Protect integration
|
|
||||||
│ │ ├── acronis-backup-manager.sh
|
|
||||||
│ │ ├── acronis-trigger-backup.sh
|
|
||||||
│ │ ├── acronis-agent-status.sh
|
|
||||||
│ │ └── [14 other acronis scripts]
|
|
||||||
│ │
|
|
||||||
│ ├── diagnostics/ # System diagnostics
|
|
||||||
│ │ └── system-health-check.sh
|
|
||||||
│ │
|
|
||||||
│ ├── performance/ # Performance analysis
|
|
||||||
│ │ ├── mysql-query-analyzer.sh
|
|
||||||
│ │ └── hardware-health-check.sh
|
|
||||||
│ │
|
|
||||||
│ └── maintenance/ # System maintenance
|
|
||||||
│ └── cleanup-toolkit-data.sh
|
|
||||||
│
|
|
||||||
└── tools/ # Utility scripts
|
|
||||||
└── diagnostic-report.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### Key File Locations Quick Reference
|
|
||||||
|
|
||||||
| Resource | Path | Purpose |
|
|
||||||
|----------|------|---------|
|
|
||||||
| **Main Launcher** | `/root/server-toolkit/launcher.sh` | Menu system entry point |
|
|
||||||
| **Reference DB** | `/root/server-toolkit/.sysref` | Session-based intelligence sharing |
|
|
||||||
| **Common Functions** | `/root/server-toolkit/lib/common-functions.sh` | Shared utilities all scripts use |
|
|
||||||
| **IP Reputation** | `/root/server-toolkit/lib/ip-reputation.sh` | Centralized IP tracking |
|
|
||||||
| **User Manager** | `/root/server-toolkit/lib/user-manager.sh` | cPanel user selection |
|
|
||||||
| **Website Errors** | `/root/server-toolkit/modules/website/website-error-analyzer.sh` | Intelligent error analysis |
|
|
||||||
| **WordPress Cron** | `/root/server-toolkit/modules/website/wordpress/wordpress-cron-manager.sh` | WP cron management |
|
|
||||||
| **Security Menu** | Search `show_security_menu` in launcher.sh | 3-tier security structure |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Script Design Standards
|
|
||||||
|
|
||||||
### Standard Script Template
|
|
||||||
|
|
||||||
Every module script should follow this structure:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
#!/bin/bash
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# Script Name
|
|
||||||
################################################################################
|
|
||||||
# Purpose: Clear description of what this script does
|
|
||||||
# Features:
|
|
||||||
# - Feature 1
|
|
||||||
# - Feature 2
|
|
||||||
# - Feature 3
|
|
||||||
################################################################################
|
|
||||||
|
|
||||||
# Determine correct path to root based on script location
|
|
||||||
# Example paths:
|
|
||||||
# modules/security/script.sh → ../../
|
|
||||||
# modules/website/script.sh → ../../
|
|
||||||
# modules/website/wordpress/script.sh → ../../../
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
source "$SCRIPT_DIR/lib/common-functions.sh"
|
|
||||||
source "$SCRIPT_DIR/lib/system-detect.sh"
|
|
||||||
|
|
||||||
# Optional libraries (only include if needed)
|
|
||||||
# source "$SCRIPT_DIR/lib/user-manager.sh"
|
|
||||||
# source "$SCRIPT_DIR/lib/ip-reputation.sh"
|
|
||||||
# source "$SCRIPT_DIR/lib/reference-db.sh"
|
|
||||||
|
|
||||||
# Root check
|
|
||||||
if [ "$EUID" -ne 0 ]; then
|
|
||||||
print_error "This script must be run as root"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Main script logic starts here
|
|
||||||
clear
|
|
||||||
print_banner "Script Name"
|
|
||||||
|
|
||||||
echo ""
|
|
||||||
# ... script functionality ...
|
|
||||||
echo ""
|
|
||||||
|
|
||||||
press_enter
|
|
||||||
```
|
|
||||||
|
|
||||||
### Path Resolution Rules
|
|
||||||
|
|
||||||
**CRITICAL:** Always verify `SCRIPT_DIR` calculation based on script location:
|
|
||||||
|
|
||||||
| Script Location | Correct SCRIPT_DIR |
|
|
||||||
|----------------|-------------------|
|
|
||||||
| `modules/category/script.sh` | `../../` (up 2 levels) |
|
|
||||||
| `modules/category/subcategory/script.sh` | `../../../` (up 3 levels) |
|
|
||||||
| `tools/script.sh` | `../` (up 1 level) |
|
|
||||||
|
|
||||||
**Test after moving scripts:**
|
|
||||||
```bash
|
|
||||||
bash /path/to/script.sh # Should load libraries without errors
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## User Experience Guidelines
|
|
||||||
|
|
||||||
### 1. Cancel/Back Options (MANDATORY)
|
|
||||||
|
|
||||||
**Every user input must allow cancellation:**
|
|
||||||
|
|
||||||
#### Menu Options
|
|
||||||
```bash
|
|
||||||
echo " 1) Option One"
|
|
||||||
echo " 2) Option Two"
|
|
||||||
echo " 3) Option Three"
|
|
||||||
echo " 0) Cancel and return to menu" # REQUIRED
|
|
||||||
echo ""
|
|
||||||
read -p "Select option: " choice
|
|
||||||
|
|
||||||
case $choice in
|
|
||||||
0)
|
|
||||||
echo ""
|
|
||||||
echo "Operation cancelled."
|
|
||||||
echo ""
|
|
||||||
exit 0
|
|
||||||
;;
|
|
||||||
# ... other options ...
|
|
||||||
esac
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Text Input Prompts
|
|
||||||
```bash
|
|
||||||
echo -n "Enter domain name (or 0 to cancel): "
|
|
||||||
read -r domain
|
|
||||||
|
|
||||||
if [ -z "$domain" ] || [ "$domain" = "0" ]; then
|
|
||||||
echo "Operation cancelled."
|
|
||||||
press_enter
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Confirmation Prompts
|
|
||||||
```bash
|
|
||||||
echo -n "Are you sure? (y/n) [n]: "
|
|
||||||
read -r confirm
|
|
||||||
|
|
||||||
if [ "$confirm" != "y" ] && [ "$confirm" != "Y" ]; then
|
|
||||||
echo "Operation cancelled."
|
|
||||||
press_enter
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. Consistent Messaging
|
|
||||||
|
|
||||||
Use these standard messages:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Success
|
|
||||||
print_success "Operation completed successfully"
|
|
||||||
|
|
||||||
# Error
|
|
||||||
print_error "Something went wrong"
|
|
||||||
|
|
||||||
# Warning
|
|
||||||
print_warning "This action cannot be undone"
|
|
||||||
|
|
||||||
# Info
|
|
||||||
print_info "Processing data..."
|
|
||||||
|
|
||||||
# Cancellation
|
|
||||||
echo "Operation cancelled."
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. Always Use press_enter
|
|
||||||
|
|
||||||
**REQUIRED at end of every script:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
echo ""
|
|
||||||
print_success "Task completed"
|
|
||||||
echo ""
|
|
||||||
press_enter # Gives user time to read output before returning to menu
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Shared Resources
|
|
||||||
|
|
||||||
### 1. Reference Database (.sysref)
|
|
||||||
|
|
||||||
**Session-based intelligence sharing between modules**
|
|
||||||
|
|
||||||
Location: `/root/server-toolkit/.sysref`
|
|
||||||
Purpose: Share discovered information during a single session
|
|
||||||
**NOT persistent** - cleared on cleanup
|
|
||||||
|
|
||||||
#### Usage:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
source "$SCRIPT_DIR/lib/reference-db.sh"
|
|
||||||
|
|
||||||
# Store WordPress installation
|
|
||||||
db_store_wordpress "domain.com" "username" "/home/user/public_html" "wp_db" "wp_user" "6.4" "15" "3"
|
|
||||||
|
|
||||||
# Retrieve WordPress installations
|
|
||||||
wp_sites=$(db_get_all_wordpress)
|
|
||||||
while IFS='|' read -r type domain owner path db_name db_user version plugins themes; do
|
|
||||||
echo "Found: $domain owned by $owner"
|
|
||||||
done <<< "$wp_sites"
|
|
||||||
|
|
||||||
# Query specific domain
|
|
||||||
wp_info=$(db_get_wordpress_by_domain "example.com")
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Available Functions:
|
|
||||||
|
|
||||||
| Function | Purpose |
|
|
||||||
|----------|---------|
|
|
||||||
| `db_store_wordpress` | Store WP installation details |
|
|
||||||
| `db_get_all_wordpress` | Get all WP sites |
|
|
||||||
| `db_get_wordpress_by_domain` | Get specific domain |
|
|
||||||
| `db_store_user_domain_map` | Map user to domains |
|
|
||||||
| `db_get_user_domains` | Get user's domains |
|
|
||||||
|
|
||||||
### 2. IP Reputation System
|
|
||||||
|
|
||||||
**Centralized IP tracking across all security modules**
|
|
||||||
|
|
||||||
Location: `/root/server-toolkit/lib/ip-reputation.sh`
|
|
||||||
|
|
||||||
#### Usage:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
source "$SCRIPT_DIR/lib/ip-reputation.sh"
|
|
||||||
|
|
||||||
# Check if IP is a known bot
|
|
||||||
if is_known_bot "1.2.3.4"; then
|
|
||||||
echo "This is a bot"
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Check if IP should be filtered
|
|
||||||
if should_filter_ip "1.2.3.4"; then
|
|
||||||
continue # Skip this IP
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Log threat detection
|
|
||||||
log_ip_threat "1.2.3.4" "SSH brute force" "80"
|
|
||||||
|
|
||||||
# Query reputation
|
|
||||||
rep_score=$(get_ip_reputation "1.2.3.4")
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. User Manager
|
|
||||||
|
|
||||||
**Consistent cPanel user selection**
|
|
||||||
|
|
||||||
Location: `/root/server-toolkit/lib/user-manager.sh`
|
|
||||||
|
|
||||||
#### Usage:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
source "$SCRIPT_DIR/lib/user-manager.sh"
|
|
||||||
|
|
||||||
# Interactive user selection
|
|
||||||
select_user_interactive "Select cPanel user to analyze"
|
|
||||||
if [ -n "$SELECTED_USER" ]; then
|
|
||||||
echo "Selected: $SELECTED_USER"
|
|
||||||
else
|
|
||||||
echo "No user selected"
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
# List all users
|
|
||||||
list_all_users
|
|
||||||
|
|
||||||
# Get user's domains
|
|
||||||
domains=$(get_user_domains "username")
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. Common Functions
|
|
||||||
|
|
||||||
**Core utilities all scripts use**
|
|
||||||
|
|
||||||
Location: `/root/server-toolkit/lib/common-functions.sh`
|
|
||||||
|
|
||||||
#### Essential Functions:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Output formatting
|
|
||||||
print_banner "My Script Title"
|
|
||||||
print_success "Operation completed"
|
|
||||||
print_error "Failed to complete"
|
|
||||||
print_warning "Proceed with caution"
|
|
||||||
print_info "Processing..."
|
|
||||||
|
|
||||||
# User interaction
|
|
||||||
press_enter # Wait for user before returning to menu
|
|
||||||
|
|
||||||
# Colors (use sparingly, prefer print_* functions)
|
|
||||||
echo -e "${GREEN}Success${NC}"
|
|
||||||
echo -e "${RED}Error${NC}"
|
|
||||||
echo -e "${YELLOW}Warning${NC}"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Testing Guidelines
|
|
||||||
|
|
||||||
### Before Committing
|
|
||||||
|
|
||||||
**ALWAYS test these scenarios:**
|
|
||||||
|
|
||||||
1. **Cancel Functionality**
|
|
||||||
```bash
|
|
||||||
# Test each user input accepts "0" to cancel
|
|
||||||
bash script.sh
|
|
||||||
# Enter 0 at each prompt
|
|
||||||
# Verify clean exit
|
|
||||||
```
|
|
||||||
|
|
||||||
2. **Path Resolution**
|
|
||||||
```bash
|
|
||||||
# Verify libraries load correctly
|
|
||||||
bash -n /path/to/script.sh # Syntax check
|
|
||||||
bash /path/to/script.sh # Run from anywhere
|
|
||||||
```
|
|
||||||
|
|
||||||
3. **Menu Navigation**
|
|
||||||
```bash
|
|
||||||
# Navigate: Main → Category → Script → Cancel
|
|
||||||
bash launcher.sh
|
|
||||||
# Select through menus
|
|
||||||
# Cancel and verify return to previous menu
|
|
||||||
```
|
|
||||||
|
|
||||||
4. **Error Handling**
|
|
||||||
```bash
|
|
||||||
# Test with invalid input
|
|
||||||
# Test with missing files
|
|
||||||
# Test with permission issues
|
|
||||||
```
|
|
||||||
|
|
||||||
### Test Checklist
|
|
||||||
|
|
||||||
- [ ] Script loads libraries without errors
|
|
||||||
- [ ] All menus have "0) Cancel" option
|
|
||||||
- [ ] All text inputs accept "0" to cancel
|
|
||||||
- [ ] `press_enter` called at end
|
|
||||||
- [ ] `print_banner` used for title
|
|
||||||
- [ ] Error messages use `print_error`
|
|
||||||
- [ ] Success messages use `print_success`
|
|
||||||
- [ ] Root check present
|
|
||||||
- [ ] No syntax errors (`bash -n script.sh`)
|
|
||||||
- [ ] Works when called from launcher
|
|
||||||
- [ ] Works when called directly
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Git Workflow
|
|
||||||
|
|
||||||
### Commit Message Template
|
|
||||||
|
|
||||||
```
|
|
||||||
Brief summary of changes (50 chars max)
|
|
||||||
|
|
||||||
Changes:
|
|
||||||
- Specific change 1
|
|
||||||
- Specific change 2
|
|
||||||
- Specific change 3
|
|
||||||
|
|
||||||
[Optional sections:]
|
|
||||||
User Experience Improvements:
|
|
||||||
- Improvement 1
|
|
||||||
- Improvement 2
|
|
||||||
|
|
||||||
Technical Details:
|
|
||||||
- Detail 1
|
|
||||||
- Detail 2
|
|
||||||
|
|
||||||
Tested:
|
|
||||||
✓ Test scenario 1
|
|
||||||
✓ Test scenario 2
|
|
||||||
✓ Test scenario 3
|
|
||||||
|
|
||||||
🤖 Generated with Claude Code
|
|
||||||
https://claude.com/claude-code
|
|
||||||
|
|
||||||
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
||||||
```
|
|
||||||
|
|
||||||
### Commit Workflow
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Check status
|
|
||||||
git status
|
|
||||||
|
|
||||||
# 2. Add all changes
|
|
||||||
git add -A
|
|
||||||
|
|
||||||
# 3. Commit with descriptive message
|
|
||||||
git commit -m "$(cat <<'EOF'
|
|
||||||
Your commit message here
|
|
||||||
EOF
|
|
||||||
)"
|
|
||||||
|
|
||||||
# 4. Push to remote
|
|
||||||
git push origin main
|
|
||||||
```
|
|
||||||
|
|
||||||
### What to Commit
|
|
||||||
|
|
||||||
**DO commit:**
|
|
||||||
- Script files (*.sh)
|
|
||||||
- Configuration templates
|
|
||||||
- Documentation (*.md)
|
|
||||||
- Library updates
|
|
||||||
|
|
||||||
**DON'T commit:**
|
|
||||||
- `.sysref` (session data)
|
|
||||||
- `.sysref.timestamp`
|
|
||||||
- Temporary files (`/tmp/*`)
|
|
||||||
- User-specific configs with credentials
|
|
||||||
- Cache files
|
|
||||||
|
|
||||||
### Commit Frequency
|
|
||||||
|
|
||||||
- After completing a feature
|
|
||||||
- After fixing bugs
|
|
||||||
- Before major refactoring
|
|
||||||
- At end of work session
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Menu Structure Standards
|
|
||||||
|
|
||||||
### Menu Hierarchy Rules
|
|
||||||
|
|
||||||
1. **Main Menu** → Categories (Security, Website, Performance, etc.)
|
|
||||||
2. **Category Menu** → Sub-categories or direct scripts
|
|
||||||
3. **Sub-Menu** → Specific tools or actions
|
|
||||||
4. **Script** → Actual functionality
|
|
||||||
|
|
||||||
### Adding New Scripts
|
|
||||||
|
|
||||||
#### 1. Create Script in Correct Category
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# modules/category/new-script.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
#### 2. Add to Category Menu
|
|
||||||
|
|
||||||
Edit `launcher.sh`:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Add to show_category_menu function
|
|
||||||
echo " X) New Script Name - Description"
|
|
||||||
|
|
||||||
# Add to handle_category_menu case statement
|
|
||||||
case $choice in
|
|
||||||
X) run_module "category" "new-script.sh" ;;
|
|
||||||
esac
|
|
||||||
```
|
|
||||||
|
|
||||||
#### 3. Update README.md
|
|
||||||
|
|
||||||
Add to feature list and structure documentation
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Quick Reference
|
|
||||||
|
|
||||||
### Common Tasks
|
|
||||||
|
|
||||||
| Task | Command/Location |
|
|
||||||
|------|------------------|
|
|
||||||
| Test script syntax | `bash -n script.sh` |
|
|
||||||
| Run specific script | `bash /root/server-toolkit/modules/category/script.sh` |
|
|
||||||
| Clear session data | Option 8 in main menu |
|
|
||||||
| View logs | Check `/tmp/` for script outputs |
|
|
||||||
| Check running shells | `/bashes` command |
|
|
||||||
| Force library reload | Re-source in script or restart launcher |
|
|
||||||
|
|
||||||
### File Naming Conventions
|
|
||||||
|
|
||||||
- Use kebab-case: `my-script-name.sh`
|
|
||||||
- Be descriptive: `wordpress-cron-manager.sh` not `wp-cron.sh`
|
|
||||||
- Match purpose: `enable-cphulk.sh` clearly states what it does
|
|
||||||
|
|
||||||
### Variable Naming
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Global constants (UPPERCASE)
|
|
||||||
APACHE_LOG_DIR="/var/log/apache2"
|
|
||||||
MAX_RETRIES=3
|
|
||||||
|
|
||||||
# Local variables (lowercase with underscores)
|
|
||||||
domain_name="example.com"
|
|
||||||
error_count=0
|
|
||||||
|
|
||||||
# Function names (lowercase with underscores)
|
|
||||||
check_wp_installation() {
|
|
||||||
# ...
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Troubleshooting Development Issues
|
|
||||||
|
|
||||||
### Script Can't Find Libraries
|
|
||||||
|
|
||||||
**Problem:** `source: file not found`
|
|
||||||
|
|
||||||
**Solution:**
|
|
||||||
```bash
|
|
||||||
# Check SCRIPT_DIR calculation
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
echo "SCRIPT_DIR: $SCRIPT_DIR"
|
|
||||||
|
|
||||||
# Verify from script location:
|
|
||||||
# modules/category/script.sh → goes up 2 levels
|
|
||||||
# modules/category/sub/script.sh → goes up 3 levels
|
|
||||||
```
|
|
||||||
|
|
||||||
### Menu Option Not Working
|
|
||||||
|
|
||||||
**Problem:** Script doesn't launch from menu
|
|
||||||
|
|
||||||
**Solution:**
|
|
||||||
1. Check `run_module` call in launcher.sh uses correct category and filename
|
|
||||||
2. Verify script is executable: `chmod +x script.sh`
|
|
||||||
3. Check script has `#!/bin/bash` shebang
|
|
||||||
|
|
||||||
### Cancel Not Working
|
|
||||||
|
|
||||||
**Problem:** User can't exit from prompt
|
|
||||||
|
|
||||||
**Solution:**
|
|
||||||
1. Add `0) Cancel` to all menus
|
|
||||||
2. Add `(or 0 to cancel)` to all text prompts
|
|
||||||
3. Check for empty or "0" input and exit cleanly
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Remember:** Consistency is key. Follow these guidelines for every new script and update existing scripts to match these standards as you work on them.
|
|
||||||
|
|
||||||
**When in doubt:** Look at `modules/website/wordpress/wordpress-cron-manager.sh` as a reference implementation that follows all standards.
|
|
||||||
@@ -1,130 +0,0 @@
|
|||||||
# Server Toolkit - Project Structure
|
|
||||||
|
|
||||||
## Directory Layout
|
|
||||||
|
|
||||||
```
|
|
||||||
server-toolkit/
|
|
||||||
├── launcher.sh # Main entry point
|
|
||||||
├── README.md # Project documentation
|
|
||||||
├── TROUBLESHOOTING.md # Troubleshooting guide
|
|
||||||
├── AUDIT-REPORT.md # Project audit results
|
|
||||||
├── REFDB_FORMAT.txt # Development notes & bug tracker
|
|
||||||
│
|
|
||||||
├── config/ # Configuration files
|
|
||||||
│ ├── settings.conf # Main configuration
|
|
||||||
│ ├── settings.conf.minimal # Minimal config (template)
|
|
||||||
│ ├── whitelist-ips.txt # IP whitelist for bot analyzer
|
|
||||||
│ └── whitelist-user-agents.txt # User-agent whitelist
|
|
||||||
│
|
|
||||||
├── lib/ # Core libraries
|
|
||||||
│ ├── common-functions.sh # Shared utilities (print, colors, etc.)
|
|
||||||
│ ├── system-detect.sh # Auto-detect control panel, OS, etc.
|
|
||||||
│ ├── user-manager.sh # User/domain selection functions
|
|
||||||
│ ├── reference-db.sh # System reference database builder
|
|
||||||
│ └── mysql-analyzer.sh # MySQL analysis functions
|
|
||||||
│
|
|
||||||
├── modules/ # Feature modules
|
|
||||||
│ ├── security/
|
|
||||||
│ │ └── bot-analyzer.sh # ✓ Bot & botnet analysis (WORKING)
|
|
||||||
│ ├── performance/
|
|
||||||
│ │ └── mysql-query-analyzer.sh # ✓ MySQL query analysis (WORKING)
|
|
||||||
│ ├── wordpress/ # (Empty - future development)
|
|
||||||
│ ├── backup/ # (Empty - future development)
|
|
||||||
│ ├── monitoring/ # (Empty - future development)
|
|
||||||
│ ├── troubleshooting/ # (Empty - future development)
|
|
||||||
│ └── reporting/ # (Empty - future development)
|
|
||||||
│
|
|
||||||
└── tools/ # Diagnostic & testing tools
|
|
||||||
├── diagnostic-report.sh # System diagnostic collector
|
|
||||||
└── test-domain-detection.sh # Domain detection validator
|
|
||||||
```
|
|
||||||
|
|
||||||
## File Purposes
|
|
||||||
|
|
||||||
### Root Level
|
|
||||||
- **launcher.sh** - Main menu system, calls modules
|
|
||||||
- **README.md** - User-facing documentation
|
|
||||||
- **TROUBLESHOOTING.md** - Help guide for common issues
|
|
||||||
- **AUDIT-REPORT.md** - Technical audit results (for developers)
|
|
||||||
- **REFDB_FORMAT.txt** - Development log, bug tracking, enhancement notes
|
|
||||||
|
|
||||||
### Config Directory
|
|
||||||
Contains user-configurable settings:
|
|
||||||
- **settings.conf** - Main config (includes unused future settings)
|
|
||||||
- **settings.conf.minimal** - Clean template with only current settings
|
|
||||||
- **whitelist-*.txt** - Bot analyzer whitelists
|
|
||||||
|
|
||||||
### Lib Directory
|
|
||||||
Core library functions sourced by modules:
|
|
||||||
- **common-functions.sh** - Colors, print functions, formatting
|
|
||||||
- **system-detect.sh** - Auto-detect environment (cPanel/Plesk/etc)
|
|
||||||
- **user-manager.sh** - User selection, domain detection
|
|
||||||
- **reference-db.sh** - Build/manage system reference database
|
|
||||||
- **mysql-analyzer.sh** - MySQL analysis helper functions
|
|
||||||
|
|
||||||
### Modules Directory
|
|
||||||
Feature implementations:
|
|
||||||
- **security/** - Security tools (bot analyzer, etc.)
|
|
||||||
- **performance/** - Performance tools (MySQL analyzer, etc.)
|
|
||||||
- **wordpress/** through **reporting/** - Placeholder for future
|
|
||||||
|
|
||||||
### Tools Directory
|
|
||||||
Diagnostic and testing utilities:
|
|
||||||
- **diagnostic-report.sh** - Generates comprehensive system report
|
|
||||||
- **test-domain-detection.sh** - Quick validation of domain detection
|
|
||||||
|
|
||||||
## Working Features
|
|
||||||
|
|
||||||
### Fully Implemented (✓)
|
|
||||||
1. **Bot & Botnet Analyzer** (`modules/security/bot-analyzer.sh`)
|
|
||||||
- Comprehensive log analysis
|
|
||||||
- Threat scoring
|
|
||||||
- IP blocking recommendations
|
|
||||||
- CSF integration
|
|
||||||
- Attack vector detection
|
|
||||||
|
|
||||||
2. **MySQL Query Analyzer** (`modules/performance/mysql-query-analyzer.sh`)
|
|
||||||
- Slow query detection
|
|
||||||
- Query performance analysis
|
|
||||||
|
|
||||||
3. **System Detection** (`lib/system-detect.sh`)
|
|
||||||
- Auto-detect: cPanel, Plesk, InterWorx
|
|
||||||
- OS, web server, database detection
|
|
||||||
- Resource monitoring
|
|
||||||
|
|
||||||
4. **User Management** (`lib/user-manager.sh`)
|
|
||||||
- Interactive user selection
|
|
||||||
- Arrow-key navigation
|
|
||||||
- Search with confirmation
|
|
||||||
- Domain detection
|
|
||||||
|
|
||||||
## In Development (Future)
|
|
||||||
|
|
||||||
- WordPress Management (11 planned scripts)
|
|
||||||
- Backup & Recovery (7 planned scripts)
|
|
||||||
- Monitoring & Alerts (5 planned scripts)
|
|
||||||
- Troubleshooting (9 planned scripts)
|
|
||||||
- Reporting (6 planned scripts)
|
|
||||||
|
|
||||||
See AUDIT-REPORT.md for complete list.
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
Most settings auto-detect on first run. Manual configuration available in:
|
|
||||||
- `config/settings.conf` - All settings (includes future features)
|
|
||||||
- `config/settings.conf.minimal` - Only current features
|
|
||||||
|
|
||||||
## Logs & Cache
|
|
||||||
|
|
||||||
Runtime files (auto-created):
|
|
||||||
- `.sysref` - System reference database cache
|
|
||||||
- `/tmp/bot_analysis_*.txt` - Bot analysis reports
|
|
||||||
- `/tmp/mysql_analysis_*.txt` - MySQL analysis reports
|
|
||||||
- `/tmp/server-toolkit-*` - Temporary session directories
|
|
||||||
|
|
||||||
## For Developers
|
|
||||||
|
|
||||||
Key technical documentation:
|
|
||||||
- **AUDIT-REPORT.md** - What's implemented vs. planned
|
|
||||||
- **REFDB_FORMAT.txt** - Bug fixes, enhancements, lessons learned
|
|
||||||
- **TROUBLESHOOTING.md** - Common issues and debug procedures
|
|
||||||
+428
-495
File diff suppressed because it is too large
Load Diff
@@ -1,283 +0,0 @@
|
|||||||
# SESSION INTELLIGENCE - Cross-Module Data Sharing
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
The Server Toolkit now implements **Session Intelligence** - allowing modules to reference data collected by other modules during the current troubleshooting session. This is optimized for the **download → diagnose → troubleshoot → delete** workflow.
|
|
||||||
|
|
||||||
## Use Case
|
|
||||||
|
|
||||||
Since the toolkit is meant to be temporary (not permanently installed), we don't track historical trends. Instead, we enable **cross-module intelligence** so modules can make smarter recommendations based on what's happening RIGHT NOW.
|
|
||||||
|
|
||||||
## Example Scenarios
|
|
||||||
|
|
||||||
### Scenario 1: Bot Attack During System Load
|
|
||||||
```bash
|
|
||||||
# User runs System Health Check first
|
|
||||||
# Discovers: CPU at 95%, Memory at 92%, HIGH LOAD
|
|
||||||
|
|
||||||
# User then runs Bot Analyzer
|
|
||||||
# Bot analyzer checks: db_is_system_under_load
|
|
||||||
# Result: "High bot traffic detected, but system is already under load.
|
|
||||||
# Performance issues may be partially due to system resources,
|
|
||||||
# not just bots. Recommend addressing system load first."
|
|
||||||
```
|
|
||||||
|
|
||||||
### Scenario 2: Slow MySQL During Network Issues
|
|
||||||
```bash
|
|
||||||
# User runs System Health Check
|
|
||||||
# Discovers: TCP retransmission at 15%, HIGH network issues
|
|
||||||
|
|
||||||
# User then runs MySQL Query Analyzer
|
|
||||||
# MySQL analyzer checks: db_has_network_issues
|
|
||||||
# Result: "Slow queries detected, but network is experiencing high
|
|
||||||
# retransmission rates. Some query timeouts may be network-
|
|
||||||
# related rather than database performance."
|
|
||||||
```
|
|
||||||
|
|
||||||
### Scenario 3: Bot Attack + SSH Brute Force
|
|
||||||
```bash
|
|
||||||
# User runs System Health Check
|
|
||||||
# Discovers: 5,000 failed SSH attempts today
|
|
||||||
|
|
||||||
# User then runs Bot Analyzer
|
|
||||||
# Bot analyzer checks: db_is_under_attack
|
|
||||||
# Result: "Bot traffic detected AND system is under active SSH attack.
|
|
||||||
# Recommend immediate firewall hardening and cPHulk enablement."
|
|
||||||
```
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
### Data Storage: Reference Database (`.sysref`)
|
|
||||||
|
|
||||||
The health check saves current session metrics to `[HEALTH_BASELINE]` section:
|
|
||||||
|
|
||||||
**System Resources:**
|
|
||||||
- MEMORY_TOTAL_MB, MEMORY_USED_PERCENT
|
|
||||||
- CPU_LOAD_1MIN, CPU_CORES
|
|
||||||
- DISK_USED_PERCENT, IOWAIT_PERCENT
|
|
||||||
|
|
||||||
**Services:**
|
|
||||||
- HTTPD_STATUS, MYSQL_STATUS
|
|
||||||
- FIREWALL_STATUS, EMAIL_QUEUE_SIZE
|
|
||||||
- ZOMBIE_PROCESSES
|
|
||||||
|
|
||||||
**Network Status:**
|
|
||||||
- NETWORK_INTERFACE, NETWORK_MTU
|
|
||||||
- NETWORK_RX_ERRORS, NETWORK_TX_ERRORS
|
|
||||||
- NETWORK_RX_DROPPED, NETWORK_TX_DROPPED
|
|
||||||
- TCP_RETRANS_PERCENT
|
|
||||||
|
|
||||||
**Hardware Status:**
|
|
||||||
- DISK_SMART_STATUS
|
|
||||||
- HARDWARE_ERRORS
|
|
||||||
|
|
||||||
**Security Status:**
|
|
||||||
- SSH_FAILED_ATTEMPTS_TOTAL
|
|
||||||
- SSH_ATTACKS_TODAY
|
|
||||||
- CPHULK_STATUS
|
|
||||||
|
|
||||||
**Issue Counts:**
|
|
||||||
- CRITICAL_ISSUES, HIGH_ISSUES
|
|
||||||
- MEDIUM_ISSUES, LOW_ISSUES
|
|
||||||
|
|
||||||
### Helper Functions (`lib/reference-db.sh`)
|
|
||||||
|
|
||||||
#### Query Individual Metrics
|
|
||||||
```bash
|
|
||||||
value=$(db_get_health_metric "MEMORY_USED_PERCENT")
|
|
||||||
echo "Memory: $value%"
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Intelligence Functions
|
|
||||||
|
|
||||||
**Check System Load:**
|
|
||||||
```bash
|
|
||||||
if db_is_system_under_load; then
|
|
||||||
echo "System under heavy load (CPU > 80% or Memory > 90%)"
|
|
||||||
# Adjust recommendations
|
|
||||||
fi
|
|
||||||
```
|
|
||||||
|
|
||||||
**Check Network Issues:**
|
|
||||||
```bash
|
|
||||||
if db_has_network_issues; then
|
|
||||||
echo "Network problems detected (retrans > 5% or errors > 100)"
|
|
||||||
# Consider network factors in analysis
|
|
||||||
fi
|
|
||||||
```
|
|
||||||
|
|
||||||
**Check Security Status:**
|
|
||||||
```bash
|
|
||||||
if db_is_under_attack; then
|
|
||||||
echo "Active attacks detected (> 100 SSH failures today)"
|
|
||||||
# Correlate with security findings
|
|
||||||
fi
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Get All Metrics
|
|
||||||
```bash
|
|
||||||
db_get_all_health # Returns all HEALTH| lines
|
|
||||||
```
|
|
||||||
|
|
||||||
## Implementation in Modules
|
|
||||||
|
|
||||||
### Pattern 1: Contextual Recommendations
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# In any module, after sourcing reference-db.sh
|
|
||||||
|
|
||||||
# Check system context
|
|
||||||
if db_is_system_under_load; then
|
|
||||||
echo "NOTE: System is currently under heavy load."
|
|
||||||
echo " Some issues may be resource-related."
|
|
||||||
fi
|
|
||||||
|
|
||||||
if db_has_network_issues; then
|
|
||||||
echo "NOTE: Network experiencing high retransmission rates."
|
|
||||||
echo " Connection issues may be network-related."
|
|
||||||
fi
|
|
||||||
|
|
||||||
if db_is_under_attack; then
|
|
||||||
echo "WARNING: System under active SSH attack."
|
|
||||||
echo " Security hardening recommended."
|
|
||||||
fi
|
|
||||||
```
|
|
||||||
|
|
||||||
### Pattern 2: Adjusted Thresholds
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# MySQL slow query analyzer
|
|
||||||
|
|
||||||
# Normal threshold: 5 seconds
|
|
||||||
SLOW_THRESHOLD=5
|
|
||||||
|
|
||||||
# But if system is under load, adjust threshold
|
|
||||||
if db_is_system_under_load; then
|
|
||||||
SLOW_THRESHOLD=10
|
|
||||||
echo "System under load - using relaxed slow query threshold"
|
|
||||||
fi
|
|
||||||
```
|
|
||||||
|
|
||||||
### Pattern 3: Root Cause Analysis
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Website performance analyzer
|
|
||||||
|
|
||||||
if db_has_network_issues; then
|
|
||||||
echo "Website slow, AND network has issues."
|
|
||||||
echo "Root cause may be network, not website code."
|
|
||||||
echo "Recommendation: Fix network first, then re-test."
|
|
||||||
fi
|
|
||||||
```
|
|
||||||
|
|
||||||
## Testing
|
|
||||||
|
|
||||||
Run the test script to verify cross-module intelligence:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# First, generate session data
|
|
||||||
./launcher.sh
|
|
||||||
# Choose option 1: System Health Check
|
|
||||||
|
|
||||||
# Then test intelligence
|
|
||||||
./tools/test-cross-module-intelligence.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected output shows:
|
|
||||||
- All health metrics populated
|
|
||||||
- Intelligence functions working
|
|
||||||
- System status correctly identified
|
|
||||||
|
|
||||||
## Best Practices
|
|
||||||
|
|
||||||
### DO:
|
|
||||||
✅ Run System Health Check **FIRST** in troubleshooting session
|
|
||||||
✅ Use intelligence functions to provide context-aware recommendations
|
|
||||||
✅ Correlate findings across modules
|
|
||||||
✅ Adjust thresholds based on system state
|
|
||||||
|
|
||||||
### DON'T:
|
|
||||||
❌ Rely on this data for historical trend analysis (it's session-only)
|
|
||||||
❌ Assume data exists (always check if metric is populated)
|
|
||||||
❌ Make critical decisions solely on this data
|
|
||||||
❌ Store this long-term (it gets cleaned up)
|
|
||||||
|
|
||||||
## Example: Enhanced Bot Analyzer (Future)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# modules/security/bot-analyzer.sh
|
|
||||||
|
|
||||||
source "$SCRIPT_DIR/lib/reference-db.sh"
|
|
||||||
|
|
||||||
# After analysis, provide context
|
|
||||||
|
|
||||||
if db_has_network_issues; then
|
|
||||||
echo ""
|
|
||||||
print_warning "Network Issues Detected"
|
|
||||||
echo "System experiencing:"
|
|
||||||
echo " • TCP Retransmission: $(db_get_health_metric 'TCP_RETRANS_PERCENT')%"
|
|
||||||
echo " • Network errors: $(db_get_health_metric 'NETWORK_RX_ERRORS')"
|
|
||||||
echo ""
|
|
||||||
echo "Bot traffic may be compounded by network problems."
|
|
||||||
echo "Recommendation: Address network issues first (see System Health Check)"
|
|
||||||
fi
|
|
||||||
|
|
||||||
if db_is_system_under_load; then
|
|
||||||
echo ""
|
|
||||||
print_warning "System Under Heavy Load"
|
|
||||||
echo "Current state:"
|
|
||||||
echo " • CPU Load: $(db_get_health_metric 'CPU_LOAD_1MIN')"
|
|
||||||
echo " • Memory: $(db_get_health_metric 'MEMORY_USED_PERCENT')%"
|
|
||||||
echo ""
|
|
||||||
echo "High bot traffic + system load = performance degradation."
|
|
||||||
echo "Recommendation: Block bots AND investigate resource usage."
|
|
||||||
fi
|
|
||||||
```
|
|
||||||
|
|
||||||
## Files Modified
|
|
||||||
|
|
||||||
1. **modules/diagnostics/system-health-check.sh**
|
|
||||||
- Enhanced `save_health_baseline()` function
|
|
||||||
- Now saves network, hardware, and security metrics
|
|
||||||
- Lines: 1660-1758
|
|
||||||
|
|
||||||
2. **lib/reference-db.sh**
|
|
||||||
- Added `db_get_health_metric()` - query individual metrics
|
|
||||||
- Added `db_is_system_under_load()` - check if CPU/memory high
|
|
||||||
- Added `db_has_network_issues()` - check for network problems
|
|
||||||
- Added `db_is_under_attack()` - check for active attacks
|
|
||||||
- Added `db_get_all_health()` - get all health data
|
|
||||||
- Lines: 446-497
|
|
||||||
|
|
||||||
3. **tools/test-cross-module-intelligence.sh** (NEW)
|
|
||||||
- Test script demonstrating cross-module queries
|
|
||||||
- Shows how to use intelligence functions
|
|
||||||
|
|
||||||
## Data Lifetime
|
|
||||||
|
|
||||||
- **Created:** When System Health Check runs
|
|
||||||
- **Stored:** In `.sysref` file (memory + disk)
|
|
||||||
- **Expires:** After 1 hour OR when cleanup/reset runs
|
|
||||||
- **Removed:** When toolkit is deleted
|
|
||||||
|
|
||||||
## Future Enhancements
|
|
||||||
|
|
||||||
Potential modules that could benefit:
|
|
||||||
|
|
||||||
1. **WordPress Health Check**
|
|
||||||
- Check if slow WP sites correlate with network/load issues
|
|
||||||
|
|
||||||
2. **Backup Analyzer**
|
|
||||||
- Check if backup failures correlate with disk/load issues
|
|
||||||
|
|
||||||
3. **Email Troubleshooter**
|
|
||||||
- Check if email issues correlate with network/disk problems
|
|
||||||
|
|
||||||
4. **Resource Monitor**
|
|
||||||
- Compare current metrics vs health check baseline
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Session Intelligence transforms the toolkit from **isolated modules** into an **integrated diagnostic platform**. Each module can now make smarter, context-aware recommendations based on the complete picture of what's happening on the server RIGHT NOW.
|
|
||||||
|
|
||||||
No historical data needed. No complex trending. Just smart, session-aware troubleshooting.
|
|
||||||
-379
@@ -1,379 +0,0 @@
|
|||||||
# 🚀 Server Management Toolkit - Setup Guide
|
|
||||||
|
|
||||||
## ✅ What You Have Now
|
|
||||||
|
|
||||||
A **modular, scalable server management system** with:
|
|
||||||
|
|
||||||
✨ **Professional Menu System**
|
|
||||||
- Clean, organized category-based menus
|
|
||||||
- Color-coded interface
|
|
||||||
- Easy navigation
|
|
||||||
|
|
||||||
📦 **Modular Architecture**
|
|
||||||
- 7 main categories (80+ potential modules)
|
|
||||||
- Easy to add new modules
|
|
||||||
- Organized by function
|
|
||||||
|
|
||||||
☁️ **Nextcloud Integration**
|
|
||||||
- Download modules on-demand
|
|
||||||
- Easy updates
|
|
||||||
- Share across multiple servers
|
|
||||||
|
|
||||||
🎯 **First Module Ready**
|
|
||||||
- `bot-analyzer.sh` - Enhanced v3.0
|
|
||||||
- All improvements we made today
|
|
||||||
- Ready to use immediately
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📋 Directory Structure
|
|
||||||
|
|
||||||
```
|
|
||||||
/root/server-toolkit/
|
|
||||||
├── launcher.sh ← Main menu (run this!)
|
|
||||||
├── install.sh ← Quick installer
|
|
||||||
├── README.md ← Full documentation
|
|
||||||
├── manifest.txt.example ← Template for Nextcloud
|
|
||||||
│
|
|
||||||
├── modules/
|
|
||||||
│ ├── security/
|
|
||||||
│ │ └── bot-analyzer.sh ✅ READY (v3.0 Enhanced)
|
|
||||||
│ ├── wordpress/ (empty - add modules here)
|
|
||||||
│ ├── performance/ (empty - add modules here)
|
|
||||||
│ ├── backup/ (empty - add modules here)
|
|
||||||
│ ├── monitoring/ (empty - add modules here)
|
|
||||||
│ ├── troubleshooting/ (empty - add modules here)
|
|
||||||
│ └── reporting/ (empty - add modules here)
|
|
||||||
│
|
|
||||||
├── lib/ (common functions - future)
|
|
||||||
├── config/ (created on first run)
|
|
||||||
└── logs/ (created on first run)
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🎯 Quick Start (3 Steps)
|
|
||||||
|
|
||||||
### Step 1: Run the Installer
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd /root/server-toolkit
|
|
||||||
chmod +x install.sh
|
|
||||||
./install.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
**What it does:**
|
|
||||||
- Creates directory structure
|
|
||||||
- Sets permissions
|
|
||||||
- Offers to create `/usr/local/bin/server-toolkit` symlink
|
|
||||||
|
|
||||||
### Step 2: Launch & Configure
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Option A: Direct
|
|
||||||
/root/server-toolkit/launcher.sh
|
|
||||||
|
|
||||||
# Option B: If symlink created
|
|
||||||
server-toolkit
|
|
||||||
```
|
|
||||||
|
|
||||||
**First time:**
|
|
||||||
1. Select `9` (Configuration)
|
|
||||||
2. Set your Nextcloud URL (optional, for module downloads)
|
|
||||||
3. Review other settings
|
|
||||||
4. Save and exit
|
|
||||||
|
|
||||||
### Step 3: Test the Bot Analyzer
|
|
||||||
|
|
||||||
From the launcher:
|
|
||||||
1. Select `1` (Security & Threat Analysis)
|
|
||||||
2. Select `1` (Full Bot Analysis)
|
|
||||||
3. Watch it run!
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## ☁️ Nextcloud Setup (Optional but Recommended)
|
|
||||||
|
|
||||||
### Why Use Nextcloud?
|
|
||||||
|
|
||||||
✅ Store all modules in one place
|
|
||||||
✅ Easy updates across multiple servers
|
|
||||||
✅ No need to manually copy files
|
|
||||||
✅ Version control your modules
|
|
||||||
|
|
||||||
### Setup Process
|
|
||||||
|
|
||||||
**1. Upload to Nextcloud**
|
|
||||||
|
|
||||||
```
|
|
||||||
your-nextcloud/
|
|
||||||
└── server-toolkit/
|
|
||||||
├── manifest.txt ← Copy from manifest.txt.example
|
|
||||||
└── modules/
|
|
||||||
├── security/
|
|
||||||
│ ├── bot-analyzer.sh
|
|
||||||
│ ├── live-monitor.sh
|
|
||||||
│ └── ...
|
|
||||||
├── wordpress/
|
|
||||||
│ ├── wp-cron-status.sh
|
|
||||||
│ └── ...
|
|
||||||
└── ...
|
|
||||||
```
|
|
||||||
|
|
||||||
**2. Share the Folder**
|
|
||||||
- Right-click folder → Share
|
|
||||||
- Create public link
|
|
||||||
- Enable "Allow download"
|
|
||||||
- Copy the share link
|
|
||||||
|
|
||||||
**3. Convert Link to Download URL**
|
|
||||||
|
|
||||||
Original link:
|
|
||||||
```
|
|
||||||
https://nextcloud.example.com/s/AbC123DeF
|
|
||||||
```
|
|
||||||
|
|
||||||
Convert to:
|
|
||||||
```
|
|
||||||
https://nextcloud.example.com/s/AbC123DeF/download?path=/
|
|
||||||
```
|
|
||||||
|
|
||||||
**4. Configure**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
nano /root/server-toolkit/config/settings.conf
|
|
||||||
```
|
|
||||||
|
|
||||||
Set:
|
|
||||||
```bash
|
|
||||||
NEXTCLOUD_BASE_URL="https://nextcloud.example.com/s/AbC123DeF/download?path=/"
|
|
||||||
```
|
|
||||||
|
|
||||||
**5. Update Modules**
|
|
||||||
|
|
||||||
From launcher: Select `8` (Update All Modules)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔧 Adding New Modules
|
|
||||||
|
|
||||||
### Method 1: Create Locally
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Create new module
|
|
||||||
nano /root/server-toolkit/modules/wordpress/wp-cron-status.sh
|
|
||||||
|
|
||||||
# Make executable
|
|
||||||
chmod +x /root/server-toolkit/modules/wordpress/wp-cron-status.sh
|
|
||||||
|
|
||||||
# Test it
|
|
||||||
/root/server-toolkit/modules/wordpress/wp-cron-status.sh
|
|
||||||
|
|
||||||
# It's now available in the launcher menu!
|
|
||||||
```
|
|
||||||
|
|
||||||
### Method 2: Download from Nextcloud
|
|
||||||
|
|
||||||
1. Upload to Nextcloud: `modules/wordpress/wp-cron-status.sh`
|
|
||||||
2. Add to `manifest.txt`: `wordpress:wp-cron-status.sh`
|
|
||||||
3. From launcher: Select `8` (Update All Modules)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📊 Current Features
|
|
||||||
|
|
||||||
### ✅ Working Now
|
|
||||||
|
|
||||||
| Feature | Status |
|
|
||||||
|---------|--------|
|
|
||||||
| Modular architecture | ✅ Complete |
|
|
||||||
| Category-based menus | ✅ Complete |
|
|
||||||
| Bot analyzer v3.0 | ✅ Working |
|
|
||||||
| Server IP detection | ✅ Working |
|
|
||||||
| Threat scoring | ✅ Working |
|
|
||||||
| Nextcloud integration | ✅ Working |
|
|
||||||
| Configuration system | ✅ Working |
|
|
||||||
| Auto-updates | ✅ Working |
|
|
||||||
|
|
||||||
### 🔜 Coming Soon (As You Build Them)
|
|
||||||
|
|
||||||
| Module | Priority | Category |
|
|
||||||
|--------|----------|----------|
|
|
||||||
| wp-cron-status.sh | High | WordPress |
|
|
||||||
| wp-cron-mass-fix.sh | High | WordPress |
|
|
||||||
| oom-killer-plotter.sh | Medium | Troubleshooting |
|
|
||||||
| resource-monitor.sh | Medium | Performance |
|
|
||||||
| disk-usage-report.sh | Medium | Performance |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🎓 Example Workflows
|
|
||||||
|
|
||||||
### Daily Security Check
|
|
||||||
|
|
||||||
```bash
|
|
||||||
server-toolkit
|
|
||||||
→ 1 (Security)
|
|
||||||
→ 2 (Quick Scan - 1 hour)
|
|
||||||
→ Review threats
|
|
||||||
→ 5 (Auto-Block if needed)
|
|
||||||
```
|
|
||||||
|
|
||||||
### WordPress Maintenance
|
|
||||||
|
|
||||||
```bash
|
|
||||||
server-toolkit
|
|
||||||
→ 2 (WordPress)
|
|
||||||
→ 2 (Check WP-Cron status)
|
|
||||||
→ 3 (Fix if broken)
|
|
||||||
→ 7 (Optimize databases)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Performance Investigation
|
|
||||||
|
|
||||||
```bash
|
|
||||||
server-toolkit
|
|
||||||
→ 3 (Performance)
|
|
||||||
→ 1 (Resource Monitor)
|
|
||||||
→ 2 (Top Processes)
|
|
||||||
→ Identify issues
|
|
||||||
```
|
|
||||||
|
|
||||||
### Troubleshoot Out-of-Memory
|
|
||||||
|
|
||||||
```bash
|
|
||||||
server-toolkit
|
|
||||||
→ 6 (Troubleshooting)
|
|
||||||
→ 1 (OOM Killer Plotter)
|
|
||||||
→ Review memory spikes
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔐 Security Best Practices
|
|
||||||
|
|
||||||
### Before Running
|
|
||||||
|
|
||||||
✅ Always backup first
|
|
||||||
✅ Test on staging if possible
|
|
||||||
✅ Review whitelist before blocking
|
|
||||||
✅ Check false positives
|
|
||||||
|
|
||||||
### Regular Maintenance
|
|
||||||
|
|
||||||
📅 **Daily**: Quick security scan
|
|
||||||
📅 **Weekly**: Full bot analysis
|
|
||||||
📅 **Monthly**: Update all modules
|
|
||||||
📅 **Quarterly**: Review all whitelists
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🆘 Troubleshooting
|
|
||||||
|
|
||||||
### Launcher Won't Start
|
|
||||||
|
|
||||||
```bash
|
|
||||||
chmod +x /root/server-toolkit/launcher.sh
|
|
||||||
bash /root/server-toolkit/launcher.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### Module Not Found
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check if it exists
|
|
||||||
ls -la /root/server-toolkit/modules/security/bot-analyzer.sh
|
|
||||||
|
|
||||||
# Redownload from Nextcloud
|
|
||||||
server-toolkit → 8 (Update)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Config Issues
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Recreate config
|
|
||||||
rm /root/server-toolkit/config/settings.conf
|
|
||||||
server-toolkit → 9 (Configuration)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Nextcloud Download Fails
|
|
||||||
|
|
||||||
1. Check NEXTCLOUD_BASE_URL format
|
|
||||||
2. Ensure Nextcloud folder is shared publicly
|
|
||||||
3. Test URL in browser first
|
|
||||||
4. Check manifest.txt format
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📞 Next Steps
|
|
||||||
|
|
||||||
### Immediate
|
|
||||||
|
|
||||||
1. ✅ Run installer
|
|
||||||
2. ✅ Test bot analyzer
|
|
||||||
3. ✅ Configure settings
|
|
||||||
|
|
||||||
### Short Term
|
|
||||||
|
|
||||||
1. 📝 Create wp-cron-status.sh module
|
|
||||||
2. 📝 Create wp-cron-mass-fix.sh module
|
|
||||||
3. ☁️ Setup Nextcloud distribution
|
|
||||||
|
|
||||||
### Long Term
|
|
||||||
|
|
||||||
1. 📦 Build remaining modules
|
|
||||||
2. 🔄 Setup automated updates
|
|
||||||
3. 📧 Configure email alerts
|
|
||||||
4. 📊 Create custom dashboards
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 💡 Pro Tips
|
|
||||||
|
|
||||||
### Performance
|
|
||||||
|
|
||||||
- Bot analyzer runs in < 1 second for small logs
|
|
||||||
- Use `-H 1` for quick scans
|
|
||||||
- Schedule daily cron for security checks
|
|
||||||
|
|
||||||
### Organization
|
|
||||||
|
|
||||||
- Keep modules organized by category
|
|
||||||
- Use descriptive names
|
|
||||||
- Add comments in scripts
|
|
||||||
- Update manifest when adding modules
|
|
||||||
|
|
||||||
### Distribution
|
|
||||||
|
|
||||||
- Use Nextcloud for easy sharing
|
|
||||||
- Keep manifest.txt updated
|
|
||||||
- Version your modules
|
|
||||||
- Test before distributing
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📚 Documentation
|
|
||||||
|
|
||||||
- `README.md` - Full documentation
|
|
||||||
- `launcher.sh` - Built-in help menus
|
|
||||||
- Each module - Individual usage info
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## ✅ Installation Checklist
|
|
||||||
|
|
||||||
- [ ] Ran `/root/server-toolkit/install.sh`
|
|
||||||
- [ ] Launcher runs successfully
|
|
||||||
- [ ] Created symlink (optional)
|
|
||||||
- [ ] Configured settings
|
|
||||||
- [ ] Tested bot analyzer
|
|
||||||
- [ ] Setup Nextcloud (optional)
|
|
||||||
- [ ] Updated modules (if using Nextcloud)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**You now have a professional, scalable server management system!** 🎉
|
|
||||||
|
|
||||||
Add modules as you need them, share via Nextcloud, and manage your entire infrastructure from one clean interface.
|
|
||||||
|
|
||||||
**Version**: 2.0.0
|
|
||||||
**Date**: 2025-10-30
|
|
||||||
@@ -1,273 +0,0 @@
|
|||||||
# Server Toolkit - Troubleshooting Guide
|
|
||||||
|
|
||||||
## Quick Diagnostics
|
|
||||||
|
|
||||||
### Test Domain Detection
|
|
||||||
```bash
|
|
||||||
bash /root/server-toolkit/tools/test-domain-detection.sh
|
|
||||||
```
|
|
||||||
This will tell you immediately if domain detection is working.
|
|
||||||
|
|
||||||
### Check System Detection Variables
|
|
||||||
```bash
|
|
||||||
bash -c '
|
|
||||||
source /root/server-toolkit/lib/system-detect.sh
|
|
||||||
echo "SYS_CONTROL_PANEL: [$SYS_CONTROL_PANEL]"
|
|
||||||
echo "SYS_DETECTION_COMPLETE: [$SYS_DETECTION_COMPLETE]"
|
|
||||||
'
|
|
||||||
```
|
|
||||||
Both should have values. If empty, system detection failed.
|
|
||||||
|
|
||||||
### Test User Domain Lookup
|
|
||||||
```bash
|
|
||||||
bash -c '
|
|
||||||
source /root/server-toolkit/lib/system-detect.sh
|
|
||||||
source /root/server-toolkit/lib/user-manager.sh
|
|
||||||
get_user_domains "USERNAME"
|
|
||||||
'
|
|
||||||
```
|
|
||||||
Replace USERNAME with actual username. Should return domain(s).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Common Issues
|
|
||||||
|
|
||||||
### Issue: User shows "(no domains) (0 domains)"
|
|
||||||
|
|
||||||
**Symptoms:**
|
|
||||||
- User selection menu shows 0 domains
|
|
||||||
- Bot analyzer says "No domains found for user"
|
|
||||||
- Domain exists in cPanel
|
|
||||||
|
|
||||||
**Diagnosis:**
|
|
||||||
1. Run: `echo $SYS_CONTROL_PANEL` in your shell
|
|
||||||
2. If empty, environment is corrupted
|
|
||||||
|
|
||||||
**Fix:**
|
|
||||||
- Option 1: Exit launcher completely and restart
|
|
||||||
- Option 2: Select option 8 (Cleanup/Reset) in launcher
|
|
||||||
- Option 3: Close entire SSH session and reconnect
|
|
||||||
|
|
||||||
**Why it happens:**
|
|
||||||
Launcher inherited broken environment variables from a previous session where
|
|
||||||
libraries had bugs. Child processes (like bot-analyzer) inherit these.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Issue: Functions not found / command not found
|
|
||||||
|
|
||||||
**Symptoms:**
|
|
||||||
- `bash: select_user_interactive: command not found`
|
|
||||||
- `bash: get_user_domains: command not found`
|
|
||||||
|
|
||||||
**Diagnosis:**
|
|
||||||
Libraries weren't sourced correctly.
|
|
||||||
|
|
||||||
**Fix:**
|
|
||||||
1. Check that files exist:
|
|
||||||
```bash
|
|
||||||
ls -la /root/server-toolkit/lib/*.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
2. Test sourcing manually:
|
|
||||||
```bash
|
|
||||||
source /root/server-toolkit/lib/system-detect.sh
|
|
||||||
source /root/server-toolkit/lib/user-manager.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
3. Check for syntax errors:
|
|
||||||
```bash
|
|
||||||
bash -n /root/server-toolkit/lib/system-detect.sh
|
|
||||||
bash -n /root/server-toolkit/lib/user-manager.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Issue: Menus displaying twice or garbled output
|
|
||||||
|
|
||||||
**Symptoms:**
|
|
||||||
- Same menu appears multiple times
|
|
||||||
- Detection messages appear before menus
|
|
||||||
- ANSI codes visible like `[H[J`
|
|
||||||
|
|
||||||
**Diagnosis:**
|
|
||||||
Terminal doesn't support ANSI codes or clear screen.
|
|
||||||
|
|
||||||
**Fix:**
|
|
||||||
Set high contrast mode:
|
|
||||||
```bash
|
|
||||||
export TOOLKIT_HIGH_CONTRAST=1
|
|
||||||
bash /root/server-toolkit/launcher.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
Or disable colors completely:
|
|
||||||
```bash
|
|
||||||
export TOOLKIT_NO_COLOR=1
|
|
||||||
bash /root/server-toolkit/launcher.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Issue: CSF commands not working
|
|
||||||
|
|
||||||
**Symptoms:**
|
|
||||||
- "csf: command not found"
|
|
||||||
- CSF blocking options don't work
|
|
||||||
|
|
||||||
**Diagnosis:**
|
|
||||||
CSF not installed or not in PATH.
|
|
||||||
|
|
||||||
**Check:**
|
|
||||||
```bash
|
|
||||||
which csf
|
|
||||||
csf -v
|
|
||||||
```
|
|
||||||
|
|
||||||
**Fix:**
|
|
||||||
Install CSF or use alternative security methods (Apache .htaccess, etc.)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Issue: cPanel users not detected
|
|
||||||
|
|
||||||
**Symptoms:**
|
|
||||||
- "No users found"
|
|
||||||
- list_all_users returns nothing
|
|
||||||
|
|
||||||
**Diagnosis:**
|
|
||||||
Check if cPanel user files exist:
|
|
||||||
```bash
|
|
||||||
ls -la /var/cpanel/users/
|
|
||||||
cat /etc/trueuserdomains | head
|
|
||||||
```
|
|
||||||
|
|
||||||
**Fix:**
|
|
||||||
If files missing, not a cPanel system. System will fall back to standard
|
|
||||||
user detection from /etc/passwd.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Debug Mode
|
|
||||||
|
|
||||||
### Enable Verbose Initialization
|
|
||||||
```bash
|
|
||||||
export TOOLKIT_VERBOSE_INIT=1
|
|
||||||
bash /root/server-toolkit/launcher.sh
|
|
||||||
```
|
|
||||||
Shows all system detection messages.
|
|
||||||
|
|
||||||
### Trace Execution
|
|
||||||
```bash
|
|
||||||
bash -x /root/server-toolkit/modules/security/bot-analyzer.sh 2>&1 | less
|
|
||||||
```
|
|
||||||
Shows every command executed (very verbose).
|
|
||||||
|
|
||||||
### Check Environment Variables
|
|
||||||
```bash
|
|
||||||
# Show all SYS_* variables
|
|
||||||
env | grep "^SYS_"
|
|
||||||
|
|
||||||
# Show all toolkit-related variables
|
|
||||||
env | grep -i toolkit
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## File Locations
|
|
||||||
|
|
||||||
### Logs
|
|
||||||
- Bot analysis reports: `/tmp/bot_analysis_report_*.txt`
|
|
||||||
- MySQL analysis: `/tmp/mysql_analysis_*.txt`
|
|
||||||
- Temp sessions: `/tmp/server-toolkit-*`
|
|
||||||
|
|
||||||
### Cache Files
|
|
||||||
- System reference database: `/root/server-toolkit/.sysref`
|
|
||||||
- Timestamp file: `/root/server-toolkit/.sysref.timestamp`
|
|
||||||
|
|
||||||
### Configuration
|
|
||||||
- Settings: `/root/server-toolkit/config/settings.conf`
|
|
||||||
- Custom slash commands: `/root/server-toolkit/.claude/commands/`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Performance Issues
|
|
||||||
|
|
||||||
### Issue: Slow user selection with 200+ users
|
|
||||||
|
|
||||||
**Fix:**
|
|
||||||
- Use search: `s <partial-name>`
|
|
||||||
- Searches only, doesn't list all users
|
|
||||||
- Much faster than 'L' (list all)
|
|
||||||
|
|
||||||
### Issue: Bot analyzer takes too long
|
|
||||||
|
|
||||||
**Optimization:**
|
|
||||||
1. Use time filters: Last 1 hour instead of "All logs"
|
|
||||||
2. Use user filter: Analyze specific user instead of all
|
|
||||||
3. Check log size: `du -sh /var/log/apache2/domlogs/*`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Recovery Commands
|
|
||||||
|
|
||||||
### Complete Reset
|
|
||||||
```bash
|
|
||||||
# In launcher, select option 8 (Cleanup/Reset)
|
|
||||||
# Or manually:
|
|
||||||
rm -f /root/server-toolkit/.sysref*
|
|
||||||
rm -rf /tmp/server-toolkit-*
|
|
||||||
rm -f /tmp/bot_analysis_* /tmp/mysql_analysis_*
|
|
||||||
```
|
|
||||||
|
|
||||||
### Force Library Reload
|
|
||||||
```bash
|
|
||||||
# In bash session:
|
|
||||||
for var in $(compgen -e | grep "^SYS_"); do unset "$var"; done
|
|
||||||
unset -f initialize_system_detection get_user_domains select_user_interactive
|
|
||||||
source /root/server-toolkit/lib/system-detect.sh
|
|
||||||
source /root/server-toolkit/lib/user-manager.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### Kill Stuck Processes
|
|
||||||
```bash
|
|
||||||
# Find launcher processes
|
|
||||||
ps aux | grep launcher
|
|
||||||
|
|
||||||
# Kill specific PID
|
|
||||||
kill -9 <PID>
|
|
||||||
|
|
||||||
# Kill all launcher instances
|
|
||||||
pkill -9 -f launcher.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Getting Help
|
|
||||||
|
|
||||||
### Self-Diagnostic
|
|
||||||
1. Run test script: `bash /root/server-toolkit/tools/test-domain-detection.sh`
|
|
||||||
2. Check REFDB_FORMAT.txt for known bugs and fixes
|
|
||||||
3. Review this troubleshooting guide
|
|
||||||
|
|
||||||
### Report Issues
|
|
||||||
When reporting problems, include:
|
|
||||||
1. Output of test-domain-detection.sh
|
|
||||||
2. Output of: `env | grep "^SYS_"`
|
|
||||||
3. Control panel type: `cat /usr/local/cpanel/version` or equivalent
|
|
||||||
4. Error messages (exact text)
|
|
||||||
5. Steps to reproduce
|
|
||||||
|
|
||||||
### Quick Fixes to Try First
|
|
||||||
1. Exit and restart launcher
|
|
||||||
2. Run Cleanup/Reset (option 8)
|
|
||||||
3. Close SSH and reconnect
|
|
||||||
4. Run test-domain-detection.sh to verify files are correct
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Version Information
|
|
||||||
|
|
||||||
**Created:** 2025-10-31
|
|
||||||
**Last Updated:** 2025-10-31
|
|
||||||
**Toolkit Version:** 2.0.0
|
|
||||||
**Compatible With:** cPanel, Plesk, InterWorx, Standalone Linux servers
|
|
||||||
-441
@@ -1,441 +0,0 @@
|
|||||||
# 🎉 What We Built Today - Complete Summary
|
|
||||||
|
|
||||||
## 📦 Deliverables
|
|
||||||
|
|
||||||
### 1. **Enhanced Bot Analyzer v3.0**
|
|
||||||
Location: `/root/server-toolkit/modules/security/bot-analyzer.sh`
|
|
||||||
|
|
||||||
**Major Improvements:**
|
|
||||||
- ✅ Enhanced attack vector detection (6 types)
|
|
||||||
- ✅ Threat scoring system (0-100 risk scores)
|
|
||||||
- ✅ Time-series analysis with hourly breakdown
|
|
||||||
- ✅ Response code intelligence
|
|
||||||
- ✅ False positive detection
|
|
||||||
- ✅ Server IP auto-detection
|
|
||||||
- ✅ Bandwidth cost estimation
|
|
||||||
- ✅ **60-120x performance improvement**
|
|
||||||
- ✅ Private IP filtering
|
|
||||||
- ✅ Prioritized blocklists
|
|
||||||
|
|
||||||
### 2. **Professional Server Management Toolkit**
|
|
||||||
Location: `/root/server-toolkit/`
|
|
||||||
|
|
||||||
**Complete Modular System:**
|
|
||||||
- ✅ Clean launcher with 7 category menus
|
|
||||||
- ✅ 80+ module slots organized by function
|
|
||||||
- ✅ Nextcloud integration for remote updates
|
|
||||||
- ✅ Configuration management
|
|
||||||
- ✅ Professional directory structure
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🚀 Bot Analyzer Enhancements (v3.0)
|
|
||||||
|
|
||||||
### Attack Vector Detection
|
|
||||||
|
|
||||||
**OLD**: Only detected SQL injection and generic scanners
|
|
||||||
|
|
||||||
**NEW**: Detects 6 attack types:
|
|
||||||
```
|
|
||||||
💉 SQL Injection - UNION, SELECT, hex encoding
|
|
||||||
🌐 XSS Attacks - JavaScript injection, event handlers
|
|
||||||
📁 Path Traversal - Directory traversal, LFI
|
|
||||||
📤 RCE/Shell Upload - PHP shells, backdoors
|
|
||||||
🔍 Info Disclosure - .git, .env, config files
|
|
||||||
🔓 Login Bruteforce - wp-login, xmlrpc attacks
|
|
||||||
```
|
|
||||||
|
|
||||||
### Threat Scoring System
|
|
||||||
|
|
||||||
**NEW Feature**: Each IP gets 0-100 risk score
|
|
||||||
|
|
||||||
**Example Output:**
|
|
||||||
```
|
|
||||||
[1] 143.244.57.123 - RISK: 98/100 🔴 CRITICAL
|
|
||||||
648 requests - Action: BLOCK IMMEDIATELY + INVESTIGATE
|
|
||||||
Attack vectors: SQL-Injection RCE/Upload Login-Bruteforce DDoS-Pattern
|
|
||||||
```
|
|
||||||
|
|
||||||
**Score Components:**
|
|
||||||
- Request volume: up to 10 points
|
|
||||||
- Attack patterns: up to 70 points
|
|
||||||
- Behavioral signals: up to 20 points
|
|
||||||
|
|
||||||
### Time-Series Analysis
|
|
||||||
|
|
||||||
**NEW**: Hourly traffic visualization
|
|
||||||
|
|
||||||
```
|
|
||||||
Bot Traffic Timeline (hourly):
|
|
||||||
14:00-15:00: ████████░░ 8,240 bot requests
|
|
||||||
15:00-16:00: ███░░░░░░░ 3,120 bot requests
|
|
||||||
16:00-17:00: ██████████ 12,450 bot requests ⚠️ SPIKE
|
|
||||||
```
|
|
||||||
|
|
||||||
### Response Code Intelligence
|
|
||||||
|
|
||||||
**NEW**: Shows what bots are finding
|
|
||||||
|
|
||||||
```
|
|
||||||
200 (Success): 18,432 (62%) ✓ Bots are getting data
|
|
||||||
404 (Not Found): 7,891 (27%) ⚠️ Scanning for vulnerabilities
|
|
||||||
403 (Forbidden): 2,103 (7%) ✓ Blocked by existing rules
|
|
||||||
500 (Server Error): 12 (0%) 🚨 Check if exploit triggered
|
|
||||||
```
|
|
||||||
|
|
||||||
### False Positive Detection
|
|
||||||
|
|
||||||
**NEW**: Auto-identifies legitimate services
|
|
||||||
|
|
||||||
```
|
|
||||||
⚠️ Whitelist Recommendations:
|
|
||||||
65.181.111.155 - 11,515 requests - Identified as: Pingdom Monitoring
|
|
||||||
→ Action: VERIFY OWNERSHIP then whitelist
|
|
||||||
```
|
|
||||||
|
|
||||||
**Detects:**
|
|
||||||
- Pingdom, UptimeRobot, StatusCake
|
|
||||||
- WordPress cache preload (WP Rocket, Hummingbird)
|
|
||||||
- Backup services (Jetpack, VaultPress)
|
|
||||||
|
|
||||||
### Server IP Detection
|
|
||||||
|
|
||||||
**NEW**: Auto-detects and excludes server's own IPs
|
|
||||||
|
|
||||||
**5 Detection Methods:**
|
|
||||||
1. hostname -I (network interfaces)
|
|
||||||
2. ip addr show (Linux IP command)
|
|
||||||
3. ifconfig (legacy fallback)
|
|
||||||
4. External services (public IP)
|
|
||||||
5. cPanel mainip file
|
|
||||||
|
|
||||||
**Output:**
|
|
||||||
```
|
|
||||||
✓ Detected 2 server IP(s) - excluded from threat analysis
|
|
||||||
|
|
||||||
🖥️ Server IPs Detected:
|
|
||||||
• 127.0.0.1
|
|
||||||
• 67.227.199.95
|
|
||||||
```
|
|
||||||
|
|
||||||
### Bandwidth Cost Estimation
|
|
||||||
|
|
||||||
**NEW**: Shows financial impact
|
|
||||||
|
|
||||||
```
|
|
||||||
💰 Bandwidth Impact:
|
|
||||||
Total bot bandwidth: 847 MB (0.85 GB) - 14.2% of total
|
|
||||||
Estimated cost: $0.08 (at $0.09/GB CDN pricing)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Prioritized Blocklists
|
|
||||||
|
|
||||||
**OLD**: Random order, no context
|
|
||||||
|
|
||||||
**NEW**: Sorted by threat score with annotations
|
|
||||||
|
|
||||||
```
|
|
||||||
# IPs sorted by risk score (highest first)
|
|
||||||
Deny from 91.92.243.107 # Risk score: 98/100
|
|
||||||
Deny from 34.192.124.246 # Risk score: 85/100
|
|
||||||
Deny from 4.245.190.15 # Risk score: 72/100
|
|
||||||
```
|
|
||||||
|
|
||||||
### Performance Optimization
|
|
||||||
|
|
||||||
**MASSIVE Speed Improvement:**
|
|
||||||
|
|
||||||
| Dataset | Old Method | New Method | Speedup |
|
|
||||||
|---------|------------|------------|---------|
|
|
||||||
| 1,000 IPs / 50K entries | ~2 minutes | ~2 seconds | **60x** |
|
|
||||||
| 10,000 IPs / 250K entries | ~10 minutes | ~10 seconds | **60x** |
|
|
||||||
| 25,000 IPs / 500K entries | ~30 minutes | ~30 seconds | **60x** |
|
|
||||||
| 50,000 IPs / 1M entries | ~2 hours | ~60 seconds | **120x** |
|
|
||||||
|
|
||||||
**How?**
|
|
||||||
- Eliminated 275,000 grep operations
|
|
||||||
- Pre-count requests (single pass)
|
|
||||||
- Hash table lookups (O(1) vs O(n))
|
|
||||||
- Smart caching
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📊 Server Management Toolkit
|
|
||||||
|
|
||||||
### Architecture
|
|
||||||
|
|
||||||
```
|
|
||||||
7 Categories × ~12 modules each = 80+ total module slots
|
|
||||||
|
|
||||||
🛡️ Security & Threat Analysis (10 modules)
|
|
||||||
🔧 WordPress Management (14 modules)
|
|
||||||
📊 Performance & Diagnostics (11 modules)
|
|
||||||
💾 Backup & Recovery (8 modules)
|
|
||||||
🔍 Monitoring & Alerts (8 modules)
|
|
||||||
🚨 Troubleshooting & Diagnostics (11 modules)
|
|
||||||
📈 Reporting & Analytics (7 modules)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Key Features
|
|
||||||
|
|
||||||
**✨ Clean Interface**
|
|
||||||
- Color-coded menus
|
|
||||||
- Intuitive navigation
|
|
||||||
- Consistent UX
|
|
||||||
|
|
||||||
**📦 Modular Design**
|
|
||||||
- Easy to add modules
|
|
||||||
- Independent components
|
|
||||||
- Shared libraries
|
|
||||||
|
|
||||||
**☁️ Nextcloud Integration**
|
|
||||||
- Download modules on-demand
|
|
||||||
- Easy updates
|
|
||||||
- Share across servers
|
|
||||||
|
|
||||||
**⚙️ Configuration System**
|
|
||||||
- Centralized settings
|
|
||||||
- Per-module customization
|
|
||||||
- Whitelist management
|
|
||||||
|
|
||||||
**🔄 Auto-Updates**
|
|
||||||
- One-click module updates
|
|
||||||
- Version tracking
|
|
||||||
- Manifest-based
|
|
||||||
|
|
||||||
### Future Modules (Examples)
|
|
||||||
|
|
||||||
**WordPress:**
|
|
||||||
- `wp-cron-status.sh` - Check cron health
|
|
||||||
- `wp-cron-mass-fix.sh` - Fix broken crons
|
|
||||||
- `wp-cron-mass-create.sh` - Setup system crons
|
|
||||||
- `wp-malware-scanner.sh` - Detect infections
|
|
||||||
|
|
||||||
**Troubleshooting:**
|
|
||||||
- `oom-killer-plotter.sh` - Memory event analysis
|
|
||||||
- `hard-drive-error-tracker.sh` - SMART monitoring
|
|
||||||
- `kernel-log-analyzer.sh` - System event parser
|
|
||||||
|
|
||||||
**Performance:**
|
|
||||||
- `resource-monitor.sh` - Real-time dashboard
|
|
||||||
- `disk-io-analyzer.sh` - I/O bottlenecks
|
|
||||||
- `inode-usage-checker.sh` - Find inode hogs
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📈 Comparison: Before vs After
|
|
||||||
|
|
||||||
### Bot Analyzer
|
|
||||||
|
|
||||||
| Feature | Before (v2.0) | After (v3.0) |
|
|
||||||
|---------|---------------|--------------|
|
|
||||||
| Attack types | 1 (SQL only) | 6 comprehensive |
|
|
||||||
| Threat scoring | No | Yes (0-100 scale) |
|
|
||||||
| Time analysis | No | Hourly breakdown |
|
|
||||||
| Response analysis | No | Yes with insights |
|
|
||||||
| False positives | Manual review | Auto-detection |
|
|
||||||
| Server IP handling | Not excluded | Auto-detected & excluded |
|
|
||||||
| Bandwidth cost | Not shown | Estimated with cost |
|
|
||||||
| Blocklist quality | Basic | Prioritized by risk |
|
|
||||||
| Performance (25K IPs) | 30 minutes | 30 seconds |
|
|
||||||
|
|
||||||
### Overall System
|
|
||||||
|
|
||||||
| Aspect | Before | After |
|
|
||||||
|--------|--------|-------|
|
|
||||||
| Organization | Single script | Modular system |
|
|
||||||
| Maintainability | Hard | Easy |
|
|
||||||
| Scalability | Limited | Unlimited |
|
|
||||||
| Distribution | Manual copy | Nextcloud sync |
|
|
||||||
| Updates | Manual | One-click |
|
|
||||||
| Categories | N/A | 7 organized |
|
|
||||||
| Future growth | Difficult | Simple |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🎯 What You Can Do Now
|
|
||||||
|
|
||||||
### Immediate
|
|
||||||
|
|
||||||
✅ Run full security analysis
|
|
||||||
✅ Get detailed threat reports
|
|
||||||
✅ Auto-block high-risk IPs
|
|
||||||
✅ Identify false positives
|
|
||||||
✅ Track bandwidth costs
|
|
||||||
|
|
||||||
### Short Term
|
|
||||||
|
|
||||||
📝 Add WordPress cron modules
|
|
||||||
📝 Create custom monitors
|
|
||||||
📝 Build troubleshooting tools
|
|
||||||
☁️ Setup Nextcloud distribution
|
|
||||||
|
|
||||||
### Long Term
|
|
||||||
|
|
||||||
🔄 Automated daily security scans
|
|
||||||
📊 Historical trending dashboards
|
|
||||||
📧 Alert automation
|
|
||||||
🎯 Custom report generation
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📁 File Locations
|
|
||||||
|
|
||||||
### Main Files
|
|
||||||
```
|
|
||||||
/root/server-toolkit/launcher.sh # Run this!
|
|
||||||
/root/server-toolkit/install.sh # One-time setup
|
|
||||||
/root/server-toolkit/README.md # Full docs
|
|
||||||
/root/server-toolkit/SETUP_GUIDE.md # Quick start
|
|
||||||
/root/server-toolkit/WHATS_NEW.md # This file
|
|
||||||
```
|
|
||||||
|
|
||||||
### Bot Analyzer
|
|
||||||
```
|
|
||||||
/root/server-toolkit/modules/security/bot-analyzer.sh # Enhanced v3.0
|
|
||||||
/root/bot_analyzer.sh # Original (backup)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Configuration
|
|
||||||
```
|
|
||||||
/root/server-toolkit/config/settings.conf # Main config
|
|
||||||
/root/server-toolkit/config/whitelist-ips.txt # IP whitelist
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🚀 Getting Started
|
|
||||||
|
|
||||||
### Step 1: Run Installer
|
|
||||||
```bash
|
|
||||||
cd /root/server-toolkit
|
|
||||||
./install.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 2: Launch
|
|
||||||
```bash
|
|
||||||
/root/server-toolkit/launcher.sh
|
|
||||||
# or if symlink created:
|
|
||||||
server-toolkit
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 3: Test Bot Analyzer
|
|
||||||
```
|
|
||||||
Main Menu → 1 (Security) → 1 (Full Bot Analysis)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 4: Configure (Optional)
|
|
||||||
```
|
|
||||||
Main Menu → 9 (Configuration)
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 💡 Key Improvements by Category
|
|
||||||
|
|
||||||
### Security Analysis
|
|
||||||
- 6x more attack types detected
|
|
||||||
- 98% accurate threat scoring
|
|
||||||
- False positive rate < 0.01%
|
|
||||||
- Server IPs never blocked
|
|
||||||
|
|
||||||
### Performance
|
|
||||||
- 60-120x faster processing
|
|
||||||
- Handles millions of log entries
|
|
||||||
- < 1 second for small datasets
|
|
||||||
- Minimal memory usage (~2-4 MB)
|
|
||||||
|
|
||||||
### Usability
|
|
||||||
- Professional menu system
|
|
||||||
- Clear action recommendations
|
|
||||||
- Copy-paste ready blocklists
|
|
||||||
- Detailed progress indicators
|
|
||||||
|
|
||||||
### Maintainability
|
|
||||||
- Modular architecture
|
|
||||||
- Easy to extend
|
|
||||||
- Centralized configuration
|
|
||||||
- Version control ready
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📊 Statistics
|
|
||||||
|
|
||||||
### Code Written Today
|
|
||||||
- Lines of code: ~2,500
|
|
||||||
- Functions created: 20+
|
|
||||||
- Detection patterns: 50+
|
|
||||||
- Menu items: 80+
|
|
||||||
|
|
||||||
### Features Added
|
|
||||||
- Attack vector detection: 6 types
|
|
||||||
- Threat scoring: 8 factors
|
|
||||||
- False positive detection: 5 services
|
|
||||||
- Server IP detection: 5 methods
|
|
||||||
- Performance optimization: 10x - 120x
|
|
||||||
|
|
||||||
### Documentation Created
|
|
||||||
- README.md: Complete system docs
|
|
||||||
- SETUP_GUIDE.md: Quick start guide
|
|
||||||
- WHATS_NEW.md: This summary
|
|
||||||
- Comments: Inline throughout
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🎓 What We Learned
|
|
||||||
|
|
||||||
### Best Practices Implemented
|
|
||||||
✅ Modular architecture
|
|
||||||
✅ Separation of concerns
|
|
||||||
✅ Hash tables for performance
|
|
||||||
✅ Input validation
|
|
||||||
✅ Error handling
|
|
||||||
✅ Progress indicators
|
|
||||||
✅ Configuration management
|
|
||||||
✅ Comprehensive logging
|
|
||||||
|
|
||||||
### Security Principles
|
|
||||||
✅ Never block server IPs
|
|
||||||
✅ Auto-detect false positives
|
|
||||||
✅ Multi-factor threat scoring
|
|
||||||
✅ Configurable thresholds
|
|
||||||
✅ Whitelist management
|
|
||||||
✅ Attack pattern validation
|
|
||||||
|
|
||||||
### Performance Techniques
|
|
||||||
✅ Single-pass file reading
|
|
||||||
✅ O(1) hash table lookups
|
|
||||||
✅ Batch processing
|
|
||||||
✅ Avoid redundant greps
|
|
||||||
✅ Memory-efficient data structures
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🏆 Achievement Unlocked!
|
|
||||||
|
|
||||||
You now have:
|
|
||||||
|
|
||||||
✅ **Enterprise-grade bot detection** (better than commercial tools)
|
|
||||||
✅ **Modular management system** (infinitely extensible)
|
|
||||||
✅ **60-120x performance** (handles massive datasets)
|
|
||||||
✅ **Professional UX** (clean, intuitive, organized)
|
|
||||||
✅ **Nextcloud integration** (easy distribution)
|
|
||||||
✅ **Future-proof architecture** (ready for 80+ modules)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📞 Next Steps
|
|
||||||
|
|
||||||
1. ✅ **Test everything** - Run through all features
|
|
||||||
2. 📝 **Create first custom module** - Try wp-cron-status.sh
|
|
||||||
3. ☁️ **Setup Nextcloud** - Distribute to other servers
|
|
||||||
4. 📧 **Configure alerts** - Email/Slack notifications
|
|
||||||
5. 🔄 **Schedule automation** - Daily security scans
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Version**: 3.0.0
|
|
||||||
**Date**: 2025-10-30
|
|
||||||
**Status**: ✅ Production Ready
|
|
||||||
|
|
||||||
**This is a professional, enterprise-grade system that rivals commercial solutions!** 🎉
|
|
||||||
Reference in New Issue
Block a user