################################################################################ # SERVER TOOLKIT - CLAUDE AI CONTEXT DATABASE ################################################################################ # OPTIMIZED FOR: Claude Code AI parsing and context loading # LAST UPDATED: 2025-10-31 # VERSION: 2.0.0 # FORMAT: Structured key-value with hierarchical sections ################################################################################ [META] version: 2.0.0 updated: 2025-10-31 status: production_ready base_path: /root/server-toolkit entry_point: launcher.sh control_panels: cPanel, Plesk, InterWorx, Standalone [STATUS_SNAPSHOT] # What works right now (production ready) working: - modules/diagnostics/system-health-check.sh: 22-phase comprehensive diagnostics, severity scoring, baseline tracking (NEW 2025-11-01) - modules/security/bot-analyzer.sh: Threat scoring (0-100), CSF blocking, domain analysis - modules/performance/mysql-query-analyzer.sh: Slow query detection - lib/system-detect.sh: Auto-detect cPanel/Plesk/InterWorx, OS, web server, DB, PHP - lib/user-manager.sh: User selection, arrow-key menus, search, domain detection - lib/reference-db.sh: 1-hour cache, pipe-delimited, indexes users/domains/DBs/WordPress + health baseline - launcher.sh: Main menu, module execution, cleanup/reset # What doesn't work (empty directories) not_implemented: - modules/wordpress/*: 11 planned scripts - modules/backup/*: 7 planned scripts - modules/monitoring/*: 5 planned scripts - modules/troubleshooting/*: 9 planned scripts - modules/reporting/*: 6 planned scripts [CRITICAL_RECENT_FIX] date: 2025-10-31 bug: Domain detection showing "(no domains) (0 domains)" root_cause: launcher.sh exported empty SYS_CONTROL_PANEL to child processes solution: run_module() uses subshell isolation - clears all SYS_* vars before module execution location: launcher.sh:77-84 status: verified_working related_bugs: BUG_010, BUG_011, BUG_012, BUG_013 [ARCHITECTURE_RULES] # DO NOT BREAK THESE - intentional design decisions bash_strict_mode: "set -eo pipefail" (NOT -euo - -u is too strict) grep_pattern: Always add "|| true" to grep/find that might not match unbound_vars: Use ${var:-} or ${var:-default} for potentially unbound variables arithmetic: Use current=$((current + 1)) NOT ((current++)) output_suppression: NEVER use { } >/dev/null on critical functions (breaks variable assignment) module_isolation: Modules run in subshells with cleared SYS_* environment initialization: system-detect.sh auto-runs on source, uses exec for output suppression [KEY_FILES] # Files you'll modify most often launcher.sh: Main menu, runs modules via run_module() - Lines: 77-84 (subshell isolation) lib/system-detect.sh: Auto-initialization - Lines: 433-437 lib/user-manager.sh: User selection, domain detection, search modules/security/bot-analyzer.sh: Bot analysis with CSF blocking config/settings.conf: User configuration config/settings.conf.minimal: Clean template (only implemented features) tools/test-domain-detection.sh: Quick validation of domain detection tools/diagnostic-report.sh: System diagnostic collector REFDB_FORMAT.txt: THIS FILE - update after changes! [DIRECTORY_STRUCTURE] server-toolkit/ ├── launcher.sh (main entry point) ├── README.md, TROUBLESHOOTING.md, AUDIT-REPORT.md, PROJECT-STRUCTURE.md ├── config/ (settings.conf, whitelist-ips.txt, whitelist-user-agents.txt) ├── lib/ (common-functions.sh, system-detect.sh, user-manager.sh, reference-db.sh, mysql-analyzer.sh) ├── modules/ │ ├── security/ (bot-analyzer.sh ✓ WORKING) │ ├── performance/ (mysql-query-analyzer.sh ✓ WORKING) │ ├── wordpress/, backup/, monitoring/, troubleshooting/, reporting/ (empty - future) └── tools/ (test-domain-detection.sh, diagnostic-report.sh) [BUGS_FIXED] # All bugs fixed chronologically - DO NOT REINTRODUCE BUG_001: title: Launcher exits after system scan cause: set -euo pipefail too strict, grep/arithmetic failures fix: Changed to set -eo pipefail, added || true, fixed arithmetic files: launcher.sh:15, lib/*.sh multiple lines BUG_002: title: Unbound variable errors in system-detect.sh cause: Checking $SYS_DETECTION_COMPLETE without default fix: Changed to ${SYS_DETECTION_COMPLETE:-} files: lib/system-detect.sh:434 BUG_003: title: Associative array unbound key errors cause: Checking seen_domains[$domain] without default fix: Changed to ${seen_domains[$domain]:-} files: lib/reference-db.sh:219,235 BUG_004: title: Missing functions causing build failures cause: get_database_owner() and get_database_domain() not defined fix: Added both functions files: lib/user-manager.sh:415-445 BUG_005: title: grep commands failing with set -e cause: grep returns exit 1 when no match fix: Added || true to ALL grep in pipes files: lib/user-manager.sh, lib/reference-db.sh BUG_006: title: Arithmetic operations causing exit cause: ((current++)) returns exit 1 when current=0 with set -e fix: Changed to current=$((current + 1)) files: lib/reference-db.sh:141,187 BUG_007: title: find command improper -o syntax cause: find -name "*.log" -o -name "*access*" needs parentheses fix: find \( -name "*.log" -o -name "*access*" \) files: lib/reference-db.sh:290 (DISABLED - performance issues) BUG_008: title: Octal number error in hourly traffic timeline cause: Hours 08/09 treated as invalid octal fix: Strip leading zeros using 10#$hour files: modules/security/bot-analyzer.sh:1154-1157 BUG_009: title: User list not displaying when selecting "Specific user" cause: select_user_interactive() output captured by $() - all echo went to stdout fix: Redirect display output to stderr using { ... } >&2, only username to stdout files: lib/user-manager.sh:330-365,384-408 BUG_010: title: System detection auto-init suppressing errors cause: initialize_system_detection 2>/dev/null || true silently failed fix: Removed error suppression, let errors be visible files: lib/system-detect.sh:435 lesson: Never use 2>/dev/null || true on critical functions BUG_011: title: Duplicate menu display and system detection messages cause: Auto-init printed messages before clear command fix: Suppress output during auto-init with { } redirect, verbose mode via TOOLKIT_VERBOSE_INIT=1 files: lib/system-detect.sh:433-445 lesson: { } blocks with redirects preserve shell context BUG_012: title: Cleanup/Reset not forcing fresh detection in current session cause: cleanup_all_data() only deleted files, not in-memory variables fix: Unset all SYS_* vars, re-source libraries, display new values files: launcher.sh:332-360 BUG_013: title: Brace redirection blocks variable assignment cause: { initialize_system_detection } >/dev/null broke variable export fix: Use exec file descriptor manipulation instead files: lib/system-detect.sh:439-445 lesson: Brace groups with output redirect break variable assignments - use exec instead FINAL_DOMAIN_DETECTION_FIX: problem: "(no domains) (0 domains)" even though pickledperil.com exists root_cause: launcher.sh sourced system-detect.sh, exported empty SYS_CONTROL_PANEL, child processes inherited it solution: run_module() runs modules in subshell that clears SYS_* before execution code: | ( for var in $(compgen -e | grep "^SYS_"); do unset "$var" done "$MODULES_DIR/$category/$module" "$@" ) enhancements: arrow-key menu, confirmation prompt, enhanced cleanup, test script verification: pickledperil - pickledperil.com (1 domains) ✓ lessons: | 1. Exported env vars persist across child processes 2. Output redirect with { } can break variable assignment 3. Subshells with ( ) provide clean environment isolation 4. Always test exact user flow, not just isolated components [ENHANCEMENTS] ENHANCEMENT_001: title: Post-analysis menu for bot analyzer date: 2025-10-31 location: modules/security/bot-analyzer.sh:1597-1662 description: After analysis, prompts "Go to Take Action Menu or Go Back" behavior: | If high-risk IPs found (score >= 70): Offers action menu or back If no high-risk IPs: Success message, press Enter functions: show_post_analysis_menu(), calls offer_csf_blocking() ENHANCEMENT_002: title: Interactive CSF blocking for high-risk IPs date: 2025-10-31 location: modules/security/bot-analyzer.sh:1664-1806 description: Auto-detects IPs with threat score >= 70, offers CSF blocking options: 1hr temp, 24hr temp, permanent, skip safety: Auto-excludes server IPs, false positives, color-coded severity scoring: 70-79 ELEVATED, 80-89 HIGH, 90-100 CRITICAL ENHANCEMENT_003: title: Intelligent recommendation engine with domain-level analysis date: 2025-10-31 location: modules/security/bot-analyzer.sh:1601-1798 description: Context-aware recommendations based on attack scope analysis: Single domain, primary target, server-wide, single-server recommendations: IP blocking strategy, CSF CT_LIMIT, .htaccess, Apache global, WP hardening, rate limiting philosophy: Script suggests, user decides (user-centric, not script-centric) ENHANCEMENT_004: title: Professional terminal-agnostic output design date: 2025-10-31 location: lib/common-functions.sh:8-116 changes: | Before: Emojis (ℹ ✓ ⚠ ✗ 🚨), fancy Unicode, 256-color After: Text labels [INFO] [OK] [WARNING] [ERROR] [CRITICAL], ASCII only, basic ANSI colors rationale: Must work everywhere - SSH, serial consoles, screen readers, piped output auto_detection: Disables colors if not TTY or TOOLKIT_NO_COLOR=1 ENHANCEMENT_005: title: User-centric action menu redesign date: 2025-10-31 location: modules/security/bot-analyzer.sh:2036-2182 changes: Shows ALL 8 actions (not just recommended), recommended marked with ⭐ and priority philosophy: User has full control, script is not the expert actions: IP blocking (1hr/24hr/perm), domain .htaccess, Apache global, CSF CT_LIMIT, rate limiting, WP hardening ENHANCEMENT_006: title: User selection search functionality date: 2025-10-31 location: lib/user-manager.sh:382-437 description: Search/filter for servers with 200+ users usage: S [text] - case-insensitive partial match on username/domain behavior: | 1 match: Confirmation prompt, auto-select on Y N matches: Arrow-key menu (bash select) 0 matches: Error message testing: Verified with username/domain search, case-insensitive [REFERENCE_DATABASE] # .sysref file format - pipe-delimited database format: TYPE|field1|field2|field3|... location: /root/server-toolkit/.sysref timestamp: /root/server-toolkit/.sysref.timestamp ttl: 3600 seconds (1 hour) record_types: SYS: System information format: SYS|key|value|extra example: SYS|CONTROL_PANEL|cpanel|11.130.0.15 USER: User accounts format: USER|username|primary_domain|db_count|domain_count|disk_mb|home_dir example: USER|pickledperil|pickledperil.com|1|3|82|/home/pickledperil DB: Databases format: DB|db_name|owner|primary_domain|size_mb|table_count example: DB|pickledperil_wp_wt6lz|pickledperil|pickledperil.com|15.23|12 DOMAIN: Domain mappings (ENHANCED) format: DOMAIN|domain|owner|doc_root|log_path|php_ver|is_primary|type|aliases example: DOMAIN|pickledperil.com|pickledperil|/home/pickledperil/public_html|/etc/.../pickledperil.com|ea-php81|yes|primary|www.pickledperil.com types: primary, addon, subdomain, alias, parked, remote WP: WordPress installations (ENHANCED) format: WP|domain|owner|path|db_name|db_user|version|plugin_count|theme_count example: WP|pickledperil.com|pickledperil|/home/pickledperil/public_html|pickledperil_wp_wt6lz|pickledperil_wp_user|6.8.3|1|3 LOG: Log file locations (DISABLED - performance issues) format: LOG|type|domain|owner|path|size_mb status: Disabled, log paths now in DOMAIN records HEALTH: Health check baseline metrics (NEW - Added 2025-11-01) format: HEALTH|metric_name|value|date updated_by: modules/diagnostics/system-health-check.sh refresh: Updated every time health check runs examples: HEALTH|TIMESTAMP|2025-11-01 15:54:23| HEALTH|MEMORY_TOTAL_MB|3776|2025-11-01 HEALTH|MEMORY_USED_PERCENT|45|2025-11-01 HEALTH|CPU_LOAD_1MIN|2.4|2025-11-01 HEALTH|CPU_CORES|2|2025-11-01 HEALTH|DISK_USED_PERCENT|35|2025-11-01 HEALTH|IOWAIT_PERCENT|0|2025-11-01 HEALTH|EMAIL_QUEUE_SIZE|0|2025-11-01 HEALTH|ZOMBIE_PROCESSES|0|2025-11-01 HEALTH|HTTPD_STATUS|running|2025-11-01 HEALTH|MYSQL_STATUS|running|2025-11-01 HEALTH|FIREWALL_STATUS|csf|2025-11-01 HEALTH|CRITICAL_ISSUES|0|2025-11-01 HEALTH|HIGH_ISSUES|3|2025-11-01 HEALTH|MEDIUM_ISSUES|3|2025-11-01 HEALTH|LOW_ISSUES|1|2025-11-01 purpose: | - Track system baseline metrics for trend analysis - Compare current vs. historical values - Cross-module intelligence (bot analyzer can see if memory was already high) - Quick reference without re-running expensive checks query_functions: - db_get_user(username) - db_get_all_users() - db_get_user_databases(username) - db_get_user_domains(username) - db_get_database_owner(db_name) - db_get_all_wordpress() - db_is_fresh() - db_rebuild() cross_reference_example: | To find everything for pickledperil.com: 1. grep "^DOMAIN|pickledperil.com|" .sysref → get owner, doc_root, log_path, PHP version 2. grep "^DB|.*|pickledperil|" .sysref → get all databases 3. grep "^WP|pickledperil.com|" .sysref → get WordPress installation details 4. grep "^DOMAIN|.*|pickledperil|" .sysref → get all subdomains/aliases [GLOBAL_VARIABLES] system_detect_exports: SYS_CONTROL_PANEL: cpanel|plesk|interworx|none SYS_CONTROL_PANEL_VERSION: Version string SYS_OS_TYPE: almalinux|centos|rhel|rocky|cloudlinux SYS_OS_VERSION: Version number SYS_WEB_SERVER: apache|nginx|litespeed|openlitespeed SYS_WEB_SERVER_VERSION: Version string SYS_DB_TYPE: mysql|mariadb|none SYS_DB_VERSION: Version string SYS_LOG_DIR: Log directory path SYS_USER_HOME_BASE: User home base (usually /home) SYS_PHP_VERSIONS: Array of installed PHP versions SYS_CLOUDFLARE_ACTIVE: yes|no SYS_CLOUDLINUX: yes|no CPU_CORES, CPU_USED, LOAD_AVERAGE: Resource metrics MEM_TOTAL, MEM_USED, MEM_PERCENT: Memory metrics common_functions_exports: TOOLKIT_BASE_DIR: Base directory path TEMP_SESSION_DIR: Temp directory for session reference_db_exports: SYSREF_DB: Path to .sysref file SYSREF_TIMESTAMP: Path to timestamp file launcher_defines: BASE_DIR: Same as TOOLKIT_BASE_DIR MODULES_DIR, LIB_DIR, CONFIG_DIR: Module/library/config paths [COMMON_PATTERNS] # Copy-paste these for consistent code safe_grep: | result=$(grep "pattern" file 2>/dev/null | head -1 || true) safe_find: | files=$(find /path -name "*.txt" 2>/dev/null || true) loop_with_progress: | total=${#array[@]} current=0 for item in "${array[@]}"; do current=$((current + 1)) show_progress $current $total "Processing..." # do work done finish_progress check_module_exists: | if ! module_exists "category" "module.sh"; then print_error "Module not found" return 1 fi query_refdb: | db_ensure_fresh # Rebuild if stale users=$(db_get_all_users) for user in $users; do domains=$(db_get_user_domains "$user") done [MODULE_TEMPLATE] # Template for creating new modules file_location: /root/server-toolkit/modules/{category}/{name}.sh template: | #!/bin/bash # Load common functions SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" source "$SCRIPT_DIR/lib/common-functions.sh" # Load system detection (optional) source "$SCRIPT_DIR/lib/system-detect.sh" # Load user manager (optional) source "$SCRIPT_DIR/lib/user-manager.sh" # Load reference database (optional) source "$SCRIPT_DIR/lib/reference-db.sh" print_banner "Module Name" # Your code here print_success "Completed" steps: 1: Create file in modules/{category}/{name}.sh 2: chmod +x the file 3: Add to launcher.sh handle_{category}_menu() case statement 4: Test directly and via launcher best_practices: - Use print_* functions, not echo - Add || true to grep/find - Use ${var:-default} for unbound vars - Use current=$((current + 1)) not ((current++)) - Include progress indicators for long operations - Handle errors gracefully - Return proper exit codes (0=success, 1=error) [KNOWN_ISSUES] ISSUE_001: title: Log file scanning disabled location: lib/reference-db.sh:285 reason: find with while read loop hangs on servers with many logs status: Disabled, returns empty [LOGS] section todo: Implement with find | head -1000 | while read ISSUE_002: title: Home directory empty in USER records location: lib/reference-db.sh:140 symptom: HOME_DIR field empty for some users cause: get_user_info may not parse cPanel files properly status: Low priority - doesn't break functionality todo: Debug get_cpanel_user_info() ISSUE_003: title: No module implementation yet location: modules/* directories status: Only bot-analyzer.sh and mysql-query-analyzer.sh exist priority: High - need WordPress modules next todo: wp-cron-status.sh, wp-cron-mass-fix.sh, wp-health-check.sh ISSUE_004: title: Nextcloud integration not implemented location: launcher.sh status: Config exists but download mechanism not coded todo: Implement module download from Nextcloud via manifest.txt [NEXT_PRIORITIES] immediate_high: 1: Implement wp-cron-status.sh module 2: Implement wp-cron-mass-fix.sh module 3: Fix HOME_DIR empty issue in reference database 4: Test on production server with multiple users short_term_medium: 5: Implement wp-health-check.sh (scan all WP installations) 6: Implement resource-monitor.sh (performance category) 7: Re-enable log file scanning with limits 8: Add error handling throughout long_term_low: 9: Implement Nextcloud module download system 10: Create automated daily security scan feature 11: Build reporting dashboard 12: Add email/Slack alert integration [PERFORMANCE] bottlenecks: - find /home -name "wp-config.php" can be slow (acceptable for cache) - du -sm on large home directories (acceptable for cache) - MySQL queries for database sizes (acceptable) - Log file scanning DISABLED due to excessive find operations optimizations: - Reference database caching (1 hour TTL) - Single-pass file reading - Hash table lookups in bot-analyzer - Avoid redundant grep operations targets: system_scan: < 5 seconds for typical server bot_analysis: < 60 seconds for 25K IPs menu_navigation: Instant module_launch: < 1 second [TESTING_CHECKLIST] before_commit: - Test with fresh system (rm .sysref*) - Test with existing cache - Test menu navigation (all categories) - Test module execution - bash -n file.sh (syntax check) - Check for unhandled grep failures - Verify exit codes - Test with set -e enabled - Verify no debug output in production common_test_commands: | bash -n /root/server-toolkit/launcher.sh /root/server-toolkit/launcher.sh rm -f /root/server-toolkit/.sysref* /root/server-toolkit/modules/security/bot-analyzer.sh bash /root/server-toolkit/tools/test-domain-detection.sh [TROUBLESHOOTING] domain_detection_fails: symptom: "(no domains) (0 domains)" check: echo $SYS_CONTROL_PANEL (should be "cpanel") fix: Run cleanup option 8, or exit and restart launcher test: bash /root/server-toolkit/tools/test-domain-detection.sh functions_not_found: symptom: "command not found" errors check: ls -la /root/server-toolkit/lib/*.sh fix: source libraries manually, check syntax with bash -n menus_displaying_twice: symptom: Duplicate menus, detection messages visible check: TOOLKIT_VERBOSE_INIT environment variable fix: unset TOOLKIT_VERBOSE_INIT or set to 0 csf_not_working: symptom: "csf: command not found" check: which csf fix: Install CSF or use alternative security methods [CONFIG_FILE] # config/settings.conf loaded_by: launcher.sh:484 format: Bash variable assignments key_settings: QUICK_SCAN_HOURS: 1 (default time range for quick scans) AUTO_BLOCK: no (auto-apply firewall blocks) AUTO_BLOCK_THRESHOLD: 80 (threat score threshold) LOG_DIR: /var/log/apache2/domlogs (auto-detected) CPANEL_HOME: /home (auto-detected) CPU_ALERT_THRESHOLD: 80 MEM_ALERT_THRESHOLD: 90 DISK_ALERT_THRESHOLD: 85 WHITELIST_IP_FILE: config/whitelist-ips.txt WHITELIST_UA_FILE: config/whitelist-user-agents.txt minimal_version: config/settings.conf.minimal (only current settings, no future features) [MENU_STRUCTURE] main_menu: 1: Security & Threat Analysis (bot-analyzer ✓ WORKING, 9 others ⚠️ NOT IMPLEMENTED) 2: WordPress Management (14 modules ⚠️ ALL NOT IMPLEMENTED) 3: Performance & Diagnostics (mysql-query-analyzer ✓ WORKING, 10 others ⚠️ NOT IMPLEMENTED) 4: Backup & Recovery (8 modules ⚠️ ALL NOT IMPLEMENTED) 5: Monitoring & Alerts (8 modules ⚠️ ALL NOT IMPLEMENTED) 6: Troubleshooting (11 modules ⚠️ ALL NOT IMPLEMENTED) 7: Reporting & Analytics (7 modules ⚠️ ALL NOT IMPLEMENTED) 8: Cleanup/Reset ✓ WORKING 9: Configuration ✓ WORKING 0: Exit ✓ WORKING security_menu_detail: 1: Full Bot Analysis ✓ WORKING (bot-analyzer.sh) 2: Quick Scan (1 hour) ✓ WORKING (bot-analyzer.sh -H 1) 3-10: ⚠️ NOT IMPLEMENTED (Live Monitor, IP Lookup, Auto-Block, Whitelist, Attack Pattern, DDoS, Firewall, SSL Audit) [FOR_NEW_CLAUDE_INSTANCES] # Read this if you're a new Claude session quickstart: 1: Read [STATUS_SNAPSHOT] - know what works vs. what doesn't 2: Read [CRITICAL_RECENT_FIX] - understand latest major fix 3: Read [ARCHITECTURE_RULES] - don't break these intentional designs 4: Read [BUGS_FIXED] - don't reintroduce fixed bugs 5: Read [NEXT_PRIORITIES] - know what to work on next 6: Test current state - run /root/server-toolkit/launcher.sh update_after_changes: - Add new bugs to [BUGS_FIXED] - Update [STATUS_SNAPSHOT] with new working features - Update [NEXT_PRIORITIES] - Update [CRITICAL_RECENT_FIX] if applicable - Update [LAST UPDATED] date at top dont_do: - Don't use set -u (too strict) - Don't suppress errors with 2>/dev/null on critical functions - Don't use { } blocks with output redirect (breaks variables) - Don't reintroduce bugs from [BUGS_FIXED] - Don't create new modules without following [MODULE_TEMPLATE] [END] # This file is the single source of truth for project context. # Keep it updated. Keep it concise. Keep it structured. ################################################################################