# Server Toolkit - Troubleshooting Guide

## Quick Diagnostics

### Test Domain Detection
```bash
bash /root/server-toolkit/tools/test-domain-detection.sh
```
This will tell you immediately if domain detection is working.

### Check System Detection Variables
```bash
bash -c '
source /root/server-toolkit/lib/system-detect.sh
echo "SYS_CONTROL_PANEL: [$SYS_CONTROL_PANEL]"
echo "SYS_DETECTION_COMPLETE: [$SYS_DETECTION_COMPLETE]"
'
```
Both should have values. If empty, system detection failed.

### Test User Domain Lookup
```bash
bash -c '
source /root/server-toolkit/lib/system-detect.sh
source /root/server-toolkit/lib/user-manager.sh
get_user_domains "USERNAME"
'
```
Replace USERNAME with actual username. Should return domain(s).

---

## Common Issues

### Issue: User shows "(no domains) (0 domains)"

**Symptoms:**
- User selection menu shows 0 domains
- Bot analyzer says "No domains found for user"
- Domain exists in cPanel

**Diagnosis:**
1. Run: `echo $SYS_CONTROL_PANEL` in your shell
2. If empty, environment is corrupted

**Fix:**
- Option 1: Exit launcher completely and restart
- Option 2: Select option 8 (Cleanup/Reset) in launcher
- Option 3: Close entire SSH session and reconnect

**Why it happens:**
Launcher inherited broken environment variables from a previous session where
libraries had bugs. Child processes (like bot-analyzer) inherit these.

---

### Issue: Functions not found / command not found

**Symptoms:**
- `bash: select_user_interactive: command not found`
- `bash: get_user_domains: command not found`

**Diagnosis:**
Libraries weren't sourced correctly.

**Fix:**
1. Check that files exist:
```bash
ls -la /root/server-toolkit/lib/*.sh
```

2. Test sourcing manually:
```bash
source /root/server-toolkit/lib/system-detect.sh
source /root/server-toolkit/lib/user-manager.sh
```

3. Check for syntax errors:
```bash
bash -n /root/server-toolkit/lib/system-detect.sh
bash -n /root/server-toolkit/lib/user-manager.sh
```

---

### Issue: Menus displaying twice or garbled output

**Symptoms:**
- Same menu appears multiple times
- Detection messages appear before menus
- ANSI codes visible like `[H[J`

**Diagnosis:**
Terminal doesn't support ANSI codes or clear screen.

**Fix:**
Set high contrast mode:
```bash
export TOOLKIT_HIGH_CONTRAST=1
bash /root/server-toolkit/launcher.sh
```

Or disable colors completely:
```bash
export TOOLKIT_NO_COLOR=1
bash /root/server-toolkit/launcher.sh
```

---

### Issue: CSF commands not working

**Symptoms:**
- "csf: command not found"
- CSF blocking options don't work

**Diagnosis:**
CSF not installed or not in PATH.

**Check:**
```bash
which csf
csf -v
```

**Fix:**
Install CSF or use alternative security methods (Apache .htaccess, etc.)

---

### Issue: cPanel users not detected

**Symptoms:**
- "No users found"
- list_all_users returns nothing

**Diagnosis:**
Check if cPanel user files exist:
```bash
ls -la /var/cpanel/users/
cat /etc/trueuserdomains | head
```

**Fix:**
If files missing, not a cPanel system. System will fall back to standard
user detection from /etc/passwd.

---

## Debug Mode

### Enable Verbose Initialization
```bash
export TOOLKIT_VERBOSE_INIT=1
bash /root/server-toolkit/launcher.sh
```
Shows all system detection messages.

### Trace Execution
```bash
bash -x /root/server-toolkit/modules/security/bot-analyzer.sh 2>&1 | less
```
Shows every command executed (very verbose).

### Check Environment Variables
```bash
# Show all SYS_* variables
env | grep "^SYS_"

# Show all toolkit-related variables
env | grep -i toolkit
```

---

## File Locations

### Logs
- Bot analysis reports: `/tmp/bot_analysis_report_*.txt`
- MySQL analysis: `/tmp/mysql_analysis_*.txt`
- Temp sessions: `/tmp/server-toolkit-*`

### Cache Files
- System reference database: `/root/server-toolkit/.sysref`
- Timestamp file: `/root/server-toolkit/.sysref.timestamp`

### Configuration
- Settings: `/root/server-toolkit/config/settings.conf`
- Custom slash commands: `/root/server-toolkit/.claude/commands/`

---

## Performance Issues

### Issue: Slow user selection with 200+ users

**Fix:**
- Use search: `s <partial-name>`
- Searches only, doesn't list all users
- Much faster than 'L' (list all)

### Issue: Bot analyzer takes too long

**Optimization:**
1. Use time filters: Last 1 hour instead of "All logs"
2. Use user filter: Analyze specific user instead of all
3. Check log size: `du -sh /var/log/apache2/domlogs/*`

---

## Recovery Commands

### Complete Reset
```bash
# In launcher, select option 8 (Cleanup/Reset)
# Or manually:
rm -f /root/server-toolkit/.sysref*
rm -rf /tmp/server-toolkit-*
rm -f /tmp/bot_analysis_* /tmp/mysql_analysis_*
```

### Force Library Reload
```bash
# In bash session:
for var in $(compgen -e | grep "^SYS_"); do unset "$var"; done
unset -f initialize_system_detection get_user_domains select_user_interactive
source /root/server-toolkit/lib/system-detect.sh
source /root/server-toolkit/lib/user-manager.sh
```

### Kill Stuck Processes
```bash
# Find launcher processes
ps aux | grep launcher

# Kill specific PID
kill -9 <PID>

# Kill all launcher instances
pkill -9 -f launcher.sh
```

---

## Getting Help

### Self-Diagnostic
1. Run test script: `bash /root/server-toolkit/tools/test-domain-detection.sh`
2. Check REFDB_FORMAT.txt for known bugs and fixes
3. Review this troubleshooting guide

### Report Issues
When reporting problems, include:
1. Output of test-domain-detection.sh
2. Output of: `env | grep "^SYS_"`
3. Control panel type: `cat /usr/local/cpanel/version` or equivalent
4. Error messages (exact text)
5. Steps to reproduce

### Quick Fixes to Try First
1. Exit and restart launcher
2. Run Cleanup/Reset (option 8)
3. Close SSH and reconnect
4. Run test-domain-detection.sh to verify files are correct

---

## Version Information

**Created:** 2025-10-31
**Last Updated:** 2025-10-31
**Toolkit Version:** 2.0.0
**Compatible With:** cPanel, Plesk, InterWorx, Standalone Linux servers