cschantz/Linux-Server-Management-Toolkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update documentation for MySQL restore tool and backup module

Browse Source 
Main README.md:
- Added mysql-restore-to-sql.sh to directory structure
- Created dedicated Backup & Recovery section with subsections
- Documented MySQL restore tool features:
  - Multi-control panel support
  - Intelligent Force Recovery detection
  - Safe selective restore capabilities
  - Safety features (disk space, directory protection, warnings)
  - Clean SQL export functionality
- Added MySQL restore usage example
- Updated Recent Updates section with new tool features

modules/backup/README.md (NEW):
- Comprehensive documentation for backup module
- Acronis Cyber Protect integration section:
  - All 16 scripts documented with purposes
  - Usage examples and features
- MySQL/MariaDB Database Restore Tool section:
  - Key features and capabilities
  - Control panel path support details
  - Force Recovery levels explained
  - Smart detection for selective restore
  - Use cases and safety guarantees
  - Step-by-step wizard documentation
  - Technical details (second instance, file requirements)
  - Error detection and recovery procedures
- Integration with launcher documented
- Requirements and recent updates listed

Documentation Status:
- Main README updated with new tool
- Backup module README created from scratch
- All recent changes documented (InterWorx paths, smart detection, etc.)
- Ready for user testing
This commit is contained in:
cschantz
2025-12-10 23:07:11 -05:00
parent 915ef2236c
commit fccb714cce
2 changed files with 410 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+33 -4
View File
@@ -25,8 +25,8 @@ server-toolkit/
│ │ ├── ip-reputation-manager.sh # Centralized IP reputation tracking │ │ ├── ip-reputation-manager.sh # Centralized IP reputation tracking
│ │ └── tail-*.sh # Various log monitoring scripts │ │ └── tail-*.sh # Various log monitoring scripts
│ │ │ │
│ ├── backup/ # 💾 Backup & Recovery (Acronis Cyber Protect) │ ├── backup/ # 💾 Backup & Recovery
│ │ ├── acronis-backup-manager.sh # Main backup management menu │ │ ├── acronis-backup-manager.sh # Main backup management menu (Acronis Cyber Protect)
│ │ ├── acronis-install.sh # Install Acronis agent │ │ ├── acronis-install.sh # Install Acronis agent
│ │ ├── acronis-update.sh # Update Acronis agent │ │ ├── acronis-update.sh # Update Acronis agent
│ │ ├── acronis-uninstall.sh # Uninstall Acronis agent │ │ ├── acronis-uninstall.sh # Uninstall Acronis agent
@@ -40,7 +40,8 @@ server-toolkit/
│ │ ├── acronis-schedule-viewer.sh # View backup schedules │ │ ├── acronis-schedule-viewer.sh # View backup schedules
│ │ ├── acronis-restore.sh # Restore from backup │ │ ├── acronis-restore.sh # Restore from backup
│ │ ├── acronis-logs.sh # View Acronis logs │ │ ├── acronis-logs.sh # View Acronis logs
│ │ └── acronis-troubleshoot.sh # Troubleshoot common issues │ │ ├── acronis-troubleshoot.sh # Troubleshoot common issues
│ │ └── mysql-restore-to-sql.sh # Advanced MySQL/MariaDB database restore & dump tool
│ │ │ │
│ ├── website/ # 🌐 Website Diagnostics & Troubleshooting │ ├── website/ # 🌐 Website Diagnostics & Troubleshooting
│ │ ├── website-error-analyzer.sh # Comprehensive website error analysis │ │ ├── website-error-analyzer.sh # Comprehensive website error analysis
@@ -105,7 +106,8 @@ source /root/linux-server-management-toolkit/run.sh
- **IP Reputation Tracking**: Centralized cross-module IP intelligence - **IP Reputation Tracking**: Centralized cross-module IP intelligence
- **Multi-Source Monitoring**: SSH, Web, Firewall, cPHulk integration - **Multi-Source Monitoring**: SSH, Web, Firewall, cPHulk integration
### 💾 Backup & Recovery (Acronis Cyber Protect) ### 💾 Backup & Recovery
#### Acronis Cyber Protect Integration
- **Complete Agent Management**: Install, update, uninstall, register - **Complete Agent Management**: Install, update, uninstall, register
- **Comprehensive Status Monitoring**: Agent health, registration, cloud connectivity - **Comprehensive Status Monitoring**: Agent health, registration, cloud connectivity
- **Manual Backup Triggering**: CLI-managed plans with performance optimizations - **Manual Backup Triggering**: CLI-managed plans with performance optimizations
@@ -114,6 +116,15 @@ source /root/linux-server-management-toolkit/run.sh
- **Restore Operations**: Full restore capabilities from backups - **Restore Operations**: Full restore capabilities from backups
- **Troubleshooting Tools**: Log viewing and automated diagnostics - **Troubleshooting Tools**: Log viewing and automated diagnostics
#### MySQL/MariaDB Database Restore Tool (NEW!)
- **Advanced Database Recovery**: Restore individual databases from file-based backups (Acronis, raw file backups)
- **Multi-Control Panel Support**: Automatic path detection for cPanel, InterWorx, Plesk, standalone
- **Intelligent Force Recovery**: Smart detection for missing tablespace files from other databases
- **Safe Selective Restore**: Restore single database from full backup without data loss
- **Guided Recovery Process**: Step-by-step wizard with automatic error detection
- **Safety Features**: Disk space validation, critical directory protection, recovery level warnings
- **Clean SQL Export**: Dumps restored database to clean SQL file for import into production
### 🌐 Website Diagnostics ### 🌐 Website Diagnostics
- **Error Analysis**: Comprehensive website error detection and troubleshooting - **Error Analysis**: Comprehensive website error detection and troubleshooting
- **500 Error Tracking**: Detailed analysis of application errors - **500 Error Tracking**: Detailed analysis of application errors
@@ -176,6 +187,20 @@ bash launcher.sh
# Select: Manage Protection Plans # Select: Manage Protection Plans
``` ```
### MySQL Database Restore & Recovery
```bash
bash launcher.sh
# Select: Backup & Recovery
# Select: MySQL/MariaDB Database Restore to SQL
# Follow guided wizard:
# 1. Provide backup files (ibdata1, ib_logfile*, database folders)
# 2. Select database to restore
# 3. Configure MySQL settings
# 4. Choose Force Recovery level (with intelligent recommendations)
# 5. Restore and dump to clean SQL file
```
### Website Error Analysis ### Website Error Analysis
```bash ```bash
@@ -240,6 +265,10 @@ nano /root/server-toolkit/config/settings.conf
- ✅ Comprehensive status monitoring (health, registration, connectivity) - ✅ Comprehensive status monitoring (health, registration, connectivity)
- ✅ Manual backup triggering with performance optimizations - ✅ Manual backup triggering with performance optimizations
- ✅ Protection plan management and scheduling - ✅ Protection plan management and scheduling
- ✅ MySQL/MariaDB database restore tool with intelligent recovery (NEW!)
- ✅ Multi-control panel support for database restoration paths
- ✅ Smart detection for Force Recovery Level selection
- ✅ Safe selective database restore from full backups
### Website Diagnostics ### Website Diagnostics
- ✅ Comprehensive website error analyzer (multi-panel) - ✅ Comprehensive website error analyzer (multi-panel)
modules/backup/README.md
+377
View File
@@ -0,0 +1,377 @@
# Backup & Recovery Module
Comprehensive backup and database recovery tools for server management.
## Overview
This module provides two major subsystems:
1. **Acronis Cyber Protect Integration** - Complete backup agent management
2. **MySQL/MariaDB Database Restore Tool** - Advanced database recovery from file-based backups
---
## Acronis Cyber Protect Integration
Complete command-line management for Acronis Cyber Protect backup agent on Linux servers.
### Features
- Full agent lifecycle management (install, update, uninstall)
- Cloud registration and configuration
- Manual backup triggering with performance optimizations
- Protection plan management
- Backup status monitoring and scheduling
- Comprehensive troubleshooting and log viewing
### Scripts
#### Agent Management
- **acronis-install.sh** - Install Acronis agent from local file or download
- **acronis-update.sh** - Update agent to latest version
- **acronis-uninstall.sh** - Clean uninstallation of agent
- **acronis-register.sh** - Register agent with Acronis Cloud
- **acronis-configure.sh** - Configure agent settings
#### Monitoring & Status
- **acronis-agent-status.sh** - Comprehensive agent health check
- Registration status
- Cloud connectivity
- Service status
- Version information
- **acronis-backup-status.sh** - Check backup job status
- **acronis-list-backups.sh** - List all available backups
- **acronis-schedule-viewer.sh** - View backup schedules
#### Backup Operations
- **acronis-trigger-backup.sh** - Manually trigger backups
- Full backup support
- Incremental backup support
- Differential backup support
- Performance optimizations (nice, ionice)
- **acronis-plan-manager.sh** - Manage protection plans
- View plans
- Enable/disable plans
- Delete plans
- **acronis-restore.sh** - Restore from backups
#### Troubleshooting
- **acronis-logs.sh** - View Acronis logs
- Real-time log monitoring
- Historical log viewing
- Filtered log search
- **acronis-troubleshoot.sh** - Automated diagnostics
- Common issue detection
- Fix recommendations
- Health checks
#### Menu System
- **acronis-backup-manager.sh** - Interactive menu for all Acronis operations
### Usage Example
```bash
# Check agent status
./acronis-agent-status.sh
# Trigger manual backup
./acronis-trigger-backup.sh
# View backup schedules
./acronis-schedule-viewer.sh
# Manage protection plans
./acronis-plan-manager.sh
```
---
## MySQL/MariaDB Database Restore Tool
**Script**: `mysql-restore-to-sql.sh`
Advanced database recovery tool for restoring individual databases from file-based backups (Acronis, raw file backups, etc.) and exporting them to clean SQL files.
### Key Features
#### Multi-Control Panel Support
- **cPanel**: Uses `/home` for restore directory
- **InterWorx**: Uses `/chroot/home` (actual path, not symlink)
- **Plesk**: Uses `/var/www/vhosts`
- **Standalone**: Uses `/home` as fallback
Automatic detection via `lib/system-detect.sh` ensures correct paths for all control panels.
#### Intelligent Force Recovery
- **Smart Detection**: Automatically identifies when missing tablespace files are from OTHER databases (not the one you're restoring)
- **Safe Recommendations**: Suggests Force Recovery Level 1 when appropriate for selective database restore
- **No Data Loss**: Force Recovery Level 1 ignores missing databases you don't have while preserving all data from databases you DO have
#### Safety Features
- **Disk Space Validation**: Ensures 2x required space before starting
- **Critical Directory Protection**: Prevents using `/var/lib/mysql` as restore directory
- **Force Recovery Warnings**: Risk acknowledgment for levels 5-6
- **Automatic Cleanup**: Trap handler for Ctrl+C/interruption
- **Backup-Free Operation**: Works in temporary directory, never touches production MySQL
#### Guided Wizard Process
The tool provides a step-by-step guided process:
**Step 1: Gather Backup Files**
- Collect required files: `ibdata1`, `ib_logfile0`, `ib_logfile1`, database folders
- Copy files to suggested restore directory (e.g., `/home/temp/restore20251210/mysql/`)
**Step 2: Select Database**
- Lists all databases found in backup
- Select which database to restore
**Step 3: Configure MySQL Settings**
- Port selection (default: 13306 to avoid conflicts)
- Timeout configuration
- Option to verify file integrity
**Step 4: Configure Recovery Options**
- Choose InnoDB Force Recovery level (0-6)
- Shows intelligent recommendations based on detected issues
- Explains risks and benefits of each level
**Step 5: Restore & Dump**
- Starts temporary MySQL instance in restore directory
- Monitors startup for errors
- Provides intelligent recovery guidance if issues detected
- Dumps selected database to clean SQL file
- Automatic cleanup of temporary MySQL instance
### SQL Output Location
SQL files are saved to the **parent directory** of the restore directory:
```
Restore Directory: /home/temp/restore20251210/mysql/
SQL Output Location: /home/temp/restore20251210/database_restored_20251210_150530.sql
```
This prevents cluttering control panel system directories and keeps output organized with restore files.
### Force Recovery Levels
The tool supports all InnoDB Force Recovery levels with clear explanations:
- **Level 0**: Normal operation (no recovery)
- **Level 1**: Ignore corrupt pages/missing tablespaces (safe for selective restore)
- **Level 2**: Stop master thread operations
- **Level 3**: Skip transaction rollback
- **Level 4**: Skip insert buffer merge
- **Level 5**: Ignore undo logs (data loss risk)
- **Level 6**: Skip redo log recovery (data loss risk)
### Smart Detection for Selective Restore
When you restore a single database from a full backup:
**Problem**: The `ibdata1` file contains metadata for ALL databases from the original backup. If you only restored one database folder, MySQL will report missing tablespace files for all the other databases.
**Solution**: The tool detects this scenario and recommends Force Recovery Level 1:
```
SMART DETECTION: Missing files are from OTHER databases, not 'yourdatabase'
Your selected database 'yourdatabase' appears to have all files!
RECOMMENDED ACTION: Use Force Recovery Level 1
The ibdata1 file contains references to databases you didn't restore.
Force Recovery Level 1 will:
✓ Ignore missing databases (safe - you don't have them anyway)
✓ Start MySQL successfully
✓ Allow you to dump 'yourdatabase' with NO data loss
This is the CORRECT approach for selective database restoration.
```
### Use Cases
#### Restore Single Database from Full Backup
1. You have an Acronis backup containing all databases
2. You only want to restore one specific database
3. Tool detects missing files from other databases
4. Recommends Force Recovery Level 1
5. Successfully dumps your database without data loss
#### Recover from Corrupt Backup
1. Backup has some corrupt tables
2. Tool attempts normal restore
3. Detects corruption errors
4. Recommends appropriate Force Recovery level
5. Extracts maximum recoverable data
#### Import Older Database Version
1. Restore older version of database from backup
2. Dump to SQL file
3. Drop tables in production database (keeps permissions)
4. Import SQL dump
### Safety Guarantees
- **Never touches production MySQL** - Uses isolated temporary instance
- **Disk space validation** - Ensures sufficient space before starting
- **Critical directory protection** - Prevents dangerous restore locations
- **Smart recommendations** - Only suggests recovery when safe
- **Clean SQL output** - Produces importable SQL file, not raw data files
### Control Panel Path Support
The tool automatically detects the control panel and uses the correct base path:
| Control Panel | Home Base Path | Example Restore Directory |
|---------------|----------------|--------------------------|
| cPanel | `/home` | `/home/temp/restore20251210/mysql/` |
| InterWorx | `/chroot/home` | `/chroot/home/temp/restore20251210/mysql/` |
| Plesk | `/var/www/vhosts` | `/var/www/vhosts/temp/restore20251210/mysql/` |
| Standalone | `/home` | `/home/temp/restore20251210/mysql/` |
**Note**: InterWorx uses `/chroot/home` directly (not the `/home` symlink) as the system doesn't display `/home` properly.
### Usage Example
```bash
# Run the restore tool
./mysql-restore-to-sql.sh
# Follow the guided wizard:
# 1. Copy backup files to suggested directory
# 2. Select database to restore (e.g., 'amea_wp')
# 3. Configure MySQL port (default: 13306)
# 4. Choose Force Recovery level
# - Tool will recommend Level 1 if missing files are from other databases
# 5. Wait for dump to complete
# Result: Clean SQL file saved to restore directory parent
# Example: /home/temp/restore20251210/amea_wp_restored_20251210_150530.sql
```
### Error Detection & Recovery
The tool automatically detects common issues:
#### Missing Tablespace Files
- **Detection**: Parses error log for "was not found at ./database/table.ibd"
- **Analysis**: Compares missing files against selected database
- **Recommendation**: Suggests Force Recovery Level 1 if safe
#### Corrupt Tables
- **Detection**: Identifies InnoDB corruption errors
- **Analysis**: Determines severity and affected tables
- **Recommendation**: Suggests appropriate Force Recovery level with risk warnings
#### Insufficient Disk Space
- **Detection**: Checks available space vs. required space (2x backup size)
- **Prevention**: Stops before attempting restore
- **Solution**: Suggests cleanup or alternative location
### Technical Details
#### Second MySQL Instance
The tool runs a completely separate MySQL instance:
```
Port: 13306 (configurable, avoids conflict with production)
Socket: /path/to/restore/mysql.sock
Data Directory: /path/to/restore/mysql/
PID File: /path/to/restore/mysql.pid
Error Log: /path/to/restore/mysql_error.log
```
This isolation ensures:
- No risk to production MySQL
- Can run even if production MySQL is down
- Clean environment for database recovery
#### File Requirements
Minimum required files from backup:
```
ibdata1 # InnoDB system tablespace (REQUIRED)
ib_logfile0 # InnoDB redo log file (REQUIRED)
ib_logfile1 # InnoDB redo log file (REQUIRED)
database_name/ # Folder containing database tables (REQUIRED)
*.ibd # InnoDB tablespace files for each table
*.frm # Table definition files (MySQL 5.x)
```
#### mysqldump Options
The tool uses optimized mysqldump settings:
```bash
--single-transaction # Consistent snapshot without locking
--routines # Include stored procedures/functions
--triggers # Include triggers
--events # Include events
--hex-blob # Binary data in hex format
```
### Documentation
For detailed technical documentation, see:
- **REFDB_FORMAT.txt** - Complete reference including:
- Control panel path mappings
- Force Recovery level details
- Smart detection logic
- Error handling procedures
- Safety features documentation
---
## Integration with Launcher
Both subsystems are accessible via the main toolkit launcher:
```bash
bash /root/server-toolkit/launcher.sh
# Select: Backup & Recovery
# Choose from:
# - Acronis Backup Manager (submenu)
# - MySQL/MariaDB Database Restore to SQL
```
---
## Requirements
### Acronis Tools
- Acronis Cyber Protect agent installation file or download access
- Cloud credentials for registration
- Root access
### MySQL Restore Tool
- Root access
- MySQL/MariaDB client tools (`mysql`, `mysqld`, `mysqldump`)
- Backup files (ibdata1, ib_logfile*, database folders)
- Sufficient disk space (2x backup size recommended)
---
## Recent Updates
### December 2025
- ✅ Added MySQL/MariaDB database restore tool
- ✅ Multi-control panel path support (cPanel, InterWorx, Plesk, Standalone)
- ✅ Intelligent Force Recovery detection and recommendations
- ✅ Smart detection for selective database restore scenarios
- ✅ Enhanced error detection for missing tablespace files
- ✅ SQL output location fixes (parent directory of restore dir)
- ✅ Safety enhancements (disk space, directory protection, recovery warnings)
- ✅ InterWorx path fix (/chroot/home instead of /home symlink)
### November 2025
- ✅ Complete Acronis Cyber Protect integration
- ✅ 16 management scripts covering full lifecycle
- ✅ Performance optimizations for backup triggering
- ✅ Comprehensive troubleshooting and diagnostics
---
## Support
For issues or feature requests, please refer to the main toolkit repository.