diff --git a/README.md b/README.md index 55ea1ff..c8501d5 100644 --- a/README.md +++ b/README.md @@ -25,8 +25,8 @@ server-toolkit/ │ │ ├── ip-reputation-manager.sh # Centralized IP reputation tracking │ │ └── tail-*.sh # Various log monitoring scripts │ │ -│ ├── backup/ # 💾 Backup & Recovery (Acronis Cyber Protect) -│ │ ├── acronis-backup-manager.sh # Main backup management menu +│ ├── backup/ # 💾 Backup & Recovery +│ │ ├── acronis-backup-manager.sh # Main backup management menu (Acronis Cyber Protect) │ │ ├── acronis-install.sh # Install Acronis agent │ │ ├── acronis-update.sh # Update Acronis agent │ │ ├── acronis-uninstall.sh # Uninstall Acronis agent @@ -40,7 +40,8 @@ server-toolkit/ │ │ ├── acronis-schedule-viewer.sh # View backup schedules │ │ ├── acronis-restore.sh # Restore from backup │ │ ├── acronis-logs.sh # View Acronis logs -│ │ └── acronis-troubleshoot.sh # Troubleshoot common issues +│ │ ├── acronis-troubleshoot.sh # Troubleshoot common issues +│ │ └── mysql-restore-to-sql.sh # Advanced MySQL/MariaDB database restore & dump tool │ │ │ ├── website/ # 🌐 Website Diagnostics & Troubleshooting │ │ ├── website-error-analyzer.sh # Comprehensive website error analysis @@ -105,7 +106,8 @@ source /root/linux-server-management-toolkit/run.sh - **IP Reputation Tracking**: Centralized cross-module IP intelligence - **Multi-Source Monitoring**: SSH, Web, Firewall, cPHulk integration -### 💾 Backup & Recovery (Acronis Cyber Protect) +### 💾 Backup & Recovery +#### Acronis Cyber Protect Integration - **Complete Agent Management**: Install, update, uninstall, register - **Comprehensive Status Monitoring**: Agent health, registration, cloud connectivity - **Manual Backup Triggering**: CLI-managed plans with performance optimizations @@ -114,6 +116,15 @@ source /root/linux-server-management-toolkit/run.sh - **Restore Operations**: Full restore capabilities from backups - **Troubleshooting Tools**: Log viewing and automated diagnostics +#### MySQL/MariaDB Database Restore Tool (NEW!) +- **Advanced Database Recovery**: Restore individual databases from file-based backups (Acronis, raw file backups) +- **Multi-Control Panel Support**: Automatic path detection for cPanel, InterWorx, Plesk, standalone +- **Intelligent Force Recovery**: Smart detection for missing tablespace files from other databases +- **Safe Selective Restore**: Restore single database from full backup without data loss +- **Guided Recovery Process**: Step-by-step wizard with automatic error detection +- **Safety Features**: Disk space validation, critical directory protection, recovery level warnings +- **Clean SQL Export**: Dumps restored database to clean SQL file for import into production + ### 🌐 Website Diagnostics - **Error Analysis**: Comprehensive website error detection and troubleshooting - **500 Error Tracking**: Detailed analysis of application errors @@ -176,6 +187,20 @@ bash launcher.sh # Select: Manage Protection Plans ``` +### MySQL Database Restore & Recovery + +```bash +bash launcher.sh +# Select: Backup & Recovery +# Select: MySQL/MariaDB Database Restore to SQL +# Follow guided wizard: +# 1. Provide backup files (ibdata1, ib_logfile*, database folders) +# 2. Select database to restore +# 3. Configure MySQL settings +# 4. Choose Force Recovery level (with intelligent recommendations) +# 5. Restore and dump to clean SQL file +``` + ### Website Error Analysis ```bash @@ -240,6 +265,10 @@ nano /root/server-toolkit/config/settings.conf - ✅ Comprehensive status monitoring (health, registration, connectivity) - ✅ Manual backup triggering with performance optimizations - ✅ Protection plan management and scheduling +- ✅ MySQL/MariaDB database restore tool with intelligent recovery (NEW!) +- ✅ Multi-control panel support for database restoration paths +- ✅ Smart detection for Force Recovery Level selection +- ✅ Safe selective database restore from full backups ### Website Diagnostics - ✅ Comprehensive website error analyzer (multi-panel) diff --git a/modules/backup/README.md b/modules/backup/README.md new file mode 100644 index 0000000..29313dc --- /dev/null +++ b/modules/backup/README.md @@ -0,0 +1,377 @@ +# Backup & Recovery Module + +Comprehensive backup and database recovery tools for server management. + +## Overview + +This module provides two major subsystems: + +1. **Acronis Cyber Protect Integration** - Complete backup agent management +2. **MySQL/MariaDB Database Restore Tool** - Advanced database recovery from file-based backups + +--- + +## Acronis Cyber Protect Integration + +Complete command-line management for Acronis Cyber Protect backup agent on Linux servers. + +### Features + +- Full agent lifecycle management (install, update, uninstall) +- Cloud registration and configuration +- Manual backup triggering with performance optimizations +- Protection plan management +- Backup status monitoring and scheduling +- Comprehensive troubleshooting and log viewing + +### Scripts + +#### Agent Management +- **acronis-install.sh** - Install Acronis agent from local file or download +- **acronis-update.sh** - Update agent to latest version +- **acronis-uninstall.sh** - Clean uninstallation of agent +- **acronis-register.sh** - Register agent with Acronis Cloud +- **acronis-configure.sh** - Configure agent settings + +#### Monitoring & Status +- **acronis-agent-status.sh** - Comprehensive agent health check + - Registration status + - Cloud connectivity + - Service status + - Version information +- **acronis-backup-status.sh** - Check backup job status +- **acronis-list-backups.sh** - List all available backups +- **acronis-schedule-viewer.sh** - View backup schedules + +#### Backup Operations +- **acronis-trigger-backup.sh** - Manually trigger backups + - Full backup support + - Incremental backup support + - Differential backup support + - Performance optimizations (nice, ionice) +- **acronis-plan-manager.sh** - Manage protection plans + - View plans + - Enable/disable plans + - Delete plans +- **acronis-restore.sh** - Restore from backups + +#### Troubleshooting +- **acronis-logs.sh** - View Acronis logs + - Real-time log monitoring + - Historical log viewing + - Filtered log search +- **acronis-troubleshoot.sh** - Automated diagnostics + - Common issue detection + - Fix recommendations + - Health checks + +#### Menu System +- **acronis-backup-manager.sh** - Interactive menu for all Acronis operations + +### Usage Example + +```bash +# Check agent status +./acronis-agent-status.sh + +# Trigger manual backup +./acronis-trigger-backup.sh + +# View backup schedules +./acronis-schedule-viewer.sh + +# Manage protection plans +./acronis-plan-manager.sh +``` + +--- + +## MySQL/MariaDB Database Restore Tool + +**Script**: `mysql-restore-to-sql.sh` + +Advanced database recovery tool for restoring individual databases from file-based backups (Acronis, raw file backups, etc.) and exporting them to clean SQL files. + +### Key Features + +#### Multi-Control Panel Support +- **cPanel**: Uses `/home` for restore directory +- **InterWorx**: Uses `/chroot/home` (actual path, not symlink) +- **Plesk**: Uses `/var/www/vhosts` +- **Standalone**: Uses `/home` as fallback + +Automatic detection via `lib/system-detect.sh` ensures correct paths for all control panels. + +#### Intelligent Force Recovery +- **Smart Detection**: Automatically identifies when missing tablespace files are from OTHER databases (not the one you're restoring) +- **Safe Recommendations**: Suggests Force Recovery Level 1 when appropriate for selective database restore +- **No Data Loss**: Force Recovery Level 1 ignores missing databases you don't have while preserving all data from databases you DO have + +#### Safety Features +- **Disk Space Validation**: Ensures 2x required space before starting +- **Critical Directory Protection**: Prevents using `/var/lib/mysql` as restore directory +- **Force Recovery Warnings**: Risk acknowledgment for levels 5-6 +- **Automatic Cleanup**: Trap handler for Ctrl+C/interruption +- **Backup-Free Operation**: Works in temporary directory, never touches production MySQL + +#### Guided Wizard Process + +The tool provides a step-by-step guided process: + +**Step 1: Gather Backup Files** +- Collect required files: `ibdata1`, `ib_logfile0`, `ib_logfile1`, database folders +- Copy files to suggested restore directory (e.g., `/home/temp/restore20251210/mysql/`) + +**Step 2: Select Database** +- Lists all databases found in backup +- Select which database to restore + +**Step 3: Configure MySQL Settings** +- Port selection (default: 13306 to avoid conflicts) +- Timeout configuration +- Option to verify file integrity + +**Step 4: Configure Recovery Options** +- Choose InnoDB Force Recovery level (0-6) +- Shows intelligent recommendations based on detected issues +- Explains risks and benefits of each level + +**Step 5: Restore & Dump** +- Starts temporary MySQL instance in restore directory +- Monitors startup for errors +- Provides intelligent recovery guidance if issues detected +- Dumps selected database to clean SQL file +- Automatic cleanup of temporary MySQL instance + +### SQL Output Location + +SQL files are saved to the **parent directory** of the restore directory: + +``` +Restore Directory: /home/temp/restore20251210/mysql/ +SQL Output Location: /home/temp/restore20251210/database_restored_20251210_150530.sql +``` + +This prevents cluttering control panel system directories and keeps output organized with restore files. + +### Force Recovery Levels + +The tool supports all InnoDB Force Recovery levels with clear explanations: + +- **Level 0**: Normal operation (no recovery) +- **Level 1**: Ignore corrupt pages/missing tablespaces (safe for selective restore) +- **Level 2**: Stop master thread operations +- **Level 3**: Skip transaction rollback +- **Level 4**: Skip insert buffer merge +- **Level 5**: Ignore undo logs (data loss risk) +- **Level 6**: Skip redo log recovery (data loss risk) + +### Smart Detection for Selective Restore + +When you restore a single database from a full backup: + +**Problem**: The `ibdata1` file contains metadata for ALL databases from the original backup. If you only restored one database folder, MySQL will report missing tablespace files for all the other databases. + +**Solution**: The tool detects this scenario and recommends Force Recovery Level 1: + +``` +SMART DETECTION: Missing files are from OTHER databases, not 'yourdatabase' + +Your selected database 'yourdatabase' appears to have all files! + +RECOMMENDED ACTION: Use Force Recovery Level 1 + +The ibdata1 file contains references to databases you didn't restore. +Force Recovery Level 1 will: + ✓ Ignore missing databases (safe - you don't have them anyway) + ✓ Start MySQL successfully + ✓ Allow you to dump 'yourdatabase' with NO data loss + +This is the CORRECT approach for selective database restoration. +``` + +### Use Cases + +#### Restore Single Database from Full Backup +1. You have an Acronis backup containing all databases +2. You only want to restore one specific database +3. Tool detects missing files from other databases +4. Recommends Force Recovery Level 1 +5. Successfully dumps your database without data loss + +#### Recover from Corrupt Backup +1. Backup has some corrupt tables +2. Tool attempts normal restore +3. Detects corruption errors +4. Recommends appropriate Force Recovery level +5. Extracts maximum recoverable data + +#### Import Older Database Version +1. Restore older version of database from backup +2. Dump to SQL file +3. Drop tables in production database (keeps permissions) +4. Import SQL dump + +### Safety Guarantees + +- **Never touches production MySQL** - Uses isolated temporary instance +- **Disk space validation** - Ensures sufficient space before starting +- **Critical directory protection** - Prevents dangerous restore locations +- **Smart recommendations** - Only suggests recovery when safe +- **Clean SQL output** - Produces importable SQL file, not raw data files + +### Control Panel Path Support + +The tool automatically detects the control panel and uses the correct base path: + +| Control Panel | Home Base Path | Example Restore Directory | +|---------------|----------------|--------------------------| +| cPanel | `/home` | `/home/temp/restore20251210/mysql/` | +| InterWorx | `/chroot/home` | `/chroot/home/temp/restore20251210/mysql/` | +| Plesk | `/var/www/vhosts` | `/var/www/vhosts/temp/restore20251210/mysql/` | +| Standalone | `/home` | `/home/temp/restore20251210/mysql/` | + +**Note**: InterWorx uses `/chroot/home` directly (not the `/home` symlink) as the system doesn't display `/home` properly. + +### Usage Example + +```bash +# Run the restore tool +./mysql-restore-to-sql.sh + +# Follow the guided wizard: +# 1. Copy backup files to suggested directory +# 2. Select database to restore (e.g., 'amea_wp') +# 3. Configure MySQL port (default: 13306) +# 4. Choose Force Recovery level +# - Tool will recommend Level 1 if missing files are from other databases +# 5. Wait for dump to complete + +# Result: Clean SQL file saved to restore directory parent +# Example: /home/temp/restore20251210/amea_wp_restored_20251210_150530.sql +``` + +### Error Detection & Recovery + +The tool automatically detects common issues: + +#### Missing Tablespace Files +- **Detection**: Parses error log for "was not found at ./database/table.ibd" +- **Analysis**: Compares missing files against selected database +- **Recommendation**: Suggests Force Recovery Level 1 if safe + +#### Corrupt Tables +- **Detection**: Identifies InnoDB corruption errors +- **Analysis**: Determines severity and affected tables +- **Recommendation**: Suggests appropriate Force Recovery level with risk warnings + +#### Insufficient Disk Space +- **Detection**: Checks available space vs. required space (2x backup size) +- **Prevention**: Stops before attempting restore +- **Solution**: Suggests cleanup or alternative location + +### Technical Details + +#### Second MySQL Instance +The tool runs a completely separate MySQL instance: + +``` +Port: 13306 (configurable, avoids conflict with production) +Socket: /path/to/restore/mysql.sock +Data Directory: /path/to/restore/mysql/ +PID File: /path/to/restore/mysql.pid +Error Log: /path/to/restore/mysql_error.log +``` + +This isolation ensures: +- No risk to production MySQL +- Can run even if production MySQL is down +- Clean environment for database recovery + +#### File Requirements + +Minimum required files from backup: +``` +ibdata1 # InnoDB system tablespace (REQUIRED) +ib_logfile0 # InnoDB redo log file (REQUIRED) +ib_logfile1 # InnoDB redo log file (REQUIRED) +database_name/ # Folder containing database tables (REQUIRED) + *.ibd # InnoDB tablespace files for each table + *.frm # Table definition files (MySQL 5.x) +``` + +#### mysqldump Options + +The tool uses optimized mysqldump settings: +```bash +--single-transaction # Consistent snapshot without locking +--routines # Include stored procedures/functions +--triggers # Include triggers +--events # Include events +--hex-blob # Binary data in hex format +``` + +### Documentation + +For detailed technical documentation, see: +- **REFDB_FORMAT.txt** - Complete reference including: + - Control panel path mappings + - Force Recovery level details + - Smart detection logic + - Error handling procedures + - Safety features documentation + +--- + +## Integration with Launcher + +Both subsystems are accessible via the main toolkit launcher: + +```bash +bash /root/server-toolkit/launcher.sh +# Select: Backup & Recovery +# Choose from: +# - Acronis Backup Manager (submenu) +# - MySQL/MariaDB Database Restore to SQL +``` + +--- + +## Requirements + +### Acronis Tools +- Acronis Cyber Protect agent installation file or download access +- Cloud credentials for registration +- Root access + +### MySQL Restore Tool +- Root access +- MySQL/MariaDB client tools (`mysql`, `mysqld`, `mysqldump`) +- Backup files (ibdata1, ib_logfile*, database folders) +- Sufficient disk space (2x backup size recommended) + +--- + +## Recent Updates + +### December 2025 +- ✅ Added MySQL/MariaDB database restore tool +- ✅ Multi-control panel path support (cPanel, InterWorx, Plesk, Standalone) +- ✅ Intelligent Force Recovery detection and recommendations +- ✅ Smart detection for selective database restore scenarios +- ✅ Enhanced error detection for missing tablespace files +- ✅ SQL output location fixes (parent directory of restore dir) +- ✅ Safety enhancements (disk space, directory protection, recovery warnings) +- ✅ InterWorx path fix (/chroot/home instead of /home symlink) + +### November 2025 +- ✅ Complete Acronis Cyber Protect integration +- ✅ 16 management scripts covering full lifecycle +- ✅ Performance optimizations for backup triggering +- ✅ Comprehensive troubleshooting and diagnostics + +--- + +## Support + +For issues or feature requests, please refer to the main toolkit repository.