Bash Backup Script: Automated, Rotated, & Robust!

by Hugo van Dijk 50 views

Hey guys! Let’s dive into creating a robust automated backup script using Bash. This is a hard issue, but totally achievable if we break it down. We're going to build a script that not only backs up your important files and databases but also manages those backups with rotation, logging, and error handling. Think of it as your digital safety net! Let's get started!

Understanding the Task: Fork, Commit, Merge - Hard Issue (Bash)

So, what's the deal with this Fork, Commit, Merge challenge? It's all about simulating a real-world development workflow. We start by forking the repository, making our changes (in this case, writing a killer backup script), committing those changes, and then merging them back into the main branch. It’s a fantastic way to learn Git and collaborate on projects.

This particular issue focuses on creating an automated backup script in Bash. We'll be dealing with backing up files, databases, compressing archives, handling errors, and more. It’s a meaty task, but super rewarding!

Key Requirements for the Backup Script

Before we start coding, let’s break down the requirements. This will help us stay organized and ensure we cover all the bases.

  1. File Backup: Our script needs to back up multiple directories. Think /etc, /home, and /var/www. These are the places where important configuration files and website data often live.
  2. Database Backup: We need to support both PostgreSQL and MySQL databases. This means figuring out how to dump those databases and include them in our backups.
  3. Compression: To save space, we'll compress our backups using tar and gzip. This will create nice, compact archive files.
  4. Rotation: We don’t want our backups to eat up all our disk space. So, we’ll implement a rotation system that automatically removes old backups. We're aiming to keep the last 7 backups.
  5. Logging: A good backup script keeps a detailed log of everything it does. We’ll implement comprehensive logging to track successes, failures, and everything in between.
  6. Disk Space Check: Before we even start a backup, we need to make sure there’s enough space. We'll add a check to verify sufficient disk space before proceeding.
  7. Error Handling: Things can go wrong, and our script needs to be ready. We'll implement robust error handling with notifications, so we know if a backup fails.
  8. Manifest: A manifest file will contain details about each backup, such as the date, time, and files included. This is super helpful for restoring data later.

Diving Deeper into the Script's Features

Let's explore some of the cool features we’ll be including in our script. These aren’t just nice-to-haves; they're essential for a production-ready backup solution.

  • Error Handling with set -euo pipefail: This command is a lifesaver in Bash scripting. It makes our script exit immediately if any command fails, which helps us catch errors early. set -e makes the script exit if a command exits with a non-zero status. set -u treats unset variables as an error. set -o pipefail ensures that if any command in a pipeline fails, the whole pipeline fails.
  • Comprehensive Logging: We’ll log all operations to both a file and the console. This gives us a detailed record of what happened during each backup.
  • Email Notifications: If a backup fails, we want to know ASAP. We’ll set up email notifications to alert us of any issues. This is crucial for ensuring data integrity.
  • Disk Space Management: Checking disk space before a backup prevents our script from crashing mid-operation due to lack of space. We’ll implement a check to ensure we have enough room.
  • Automatic Backup Rotation: Keeping only the last 7 backups helps us manage disk space efficiently. We’ll automate the cleanup of old backups.
  • Detailed Backup Manifest: The manifest file provides a snapshot of each backup. This includes details like the date, time, and a list of included files. It's invaluable for restoration purposes.
  • Colored Console Output: Let’s be honest, a little color makes everything better! We’ll add colored console output for a better user experience. It makes it easier to see what's going on at a glance. Using ANSI escape codes, we can add color to the output messages, making them more readable and visually appealing. For example, we can use different colors for success, warning, and error messages.

Files You'll Be Working With

You’ll primarily be working with two files:

  • backup_script.sh: This is the main script where all the magic happens. It’s where we’ll implement all the backup logic.
  • README.md: This file provides documentation for the script. It should explain how to use the script, its features, and any dependencies.

Expected Outcome: A Production-Ready Backup Script

Our goal is to create a backup script that’s ready to be used in a production environment. This means it should be reliable, efficient, and easy to use. We want a script that can be scheduled with cron for automated backups, giving us peace of mind.

Getting Started: Navigating to the Task Directory

To get started, navigate to the ./task/bash/hard directory in your repository. This is where you’ll find the files you need to work with. You can use the cd command in your terminal:

cd ./task/bash/hard

Once you're in the directory, you can start working on the backup_script.sh file.

Let's Start Building the Backup Script!

Okay, let's get our hands dirty and start building the backup_script.sh. First things first, let's set up the basic structure and error handling.

Setting Up the Script Header

Every good Bash script starts with a shebang line. This tells the system which interpreter to use to execute the script. Let’s add that, along with our strict error handling commands:

#!/bin/bash

# Enable strict mode for error handling
set -euo pipefail

# Script name: backup_script.sh
# Description: Automated backup script with rotation and error handling
# Author: Your Name
# Date: Current Date

# Constants
BACKUP_DIR="/opt/backups"
LOG_FILE="${BACKUP_DIR}/backup.log"
MANIFEST_FILE="${BACKUP_DIR}/backup_manifest.txt"
DAYS_TO_KEEP=7
EMAIL_RECIPIENT="[email protected]"

# Colors for console output
RED="\033[0;31m"
GREEN="\033[0;32m"
YELLOW="\033[0;33m"
NC="\033[0m" # No Color

Here's what we've done:

  • #!/bin/bash: This is the shebang line.
  • set -euo pipefail: This enables strict error handling.
  • We've added some comments to describe the script, its purpose, and the author. Always a good practice!
  • We’ve defined some constants for our backup directory, log file, manifest file, the number of days to keep backups, and the email recipient for notifications. This makes our script more configurable and easier to maintain.
  • We've set up color codes for console output. This will make our script output more readable.

Implementing Logging Functions

Next, let's create some logging functions. These will help us keep track of what’s happening during the backup process.

# Function to log messages to console and file
log() {
 local message="$(date +'%Y-%m-%d %H:%M:%S') - $@"
 echo "$message" | tee -a "${LOG_FILE}"
}

# Function to log error messages
log_error() {
 log "${RED}ERROR: $@${NC}"
}

# Function to log warning messages
log_warning() {
 log "${YELLOW}WARNING: $@${NC}"
}

# Function to log success messages
log_success() {
 log "${GREEN}SUCCESS: $@${NC}"
}

What’s happening here?

  • We’ve defined a log function that takes a message as input, prepends the current date and time, and then prints the message to both the console and the log file using tee -a. The -a option appends to the file.
  • We’ve also defined log_error, log_warning, and log_success functions that use the log function and add color codes to the messages. This makes it easier to identify different types of messages in the console output.

Checking for Required Tools

Before we get too far, let's make sure we have the tools we need. We'll check for tar, gzip, mysqldump, and pg_dump.

# Check if required tools are installed
check_tools() {
 log "Checking for required tools..."
 local tools=("tar" "gzip" "mysqldump" "pg_dump")
 for tool in "${tools[@]}"; do
 if ! command -v "$tool" &> /dev/null; then
 log_error "$tool is not installed. Please install it."
 exit 1
 fi
 done
 log_success "All required tools are installed."
}

Here's what this function does:

  • It logs a message saying we're checking for required tools.
  • It defines an array of tools we need (tar, gzip, mysqldump, pg_dump).
  • It loops through the tools and uses command -v to check if each tool is installed. If a tool is not found, it logs an error and exits the script.
  • If all tools are found, it logs a success message.

Checking Disk Space

We need to make sure we have enough disk space before we start the backup. Let's add a function to check that.

# Check if there is enough disk space
check_disk_space() {
 log "Checking available disk space..."
 local threshold=20 # Minimum free space in GB
 local free_space=$(df -BG / | tail -n 1 | awk '{print $4}' | tr -d 'G')
 if (( free_space < threshold )); then
 log_error "Insufficient disk space. Available: ${free_space}G, Required: ${threshold}G"
 exit 1
 fi
 log_success "Sufficient disk space available."
}

Let's break this down:

  • We log a message saying we're checking disk space.
  • We define a threshold for the minimum free space in GB (20GB in this case).
  • We use df -BG / to get the free space on the root partition, tail -n 1 to get the last line (which contains the disk space information), awk '{print $4}' to extract the available space, and tr -d 'G' to remove the 'G' from the output.
  • We then compare the free space to the threshold. If there's not enough space, we log an error and exit the script.
  • If there's enough space, we log a success message.

Next Steps: Implementing the Backup Logic

We've got a solid foundation for our script. We’ve set up error handling, logging, tool checks, and disk space checks. Now, it's time to dive into the heart of the script: the backup logic itself.

We’ll need to implement functions for backing up files and databases, compressing the backups, rotating old backups, and creating the manifest file. It’s a lot of work, but we're making great progress! Keep coding, keep testing, and you'll have a fantastic backup script in no time! Remember to commit your changes regularly and don't hesitate to ask questions if you get stuck. You've got this!

Backup Script: File Backup Implementation

Continuing from where we left off, let's implement the file backup functionality. We need to backup multiple directories, compress them, and store them in our backup directory.

# Function to backup files
backup_files() {
 log "Backing up files..."
 local directories=('/etc' '/home' '/var/www')
 local timestamp=$(date +'%Y%m%d%H%M%S')
 local backup_file="${BACKUP_DIR}/files_${timestamp}.tar.gz"

 log "Creating file backup: ${backup_file}"
 tar -czvf "${backup_file}" "${directories[@]}" \
 if [ $? -ne 0 ]; then
 log_error "File backup failed."
 return 1
 fi

 log_success "File backup created: ${backup_file}"
 return 0
}

Here's the breakdown:

  • We start by logging that we're backing up files.
  • We define an array directories containing the directories we want to backup (/etc, /home, /var/www).
  • We create a timestamp to include in the backup filename. This helps us keep track of different backup versions.
  • We construct the backup_file path using the backup directory and the timestamp.
  • We log a message indicating that we're creating the file backup.
  • We use tar -czvf to create a compressed archive of the directories. The -c option creates an archive, -z compresses it with gzip, -v makes it verbose (shows the files being added), and -f specifies the output file.
  • We check the exit status of the tar command using $?. If it's not 0, it means an error occurred, so we log an error message and return 1.
  • If the backup is successful, we log a success message and return 0.

Backup Script: Database Backup Implementation

Next, let's add support for backing up PostgreSQL and MySQL databases. We'll create separate functions for each database type.

# Function to backup PostgreSQL databases
backup_postgresql() {
 log "Backing up PostgreSQL databases..."
 local timestamp=$(date +'%Y%m%d%H%M%S')
 local backup_file="${BACKUP_DIR}/postgresql_${timestamp}.sql.gz"

 log "Creating PostgreSQL backup: ${backup_file}"
 pg_dumpall | gzip > "${backup_file}" \
 if [ $? -ne 0 ]; then
 log_error "PostgreSQL backup failed."
 return 1
 fi

 log_success "PostgreSQL backup created: ${backup_file}"
 return 0
}

# Function to backup MySQL databases
backup_mysql() {
 log "Backing up MySQL databases..."
 local timestamp=$(date +'%Y%m%d%H%M%S')
 local backup_file="${BACKUP_DIR}/mysql_${timestamp}.sql.gz"

 log "Creating MySQL backup: ${backup_file}"
 mysqldump --all-databases | gzip > "${backup_file}" \
 if [ $? -ne 0 ]; then
 log_error "MySQL backup failed."
 return 1
 fi

 log_success "MySQL backup created: ${backup_file}"
 return 0
}

Here's what these functions do:

  • For PostgreSQL, we use pg_dumpall to dump all databases, pipe the output to gzip for compression, and save it to a file with a timestamp.
  • For MySQL, we use mysqldump --all-databases to dump all databases, pipe the output to gzip, and save it to a file with a timestamp.
  • In both functions, we log messages indicating the backup status and check the exit status of the commands. If an error occurs, we log an error message and return 1.

Backup Script: Implementing Backup Rotation

To prevent our backups from consuming too much disk space, we'll implement backup rotation. This involves removing old backups, keeping only the most recent ones.

# Function to rotate backups
rotate_backups() {
 log "Rotating backups..."
 find "${BACKUP_DIR}" -type f -mtime +