build-web-application-with-golang/en/12.4.md

12.4 Backup and recovery

In this section, we'll discuss another aspect of application management: data backup and recovery on production servers. We often encounter situations where production servers don't behave as as we expect them to. Server network outages, hard drive malfunctions, operating system crashes and other similar events can cause databases to become unavailable. The need to recover from these types of events has led to the emergence of many cold standby/hot standby tools that can help to facilitate disaster recovery remotely. In this section, we'll explain how to backup deployed applications in addition to backing up and restoring any MySQL and Redis databases you might be using.

Application Backup

In most cluster environments, web applications do not need to be backed up since they are actually copies of code from our local development environment, or from a version control system. In many cases however, we need to backup data which has been supplied by the users of our site. For instance, when sites require users to upload files, we need to be able to backup any files that have been uploaded by users to our website. The current approach for providing this kind of redundancy is to utilize so-called cloud storage, where user files and other related resources are persisted into a highly available network of servers. If our system crashes, as long as user data has been persisted onto the cloud, we can at least be sure that no data will be lost.

But what about the cases where we did not backup our data to a cloud service, or where cloud storage was not an option? How do we backup data from our web applications then? Here, we describe a tool called rysnc, which can be commonly found on unix-like systems. Rsync is a tool which can be used to synchronize files residing on different systems, and a perfect use-case for this functionality is to keep our website backed up.

Note: Cwrsync is an implementation of rsync for the Windows environment

Rsync installation

You can find the latest version of rsync from its official website. Of course, because rsync is very useful software, many Linux distributions will already have it installed by default.

Package Installation:

# sudo apt-get install rsync ; Note: debian, ubuntu and other online installation methods ;
# yum install rsync ; Note: Fedora, Redhat, CentOS and other online installation methods ;
# rpm -ivh rsync ; Note: Fedora, Redhat, CentOS and other rpm package installation methods ;

For the other Linux distributions, please use the appropriate package management methods to install it. Alternatively, you can build it yourself from the source:

tar xvf rsync-xxx.tar.gz
cd rsync-xxx
./configure - prefix =/usr; make; make install 

Note: If want to compile and install the rsync from its source, you have to install gcc compiler tools such as job.

Note: Before using source packages compiled and installed, you have to install gcc compiler tools such as job

Rsync Configuration

Rsync can be configured from three main configuration files: rsyncd.conf which is the main configuration file, rsyncd.secrets which holds passwords, and rsyncd.motd which contains server information.

You can refer to the official documentation on rsync's website for more detailed explanations, but here we will simply introduce the basics of setting up rsync:.

• Starting an rsync daemon server-side:

  # /usr/bin/rsync --daemon --config=/etc/rsyncd.conf

• the --daemon parameter is for running rsync in server mode. Make this the default boot-time setting by joining it to the rc.local file:

  echo 'rsync --daemon' >> /etc/rc.d/rc.local

Setup an rsync username and password, making sure that it's owned only by root, so that local unauthorized users or exploits do not have access to it. If these permissions are not set correctly, rsync may not boot:

echo 'Your Username: Your Password' > /etc/rsyncd.secrets
chmod 600 /etc/rsyncd.secrets

• Client synchronization:

Clients can synchronize server files with the following command:

rsync -avzP --delete --password-file=rsyncd.secrets username@192.168.145.5::www /var/rsync/backup

Let's break this down into a few key points:

1. -avzP are some common options. Use rsync --help to review what these do.
2. --delete deletes extraneous files on the receiving side. For example, if files are deleted on the sending side, the next time the two machines are synchronized, the receiving sides will automatically delete the corresponding files.
3. --password-file specifies a password file for accessing an rsync daemon. On the client side, this is typically the client/etc/rsyncd.secrets file, and on the server side, it's /etc/rsyncd.secrets. When using something like Cron to automate rsync, you won't need to manually enter a password.
4. username specifies the username to be used in conjunction with the server-side /etc/rsyncd.secrets password
5. 192.168.145.5 is the IP address of the server
6. ::www (note the double colons), specifies contacting an rsync daemon directly via TCP for synchronizing the www module according to the server-side configurations located in /etc/rsyncd.conf. When only a single colon is used, the rsync daemon is not contacted directly; instead, a remote-shell program such as ssh is used as the transport .

In order to periodically synchronize files, you can set up a crontab file that will run rsync commands as often as needed. Of course, users can vary the frequency of synchronization according to how critical it is to keep certain directories or files up to date.

MySQL backup

MySQL database application is still the mainstream, the current MySQL backup in two ways: hot backup and cold backup, hot backup is currently mainly used master/slave mode (master/slave) mode is mainly used for database synchronization separate read and write, but also can be used for hot backup data ), on how to configure this information, we can find a lot. Cold backup data, then that is a certain delay, but you can guarantee that the time period before data integrity, such as may sometimes be caused by misuse of our loss of data, then the master/slave model is able to retrieve lost data, but through cold backup can partially restore the data.

Cold backup shell script is generally used to achieve regular backup of the database, and then rsync synchronization through the above described non-local one server room.

The following is a scheduled backup MySQL backup script, we use the mysqldump program, this command can be exported to a database file.

#!/bin/bash
# The following configuration information, modify their own
mysql_user="USER" #MySQL backup user
mysql_password="PASSWORD" # MySQL backup user's password
mysql_host="localhost"
mysql_port="3306"
mysql_charset="utf8" # MySQL coding
backup_db_arr=("db1" "db2") # To back up the database name, separated by spaces separated by a plurality of such("db1" "db2" "db3")
backup_location=/var/www/mysql # backup data storage location, please do not end with a "/", this can keep the default, the program will automatically create a folder
expire_backup_delete="ON" # delete outdated backups is turned OFF to ON ON to OFF
expire_days=3 # default expiration time for the three days the number of days, this is only valid when the expire_backup_delete open

# We do not need to modify the following start
backup_time=`date +%Y%m%d%H%M` # define detailed time backup
backup_Ymd=`date +%Y-%m-%d` # define the backup directory date time
backup_3ago=`date-d '3 days ago '+%Y-%m-%d` # 3 days before the date
backup_dir=$backup_location/$backup_Ymd # full path to the backup folder
welcome_msg="Welcome to use MySQL backup tools!" # greeting

# Determine whether to start MYSQL, mysql does not start the backup exit
mysql_ps=`ps-ef | grep mysql | wc-l`
mysql_listen=`netstat-an | grep LISTEN | grep $mysql_port | wc-l`
if [[$mysql_ps==0]-o [$mysql_listen==0]]; then
  echo "ERROR: MySQL is not running! backup stop!"
  exit
else
  echo $welcome_msg
fi

# Connect to mysql database, can not connect to the backup exit
mysql-h $mysql_host-P $mysql_port-u $mysql_user-p $mysql_password << end
use mysql;
select host, user from user where user='root' and host='localhost';
exit
end

flag=`echo $?`
if [$flag!="0"]; then
  echo "ERROR: Can't connect mysql server! backup stop!"
  exit
else
  echo "MySQL connect ok! Please wait......"
   # Judgment does not define the backup database, if you define a backup is started, otherwise exit the backup
  if ["$backup_db_arr"!=""]; then
	   # dbnames=$(cut-d ','-f1-5 $backup_database)
	   # echo "arr is(${backup_db_arr [@]})"
	  for dbname in ${backup_db_arr [@]}
	  do
		  echo "database $dbname backup start..."
		  `mkdir -p $backup_dir`
		  `mysqldump -h $mysql_host -P $mysql_port -u $mysql_user -p $mysql_password $dbname - default-character-set=$mysql_charset | gzip> $backup_dir/$dbname -$backup_time.sql.gz`
		  flag=`echo $?`
		  if [$flag=="0"]; then
			  echo "database $dbname success backup to $backup_dir/$dbname-$backup_time.sql.gz"
		  else
			  echo "database $dbname backup fail!"
		  fi
		  
	  done
  else
	  echo "ERROR: No database to backup! backup stop"
	  exit
  fi
   # If you open the delete expired backup, delete operation
  if ["$expire_backup_delete"=="ON" -a "$backup_location"!=""]; then
	  # `find $backup_location/-type d -o -type f -ctime + $expire_days-exec rm -rf {} \;`
	  `find $backup_location/ -type d -mtime + $expire_days | xargs rm -rf`
	  echo "Expired backup data delete complete!"
  fi
  echo "All database backup success! Thank you!"
  exit
fi

Modify shell script attributes:

chmod 600 /root/mysql_backup.sh
chmod +x /root/mysql_backup.sh

Set attributes, add the command crontab, we set up regular automatic backups every day 00:00, then the backup script directory/var/www/mysql directory is set to rsync synchronization.

00 00 *** /root/mysql_backup.sh

MySQL Recovery

Earlier MySQL backup into hot backup and cold backup, hot backup main purpose is to be able to recover in real time, such as an application server hard disk failure occurred, then we can modify the database configuration file read and write into slave so that you can minimize the time interrupt service.

But sometimes we need to perform a cold backup of the SQL data recovery, as with database backup, you can import through the command:

mysql -u username -p databse < backup.sql

You can see, export and import database data is fairly simple, but if you also need to manage permissions, or some other character set, it may be a little more complicated, but these can all be done through a number of commands.

Redis backup

Redis is our most used NoSQL, its backup is also divided into two kinds: hot backup and cold backup, Redis also supports master/slave mode, so our hot backup can be achieved in this way, we can refer to the corresponding configuration the official document profiles, quite simple. Here we introduce cold backup mode: Redis will actually timed inside the memory cache data saved to the database file inside, we just backed up the corresponding file can be, is to use rsync backup to a previously described non-local machine room can be achieved.

Redis recovery

Redis Recovery divided into hot and cold backup recovery backup and recovery, hot backup and recovery purposes and methods of recovery with MySQL, as long as the modified application of the corresponding database connection.

But sometimes we need to cold backup to recover data, Redis cold backup and recovery is actually just put the saved database file copy to Redis working directory, and then start Redis on it, Redis at boot time will be automatically loaded into the database file memory, the start speed of the database to determine the size of the file.

Summary

This section describes the application of part of our backup and recovery, that is, how to do disaster recovery, including file backup, database backup. Also introduced different systems using rsync file synchronization, MySQL database and Redis database backup and recovery, hope that through the introduction of this section, you can give as a developer of products for online disaster recovery program provides a reference solution.

Links

• Directory
• Previous section: Deployment
• Next section: Summary