Merging other languages
This commit is contained in:
James Miranda
359
en/12.4.md
359
en/12.4.md
@@ -1,174 +1,185 @@
|
||||
# 12.4 备份和恢复
|
||||
这小节我们要讨论应用程序管理的另一个方面：生产服务器上数据的备份和恢复。我们经常会遇到生产服务器的网络断了、硬盘坏了、操作系统崩溃、或者数据库不可用了等各种异常情况，所以维护人员需要对生产服务器上的应用和数据做好异地灾备，冷备热备的准备。在接下来的介绍中，讲解了如何备份应用、如何备份/恢复Mysql数据库和redis数据库。
|
||||
|
||||
## 应用备份
|
||||
在大多数集群环境下，Web应用程序基本不需要备份，因为这个其实就是一个代码副本，我们在本地开发环境中，或者版本控制系统中已经保持这些代码。但是很多时候，一些开发的站点需要用户来上传文件，那么我们需要对这些用户上传的文件进行备份。目前其实有一种合适的做法就是把和网站相关的需要存储的文件存储到云储存，这样即使系统崩溃，只要我们的文件还在云存储上，至少数据不会丢失。
|
||||
|
||||
如果我们没有采用云储存的情况下，如何做到网站的备份呢？这里我们介绍一个文件同步工具rsync：rsync能够实现网站的备份，不同系统的文件的同步，如果是windows的话，需要windows版本cwrsync。
|
||||
|
||||
### rsync安装
|
||||
rysnc的官方网站：http://rsync.samba.org/ 可以从上面获取最新版本的源码。当然，因为rsync是一款非常有用的软件，所以很多Linux的发行版本都将它收录在内了。
|
||||
|
||||
软件包安装
|
||||
|
||||
# sudo apt-get install rsync 注：在debian、ubuntu 等在线安装方法；
|
||||
# yum install rsync 注：Fedora、Redhat、CentOS 等在线安装方法；
|
||||
# rpm -ivh rsync 注：Fedora、Redhat、CentOS 等rpm包安装方法；
|
||||
|
||||
其它Linux发行版，请用相应的软件包管理方法来安装。源码包安装
|
||||
|
||||
tar xvf rsync-xxx.tar.gz
|
||||
cd rsync-xxx
|
||||
./configure --prefix=/usr ;make ;make install 注：在用源码包编译安装之前，您得安装gcc等编译工具才行；
|
||||
|
||||
### rsync配置
|
||||
rsync主要有以下三个配置文件rsyncd.conf(主配置文件)、rsyncd.secrets(密码文件)、rsyncd.motd(rysnc服务器信息)。
|
||||
|
||||
关于这几个文件的配置大家可以参考官方网站或者其他介绍rsync的网站，下面介绍服务器端和客户端如何开启
|
||||
|
||||
- 服务端开启：
|
||||
|
||||
#/usr/bin/rsync --daemon --config=/etc/rsyncd.conf
|
||||
|
||||
--daemon参数方式，是让rsync以服务器模式运行。把rsync加入开机启动
|
||||
|
||||
echo 'rsync --daemon' >> /etc/rc.d/rc.local
|
||||
|
||||
设置rsync密码
|
||||
|
||||
echo '你的用户名:你的密码' > /etc/rsyncd.secrets
|
||||
chmod 600 /etc/rsyncd.secrets
|
||||
|
||||
|
||||
- 客户端同步：
|
||||
|
||||
客户端可以通过如下命令同步服务器上的文件：
|
||||
|
||||
rsync -avzP --delete --password-file=rsyncd.secrets 用户名@192.168.145.5::www /var/rsync/backup
|
||||
|
||||
这条命令，简要的说明一下几个要点：
|
||||
|
||||
1. -avzP是啥，读者可以使用--help查看
|
||||
2. --delete 是为了比如A上删除了一个文件，同步的时候，B会自动删除相对应的文件
|
||||
3. --password-file 客户端中/etc/rsyncd.secrets设置的密码，要和服务端的 /etc/rsyncd.secrets 中的密码一样，这样cron运行的时候，就不需要密码了
|
||||
4. 这条命令中的"用户名"为服务端的 /etc/rsyncd.secrets中的用户名
|
||||
5. 这条命令中的 192.168.145.5 为服务端的IP地址
|
||||
6. ::www，注意是2个 : 号，www为服务端的配置文件 /etc/rsyncd.conf 中的[www]，意思是根据服务端上的/etc/rsyncd.conf来同步其中的[www]段内容，一个 : 号的时候，用于不根据配置文件，直接同步指定目录。
|
||||
|
||||
为了让同步实时性，可以设置crontab，保持rsync每分钟同步，当然用户也可以根据文件的重要程度设置不同的同步频率。
|
||||
|
||||
|
||||
## MySQL备份
|
||||
应用数据库目前还是MySQL为主流，目前MySQL的备份有两种方式：热备份和冷备份，热备份目前主要是采用master/slave方式（master/slave方式的同步目前主要用于数据库读写分离，也可以用于热备份数据），关于如何配置这方面的资料，大家可以找到很多。冷备份的话就是数据有一定的延迟，但是可以保证该时间段之前的数据完整，例如有些时候可能我们的误操作引起了数据的丢失，那么master/slave模式是无法找回丢失数据的，但是通过冷备份可以部分恢复数据。
|
||||
|
||||
冷备份一般使用shell脚本来实现定时备份数据库，然后通过上面介绍rsync同步非本地机房的一台服务器。
|
||||
|
||||
下面这个是定时备份mysql的备份脚本，我们使用了mysqldump程序，这个命令可以把数据库导出到一个文件中。
|
||||
|
||||
#!/bin/bash
|
||||
|
||||
# 以下配置信息请自己修改
|
||||
mysql_user="USER" #MySQL备份用户
|
||||
mysql_password="PASSWORD" #MySQL备份用户的密码
|
||||
mysql_host="localhost"
|
||||
mysql_port="3306"
|
||||
mysql_charset="utf8" #MySQL编码
|
||||
backup_db_arr=("db1" "db2") #要备份的数据库名称，多个用空格分开隔开 如("db1" "db2" "db3")
|
||||
backup_location=/var/www/mysql #备份数据存放位置，末尾请不要带"/",此项可以保持默认，程序会自动创建文件夹
|
||||
expire_backup_delete="ON" #是否开启过期备份删除 ON为开启 OFF为关闭
|
||||
expire_days=3 #过期时间天数 默认为三天，此项只有在expire_backup_delete开启时有效
|
||||
|
||||
# 本行开始以下不需要修改
|
||||
backup_time=`date +%Y%m%d%H%M` #定义备份详细时间
|
||||
backup_Ymd=`date +%Y-%m-%d` #定义备份目录中的年月日时间
|
||||
backup_3ago=`date -d '3 days ago' +%Y-%m-%d` #3天之前的日期
|
||||
backup_dir=$backup_location/$backup_Ymd #备份文件夹全路径
|
||||
welcome_msg="Welcome to use MySQL backup tools!" #欢迎语
|
||||
|
||||
# 判断MYSQL是否启动,mysql没有启动则备份退出
|
||||
mysql_ps=`ps -ef |grep mysql |wc -l`
|
||||
mysql_listen=`netstat -an |grep LISTEN |grep $mysql_port|wc -l`
|
||||
if [ [$mysql_ps == 0] -o [$mysql_listen == 0] ]; then
|
||||
echo "ERROR:MySQL is not running! backup stop!"
|
||||
exit
|
||||
else
|
||||
echo $welcome_msg
|
||||
fi
|
||||
|
||||
# 连接到mysql数据库，无法连接则备份退出
|
||||
mysql -h$mysql_host -P$mysql_port -u$mysql_user -p$mysql_password <<end
|
||||
use mysql;
|
||||
select host,user from user where user='root' and host='localhost';
|
||||
exit
|
||||
end
|
||||
|
||||
flag=`echo $?`
|
||||
if [ $flag != "0" ]; then
|
||||
echo "ERROR:Can't connect mysql server! backup stop!"
|
||||
exit
|
||||
else
|
||||
echo "MySQL connect ok! Please wait......"
|
||||
# 判断有没有定义备份的数据库，如果定义则开始备份，否则退出备份
|
||||
if [ "$backup_db_arr" != "" ];then
|
||||
#dbnames=$(cut -d ',' -f1-5 $backup_database)
|
||||
#echo "arr is (${backup_db_arr[@]})"
|
||||
for dbname in ${backup_db_arr[@]}
|
||||
do
|
||||
echo "database $dbname backup start..."
|
||||
`mkdir -p $backup_dir`
|
||||
`mysqldump -h$mysql_host -P$mysql_port -u$mysql_user -p$mysql_password $dbname --default-character-set=$mysql_charset | gzip > $backup_dir/$dbname-$backup_time.sql.gz`
|
||||
flag=`echo $?`
|
||||
if [ $flag == "0" ];then
|
||||
echo "database $dbname success backup to $backup_dir/$dbname-$backup_time.sql.gz"
|
||||
else
|
||||
echo "database $dbname backup fail!"
|
||||
fi
|
||||
|
||||
done
|
||||
else
|
||||
echo "ERROR:No database to backup! backup stop"
|
||||
exit
|
||||
fi
|
||||
# 如果开启了删除过期备份，则进行删除操作
|
||||
if [ "$expire_backup_delete" == "ON" -a "$backup_location" != "" ];then
|
||||
#`find $backup_location/ -type d -o -type f -ctime +$expire_days -exec rm -rf {} \;`
|
||||
`find $backup_location/ -type d -mtime +$expire_days | xargs rm -rf`
|
||||
echo "Expired backup data delete complete!"
|
||||
fi
|
||||
echo "All database backup success! Thank you!"
|
||||
exit
|
||||
fi
|
||||
|
||||
修改shell脚本的属性：
|
||||
|
||||
chmod 600 /root/mysql_backup.sh
|
||||
chmod +x /root/mysql_backup.sh
|
||||
|
||||
设置好属性之后，把命令加入crontab，我们设置了每天00:00定时自动备份，然后把备份的脚本目录/var/www/mysql设置为rsync同步目录。
|
||||
|
||||
00 00 * * * /root/mysql_backup.sh
|
||||
|
||||
## MySQL恢复
|
||||
前面介绍MySQL备份分为热备份和冷备份，热备份主要的目的是为了能够实时的恢复，例如应用服务器出现了硬盘故障，那么我们可以通过修改配置文件把数据库的读取和写入改成slave，这样就可以尽量少时间的中断服务。
|
||||
|
||||
但是有时候我们需要通过冷备份的SQL来进行数据恢复，既然有了数据库的备份，就可以通过命令导入：
|
||||
|
||||
mysql -u username -p databse < backup.sql
|
||||
|
||||
可以看到，导出和导入数据库数据都是相当简单，不过如果还需要管理权限，或者其他的一些字符集的设置的话，可能会稍微复杂一些，但是这些都是可以通过一些命令来完成的。
|
||||
|
||||
## redis备份
|
||||
redis是目前我们使用最多的NoSQL，它的备份也分为两种：热备份和冷备份，redis也支持master/slave模式，所以我们的热备份可以通过这种方式实现，相应的配置大家可以参考官方的文档配置，相当的简单。我们这里介绍冷备份的方式：redis其实会定时的把内存里面的缓存数据保存到数据库文件里面，我们备份只要备份相应的文件就可以，就是利用前面介绍的rsync备份到非本地机房就可以实现。
|
||||
|
||||
## redis恢复
|
||||
redis的恢复分为热备份恢复和冷备份恢复，热备份恢复的目的和方法同MySQL的恢复一样，只要修改应用的相应的数据库连接即可。
|
||||
|
||||
但是有时候我们需要根据冷备份来恢复数据，redis的冷备份恢复其实就是只要把保存的数据库文件copy到redis的工作目录，然后启动redis就可以了，redis在启动的时候会自动加载数据库文件到内存中，启动的速度根据数据库的文件大小来决定。
|
||||
|
||||
## 小结
|
||||
本小节介绍了我们的应用部分的备份和恢复，即如何做好灾备，包括文件的备份、数据库的备份。同时也介绍了使用rsync同步不同系统的文件，MySQL数据库和redis数据库的备份和恢复，希望通过本小节的介绍，能够给作为开发的你对于线上产品的灾备方案提供一个参考方案。
|
||||
|
||||
## links
|
||||
* [目录](<preface.md>)
|
||||
* 上一章: [应用部署](<12.3.md>)
|
||||
* 下一节: [小结](<12.5.md>)
|
||||
# 12.4 Backup and recovery
|
||||
|
||||
This section we discuss another aspect of application management: production server data backup and recovery. We often encounter the production server network is broken, bad hard drive, operating system crash, or if the database is unavailable a variety of unusual circumstances, so maintenance personnel need to produce applications and data on the server to do remote disaster recovery, cold prepare hot standby ready. In the next presentation, explained how the backup application, how to backup/restore MySQL database and Redis databases.
|
||||
|
||||
## Application Backup
|
||||
|
||||
In most cluster environment, Web applications, the basic need for backup, because this is actually a copy of the code, we are in the local development environment, or the version control system has to maintain the code. But many times, a number of development sites require users to upload files, then we need for these users to upload files for backup. In fact, now there is a suitable approach is to put the needs and site-related files stored on the storage to the cloud storage, so even if the system crashes, as long as we still cloud file storage, at least the data is not lost.
|
||||
|
||||
If we do not adopt cloud storage case, how to do a backup site do ? Here we introduce a file synchronization tool rsync: rsync backup site can be achieved in different file system synchronization, If the windows, then, need windows version cwrsync.
|
||||
|
||||
### Rsync installation
|
||||
|
||||
rsync 's official website: http://rsync.samba.org/can get the latest version from the above source. Of course, because rsync is a very useful software, so many Linux distributions will include it, including the.
|
||||
|
||||
Package Installation
|
||||
|
||||
# sudo apt-get install rsync ; Note: debian, ubuntu and other online installation methods ;
|
||||
# yum install rsync ; Note: Fedora, Redhat, CentOS and other online installation methods ;
|
||||
# rpm -ivh rsync ; Note: Fedora, Redhat, CentOS and other rpm package installation methods ;
|
||||
|
||||
Other Linux distributions, please use the appropriate package management methods to install. Installing source packages
|
||||
|
||||
tar xvf rsync-xxx.tar.gz
|
||||
cd rsync-xxx
|
||||
./configure - prefix =/usr; make; make install
|
||||
|
||||
<blockquote>Note: Before using source packages compiled and installed, you have to install gcc compiler tools such as job</blockquote>
|
||||
|
||||
### Rsync Configure
|
||||
|
||||
rsync mainly in the following three configuration files rsyncd.conf( main configuration file ), rsyncd.secrets( password file ), rsyncd.motd(rysnc server information ).
|
||||
|
||||
Several documents about this configuration we can refer to the official website or other websites rsync introduction, here the server and client how to open
|
||||
|
||||
- Services client opens:
|
||||
|
||||
`# /usr/bin/rsync --daemon --config=/etc/rsyncd.conf`
|
||||
|
||||
- daemon parameter approach is to run rsync in server mode. Join the rsync boot
|
||||
|
||||
`echo 'rsync - daemon' >> /etc/rc.d/rc.local`
|
||||
|
||||
Set rsync password
|
||||
|
||||
echo 'Your Username: Your Password' > /etc/rsyncd.secrets
|
||||
chmod 600 /etc/rsyncd.secrets
|
||||
|
||||
- Client synchronization:
|
||||
|
||||
Clients can use the following command to synchronize the files on the server:
|
||||
|
||||
rsync -avzP --delete --password-file=rsyncd.secrets username@192.168.145.5::www/var/rsync/backup
|
||||
|
||||
This command, briefly explain a few points:
|
||||
|
||||
1. `-avzP` is what the reader can use the `-help` Show
|
||||
2. `-delete` for example A, deleted a file, the time synchronization, B will automatically delete the corresponding files
|
||||
3. `-Password-file` client/etc/rsyncd.secrets set password, and server to `/etc/rsyncd.secrets` the password the same, so cron is running, you do not need the password
|
||||
4. This command in the " User Name" for the service side of the `/etc/rsyncd.secrets` the user name
|
||||
5. This command 192.168.145.5 as the IP address of the server
|
||||
6. :: www, note the two: number, www as a server configuration file `/etc/rsyncd.conf` in [www], meaning that according to the service on the client `/etc/rsyncd.conf` to synchronize them [www] paragraph, a: number, when used according to the configuration file does not directly specify the directory synchronization.
|
||||
|
||||
In order to synchronize real-time, you can set the crontab, keeping rsync synchronization every minute, of course, users can also set the level of importance depending on the file type of synchronization frequency.
|
||||
|
||||
## MySQL backup
|
||||
|
||||
MySQL database application is still the mainstream, the current MySQL backup in two ways: hot backup and cold backup, hot backup is currently mainly used master/slave mode (master/slave) mode is mainly used for database synchronization separate read and write, but also can be used for hot backup data ), on how to configure this information, we can find a lot. Cold backup data, then that is a certain delay, but you can guarantee that the time period before data integrity, such as may sometimes be caused by misuse of our loss of data, then the master/slave model is able to retrieve lost data, but through cold backup can partially restore the data.
|
||||
|
||||
Cold backup shell script is generally used to achieve regular backup of the database, and then rsync synchronization through the above described non-local one server room.
|
||||
|
||||
The following is a scheduled backup MySQL backup script, we use the mysqldump program, this command can be exported to a database file.
|
||||
|
||||
|
||||
#!/bin/bash
|
||||
# The following configuration information, modify their own
|
||||
mysql_user="USER" #MySQL backup user
|
||||
mysql_password="PASSWORD" # MySQL backup user's password
|
||||
mysql_host="localhost"
|
||||
mysql_port="3306"
|
||||
mysql_charset="utf8" # MySQL coding
|
||||
backup_db_arr=("db1" "db2") # To back up the database name, separated by spaces separated by a plurality of such("db1" "db2" "db3")
|
||||
backup_location=/var/www/mysql # backup data storage location, please do not end with a "/", this can keep the default, the program will automatically create a folder
|
||||
expire_backup_delete="ON" # delete outdated backups is turned OFF to ON ON to OFF
|
||||
expire_days=3 # default expiration time for the three days the number of days, this is only valid when the expire_backup_delete open
|
||||
|
||||
# We do not need to modify the following start
|
||||
backup_time=`date +%Y%m%d%H%M` # define detailed time backup
|
||||
backup_Ymd=`date +%Y-%m-%d` # define the backup directory date time
|
||||
backup_3ago=`date-d '3 days ago '+%Y-%m-%d` # 3 days before the date
|
||||
backup_dir=$backup_location/$backup_Ymd # full path to the backup folder
|
||||
welcome_msg="Welcome to use MySQL backup tools!" # greeting
|
||||
|
||||
# Determine whether to start MYSQL, mysql does not start the backup exit
|
||||
mysql_ps=`ps-ef | grep mysql | wc-l`
|
||||
mysql_listen=`netstat-an | grep LISTEN | grep $mysql_port | wc-l`
|
||||
if [[$mysql_ps==0]-o [$mysql_listen==0]]; then
|
||||
echo "ERROR: MySQL is not running! backup stop!"
|
||||
exit
|
||||
else
|
||||
echo $welcome_msg
|
||||
fi
|
||||
|
||||
# Connect to mysql database, can not connect to the backup exit
|
||||
mysql-h $mysql_host-P $mysql_port-u $mysql_user-p $mysql_password << end
|
||||
use mysql;
|
||||
select host, user from user where user='root' and host='localhost';
|
||||
exit
|
||||
end
|
||||
|
||||
flag=`echo $?`
|
||||
if [$flag!="0"]; then
|
||||
echo "ERROR: Can't connect mysql server! backup stop!"
|
||||
exit
|
||||
else
|
||||
echo "MySQL connect ok! Please wait......"
|
||||
# Judgment does not define the backup database, if you define a backup is started, otherwise exit the backup
|
||||
if ["$backup_db_arr"!=""]; then
|
||||
# dbnames=$(cut-d ','-f1-5 $backup_database)
|
||||
# echo "arr is(${backup_db_arr [@]})"
|
||||
for dbname in ${backup_db_arr [@]}
|
||||
do
|
||||
echo "database $dbname backup start..."
|
||||
`mkdir -p $backup_dir`
|
||||
`mysqldump -h $mysql_host -P $mysql_port -u $mysql_user -p $mysql_password $dbname - default-character-set=$mysql_charset | gzip> $backup_dir/$dbname -$backup_time.sql.gz`
|
||||
flag=`echo $?`
|
||||
if [$flag=="0"]; then
|
||||
echo "database $dbname success backup to $backup_dir/$dbname-$backup_time.sql.gz"
|
||||
else
|
||||
echo "database $dbname backup fail!"
|
||||
fi
|
||||
|
||||
done
|
||||
else
|
||||
echo "ERROR: No database to backup! backup stop"
|
||||
exit
|
||||
fi
|
||||
# If you open the delete expired backup, delete operation
|
||||
if ["$expire_backup_delete"=="ON" -a "$backup_location"!=""]; then
|
||||
# `find $backup_location/-type d -o -type f -ctime + $expire_days-exec rm -rf {} \;`
|
||||
`find $backup_location/ -type d -mtime + $expire_days | xargs rm -rf`
|
||||
echo "Expired backup data delete complete!"
|
||||
fi
|
||||
echo "All database backup success! Thank you!"
|
||||
exit
|
||||
fi
|
||||
|
||||
|
||||
Modify shell script attributes:
|
||||
|
||||
chmod 600 /root/mysql_backup.sh
|
||||
chmod +x /root/mysql_backup.sh
|
||||
|
||||
Set attributes, add the command crontab, we set up regular automatic backups every day 00:00, then the backup script directory/var/www/mysql directory is set to rsync synchronization.
|
||||
|
||||
00 00 *** /root/mysql_backup.sh
|
||||
|
||||
## MySQL Recovery
|
||||
|
||||
Earlier MySQL backup into hot backup and cold backup, hot backup main purpose is to be able to recover in real time, such as an application server hard disk failure occurred, then we can modify the database configuration file read and write into slave so that you can minimize the time interrupt service.
|
||||
|
||||
But sometimes we need to perform a cold backup of the SQL data recovery, as with database backup, you can import through the command:
|
||||
|
||||
mysql -u username -p databse < backup.sql
|
||||
|
||||
You can see, export and import database data is fairly simple, but if you also need to manage permissions, or some other character set, it may be a little more complicated, but these can all be done through a number of commands.
|
||||
|
||||
## Redis backup
|
||||
|
||||
Redis is our most used NoSQL, its backup is also divided into two kinds: hot backup and cold backup, Redis also supports master/slave mode, so our hot backup can be achieved in this way, we can refer to the corresponding configuration the official document profiles, quite simple. Here we introduce cold backup mode: Redis will actually timed inside the memory cache data saved to the database file inside, we just backed up the corresponding file can be, is to use rsync backup to a previously described non-local machine room can be achieved.
|
||||
|
||||
## Redis recovery
|
||||
|
||||
Redis Recovery divided into hot and cold backup recovery backup and recovery, hot backup and recovery purposes and methods of recovery with MySQL, as long as the modified application of the corresponding database connection.
|
||||
|
||||
But sometimes we need to cold backup to recover data, Redis cold backup and recovery is actually just put the saved database file copy to Redis working directory, and then start Redis on it, Redis at boot time will be automatically loaded into the database file memory, the start speed of the database to determine the size of the file.
|
||||
|
||||
## Summary
|
||||
|
||||
This section describes the application of part of our backup and recovery, that is, how to do disaster recovery, including file backup, database backup. Also introduced different systems using rsync file synchronization, MySQL database and Redis database backup and recovery, hope that through the introduction of this section, you can give as a developer of products for online disaster recovery program provides a reference solution.
|
||||
|
||||
## Links
|
||||
|
||||
- [Directory](preface.md)
|
||||
- Previous section: [Deployment](12.3.md)
|
||||
- Next section: [Summary](12.5.md)
|
||||
|
||||
Reference in New Issue
Block a user