全网备份和实时同步

备份基本情况
已知 3 台服务器主机名分别为 A(web01)、B(backup) 、C(nfs01)
要求:每天晚上 00 点整在 Web 服务器 A 上打包备份系统配置文件、网站程序目录及访问日 志并通过 rsync 命令推送备份服务器 B 上备份保留(备份思路可以是先在本地按日期打包, 然后再推到备份服务器 B 上)
要求如下:
1)Web 服务器 A 和备份服务器 B 的备份目录必须都为/backup。
2)要备份的系统配置文件包括但不限于:

  a.定时任务服务的配置文件(/var/spool/cron/root)
  b.开机自启动的配置文件(/etc/rc.local)
  c.日常脚本的目录 (/server/scripts)。
  d.防火墙 iptables 的配置文件(/etc/sysconfig/iptables)

3)Web 服务器站点目录假定为(/var/html/www)。
4)Web 服务器 A 访问日志路径假定为(/app/logs)
5)Web 服务器保留打包后的 7 天的备份数据即可(本地留存不能多于 7 天,因为太多硬盘会满)
6)备份服务器 B 上,保留每周一的所有数据副本,其它要保留 6 个月的数据副本。
7)备份服务器 B 上要按照 web 服务器的 IP 为目录保存备份,备份的文件按照时间名字保存。
说明:此部分内容为工作中是网站生产环境全网备份项目方案的一个小型模拟

NFS 共享存储搭建及优化

配置 NFS 服务
要求:

1)在NFS服务端C(nfs01)上共享/data/w_shared 及/data/r_shared两个文件目录,允许从NFS客户端A(web01)、B(backup)上分别挂载共享目录后可实现从 A(web01)、B(backup)上只读/data/r_shared可写/data/w_shared。
2)NFS客户端 A(web01)上的挂载点为/data/b_w(写)/data/b_r(读)NFS客户端B(backup)上的挂载点为/data/w_你的名字英文(写)/data/r_你名字英文(读)
3)从NFS客户端 B(backup)上的NFS可写挂载点目录创建任意文件,从NFS客户端A(web01) 上可以删除这个创建的文件,反之也可以
4)优化 NFS服务

解决网站集群后端 NFS 共享存储单点实现实时数据同步
当用户通过web服务器将数据写入到NFS服务器C(nfs01)时,同时复制到备份服务器B(backup)

环境说明

服务器角色	    外网 ip	    内网 ip	       主机名
web	            10.0.0.8	172.16.1.8	    web01
nfs	            10.0.0.31	172.16.1.31	    nfs01
backup	        10.0.0.41	172.16.1.41	    backup

目录规划

web01
   /var/html/www
   /app/logs
   /backup
   /server/scripts
nfs
   /backup
   /data/r_shared
   /data/w_shared
   /service
backup
   /server/scripts
   /nfsbacup
   /backup
   /data/r_liuliya/
   /data/w_liuliya/

部署实施
1.backup服务器搭建rsync服务
cat /server/scripts/rsyncser.sh

#################start
#!/bin/bash
#author by liuliya at 20160425
user=rsync
authuser=rsync_backup
passwd=liuliya
passwdfile=/etc/rsync.password
bakpath1=/backup
bakpath2=/nfsbackup
#install rsync
/usr/bin/yum -y install rsync
#create user
/usr/sbin/useradd $user -s /sbin/nologin -M
#create configfile
cat >> /etc/rsyncd.conf <<EOF
##rsync_config  start##
#created by liuliya 2016-04-21
#QQ 865205026 blog:https://www.liuliya.com
##rsyncd.conf start##
uid = $user
gid = $user
use chroot = no
max connections = 200
timeout = 300
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsyncd.log
ignore errors
read only = false
list = false
hosts allow = 172.16.1.0/24
#hosts deny = 0.0.0.0/32
auth users = $authuser
secrets file = $passwdfile
[backup]
path = $bakpath1
[nfsbackup]
path = $bakpath2
#rsync_config end
EOF
#create passwordfile
echo "$authuser:$passwd" > $passwdfile
/bin/chmod 600 $passwdfile
#create backuppath
/bin/mkdir -p $bakpath1
/bin/chown -R $user.$user $bakpath1
/bin/mkdir -p $bakpath2
/bin/chown -R $user.$user $bakpath2
#onboot
echo "/usr/bin/rsync --daemon"	>> /etc/rc.local
#start daemon
/usr/bin/rsync --daemon
lsof -i :873
#################end

web01和nfs配置rsync和本地备份目录

cat /server/scripts/rsyncclient.sh
#!/bin/bash
#author by liuliya at 20160425
authuser=rsync_backup
passwd=liuliya
passwdfile=/etc/rsync.password
bakpath=/backup
#install rsync
/usr/bin/yum -y install rsync
#create passwordfile
echo "$passwd" > $passwdfile
/bin/chmod 600 $passwdfile
#create backuppath
/bin/mkdir -p $bakpath

web01上的备份脚本
cat /server/scripts/backup.sh

#!/bin/bash
ip=$(/sbin/ifconfig eth1|awk -F'[ :]+' 'NR==2{print $4}')
backupdir=/backup
user=rsync_backup
backupserver=172.16.1.41
module=backup
passfile=/etc/rsync.password
if [ ! -e $backupdir/$ip ];
    then mkdir -p $backupdir/$ip
fi
if [ $(date +%w) -eq 1 ];
    then time=$(date +%F_%w)
  else
time=$(date +%F)
fi
cd / && \
tar zcfh $backupdir/$ip/web_$time.tar.gz var/spool/cron/root etc/rc.local etc/sysconfig/iptables server/scripts etc/sysctl.conf etc/security/limits.conf etc/sysconfig/iptables var/html/www app/logs && \
md5sum $backupdir/$ip/web_$time.tar.gz > $backupdir/$ip/flag_$(date +%F).log &&\ rsync -az --password-file=$passfile $backupdir/ $user@$backupserver::$module/
find $backupdir -type f -mtime +7 \( -name "*.tar.gz" -o -name "*.log" \)|xargs rm -f

backup 上的检查脚本和邮件配置

cat >> /etc/mail.rc << EOF
set from=testforlinux@139.com smtp=smtp.139.com smtp-auth-user=testforlinux smtp-auth-password=139test smtp-auth=login
EOF

cat /services/scripts/check.sh

#!/bin/bash backupdir=/backup
find $backupdir -type f -name "flag_$(date +%F).log"|xargs md5sum -c $flagfile >>$backupdir/$(date +%F)_result.log 2>&1
mail -s "$(date +%F) backup result" testforlinux@139.com < $backupdir/$(date +%F)_result.log && \ >$backupdir/$(date +%F)_result.log
find $backupdir -type f ! -name "*_1.tar.gz" -mtime +180|xargs rm -f

backup添加定时任务

cat >> /var/spool/cron/root <<EOF
#check
00 04 * * * /bin/bash /server/scripts/check.sh &> /dev/null
EOF

web01添加定时任务

cat >> /var/spool/cron/root <<EOF
#backup
00 00 * * * /bin/bash /server/scripts/backup.sh &> /dev/null
EOF

nfs01配置共享
nfs 服务器安装

cat /server/scripts/nfsser.sh
#!/bin/bash
share1=/data/r_shared
share2=/data/w_shared
user=nfsnobody
group=nfsnobody
ip=$(/sbin/ifconfig eth1|awk -F'[ :]+' 'NR==2{print $4}')
net=$(/sbin/ifconfig eth1|awk -F'[ :]+' 'NR==2{print $4}'|/bin/cut -d"." -f1-3)
mask=".0/24"
opt1="ro,sync,all_squash,anonuid=65534,anongid=65534"
opt2="rw,sync,all_squash,anonuid=65534,anongid=65534"
#install
/usr/bin/yum -y install rpcbind nfs-utils
#share folder
mkdir -p $share1
mkdir -p $share2
chown -R $user.$group $mountpoint1
chown -R $user.$group $mountpoint2
#youhua
cat >> /etc/sysctl.conf <<EOF
/proc/sys/net/core/rmem_default = 8388608
/proc/sys/net/core/rmem_max = 8388608
/proc/sys/net/core/wmem_default = 16777216
/proc/sys/net/core/wmem max = 16777216
EOF
sysctl -p
#nfs conffile
/bin/cat >> /etc/exports <<EOF
$share2 $net$mask($opt2)
$share1 $net$mask($opt1)
EOF
#start service and onboot
/etc/init.d/rpcbind start && \
/etc/init.d/nfs start
echo "/etc/init.d/rpcbind start" >> /etc/rc.local
echo "/etc/init.d/nfs start" >> /etc/rc.local
#test
/usr/sbin/showmount -e $ip

web01挂载共享
创建共享目录

cat nfsmount.sh
#!/bin/bash
nfsserver=172.16.1.31
user=nfsnobody
group=nfsnobody
mountpoint1=/data/b_r
mountpoint2=/data/b_w
mountopt="nosuid,noexec,nodev,noatime,nodiratime,rsize=131072,wsize=131072"
#install
/usr/bin/yum -y install rpcbind nfs-utils
#start rpcbind service
/etc/init.d/rpcbind start
#mkdir
mkdir -p $mountpoint1
mkdir -p $mountpoint2
chown -R $user.$group $mountpoint1
chown -R $user.$group $mountpoint2
#mount
share1=$(/usr/sbin/showmount -e $nfsserver|awk 'NR==2{print $1}')
share2=$(/usr/sbin/showmount -e $nfsserver|awk 'NR==3{print $1}')
mount -t nfs -o $mountopt $nfsserver:$share1 $mountpoint1
mount -t nfs -o $mountopt $nfsserver:$share2 $mountpoint2
#mout on boot
echo "mount -t nfs -o $mountopt	$nfsserver:$share1 $mountpoint1" >> /etc/rc.local
echo "mount -t nfs -o $mountopt	$nfsserver:$share2 $mountpoint2" >> /etc/rc.local

backup 挂载共享
创建目录

cat nfsmount.sh
#!/bin/bash
nfsserver=172.16.1.31
mountpoint1=/data/r_liuliya
mountpoint2=/data/w_liuliya
mountopt="nosuid,noexec,nodev,noatime,nodiratime,rsize=131072,wsize=131072"
#install
/usr/bin/yum -y install rpcbind nfs-utils
#mkdir
mkdir -p $mountpoint1
mkdir -p $mountpoint2
chown -R nfsnobody.nfsnobody $mountpoint1
chown -R nfsnobody.nfsnobody $mountpoint2
#start rpcbind service
/etc/init.d/rpcbind start
#mount
share1=$(/usr/sbin/showmount -e $nfsserver|awk 'NR==2{print $1}')
share2=$(/usr/sbin/showmount -e $nfsserver|awk 'NR==3{print $1}')
mount -t nfs -o $mountopt $nfsserver:$share1 $mountpoint1
mount -t nfs -o $mountopt $nfsserver:$share2 $mountpoint2
#mout on boot
echo "mount -t nfs -o $mountopt	$nfsserver:$share1 $mountpoint1" >> /etc/rc.local
echo "mount -t nfs -o $mountopt	$nfsserver:$share2 $mountpoint2" >> /etc/rc.local

nfs 实时同步
nfs 服务器安装 inotify 脚本和 inotify 服务
inotify 安装和优化
yum -y install inotify-tools
#youhua
echo 655350 >/proc/sys/fs/inotify/max_user_watches
echo 655350 >/proc/sys/fs/inotify/max_queued_events

cat /server/scripts/inotify.sh
#!/bin/bash
Path=/data/w_shared
Ip=172.16.1.41
/usr/bin/inotifywait -mrq --format '%w%f' -e create,close_write,delete $Path \
|while read file
  do
  if [ -f $file ];then
    rsync -az $file --delete rsync_backup@$Ip::nfsbackup --password-file=/etc/rsync.password else
cd $Path &&\
  rsync -az ./ --delete rsync_backup@$Ip::nfsbackup --password-file=/etc/rsync.password
  fi
done

inotify 服务脚本

cat /etc/init.d/inotify
#!/bin/bash
#chkconfig: 2345 38 46
. /etc/init.d/functions
if [ $# -ne 1 ];then
    usage: $0 {start|stop}
    exit 1
fi
case "$1" in
start)
  /bin/bash /server/scripts/inotify.sh &
  echo $$ >/var/run/inotify.pid
  if [ `ps -ef|grep inotify|wc -l` -gt 2 ];then
    action "inotify service is started" /bin/true
  else
    action "inotify
service is started" /bin/false
  fi
  ;;
stop)
  kill -9 `cat /var/run/inotify.pid` >/dev/null 2>&1
  pkill inotifywait
  sleep 2
  if [ `ps -ef|grep inotify|grep -v grep|wc -l` -eq 0 ];then
    action "inotify service is stopped" /bin/true
  else
    action "inotify service is stopped" /bin/false
  fi
  ;;
*)
  usage: $0 {start|stop}
  exit 1
esac

启动服务脚本

chmod +x /etc/init.d/inotify
/sbin/chkconfig --add inotify
/sbin/chkconfig inotify on
/sbin/chkconfig --list inotify
#start
/etc/init.d/inotify start

故障排除
说明:主要还是注意细节,文件夹权限,配置文件内容等等;nfs 脚本中添加配置文件时,反向写即可

3 条评论

发表评论

*