主机规划
主机名 |
主机IP |
系统 |
存储 |
GitLabCESvr01 |
10.37.21.21 |
centos 6.5 |
200G |
GitLabCESvr02 |
10.37.21.22 |
centos 6.5 |
200G |
将centos 6.5的内核进行了升级,并重启系统
yum update kernel
yum install kernel-devel
reboot
安装drbd
rpm -Uvh http://www.elrepo.org/elrepo-release-6-6.el6.elrepo.noarch.rpm
yum -y install drbd83-utils kmod-drbd83
加载drbd模块
modprobe drbd
对共享的磁盘分区
[root@GitLabCESvr01 ~] fdisk /dev/xvde
加载磁盘信息:
partprobe /dev/xvde
配置drbd.conf
vi /etc/drbd.conf
#include "drbd.d/global_common.conf";
#include "drbd.d/*.res";
global {
usage-count no;
}
common {
syncer { rate 200M; }
}
resource r0 {
protocol C;
startup {
wfc-timeout 120;
degr-wfc-timeout 120;
}
disk {
on-io-error detach;
}
net{
timeout 60;
connect-int 10;
ping-int 10;
max-buffers 2048;
max-epoch-size 2048;
cram-hmac-alg "sha1";
shared-secret "1234-abcD";
}
on GitLabCESvr01 {
device /dev/drbd0;
disk /dev/xvde5;
address 10.37.21.21:6666;
meta-disk internal;
}
on GitLabCESvr02 {
device /dev/drbd0;
disk /dev/xvde5;
address 10.37.21.22:6666;
meta-disk internal;
}
}
在两台机器上创建drbd元数据信息:
drbdadm create-md all
启动drbd 服务
service drbd start
看状态:
[root@GitLabCESvr01 ~]# drbd-overview
0:r0 Connected Secondary/Secondary Inconsistent/Inconsistent C r-----
[root@GitLabCESvr02 ~]# drbd-overview
0:r0 Connected Secondary/Secondary Inconsistent/Inconsistent C r-----
在主节点GitLabCESvr01上执行下面的命令让其成为主节点
drbdadm -- --overwrite-data-of-peer primary all
看同步状态:
[root@GitLabCESvr01 ~]# drbd-overview
0:r0 SyncSource Primary/Secondary UpToDate/Inconsistent C r-----
[>....................] sync'ed: 0.4% (204132/204788)M
加速同步状态
drbdsetup /dev/drbd/by-res/r0 syncer -r 500M
将/dev/drbd0格式化并挂载
mkfs.ext4 /dev/drbd0
mount /dev/drbd0 /var/opt/gitlab
验证数据同步
1.在主节点:
cd /var/opt/gitlab
echo "test123" > testfile
2.在备份节点GitLabCESvr02上执行下面的命令
mkdir /data
mount /dev/drbd0 /var/opt/gitlab
#mount 会出错,因为mount只能在Primary一端使用
3.在主节点GitLabCESvr01上执行下面的命令变成备份节点
umount /var/opt/gitlab
drbdadm secondary all
4.在备份节点GitLabCESvr02上执行下面的命令变为主节点,可以看到GitLabCESvr01同步过来的内容
drbdadm primary all
mount /dev/drbd0 /var/opt/gitlab
less /data/testfile
可以查看到文件的内容为"test123"
RBD故障处理
场景:
GitLabCESvr01主,GitLabCESvr02备,主设备故障重启,然后查看drbd状态异常
GitLabCESvr01故障后
[root@GitLabCESvr01 ~]# cat /proc/drbd
version: 8.3.8 (api:88/proto:86-94)
srcversion: 299AFE04D7AFD98B3CA0AF9
0: cs:StandAlone ro:Primary/Unknown ds:UpToDate/DUnknown r----
ns:4 nr:102664 dw:102668 dr:157 al:1 bm:8 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
GitLabCESvr02的状态:
[root@GitLabCESvr02 ~]# cat /proc/drbd
version: 8.3.8 (api:88/proto:86-94)
srcversion: 299AFE04D7AFD98B3CA0AF9
0: cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown C r----
ns:0 nr:2141684 dw:2141684 dr:0 al:0 bm:130 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
解决方法:
1.将现在的GitLabCESvr01角色修改为secondary
[root@GitLabCESvr01 ~]# drbdadm secondary r0
[root@GitLabCESvr01 ~]# drbdadm -- --discard-my-data connect r0 #该命令告诉drbd,secondary上的数据不正确,以primary上的数据为准。
2.在GitLabCESvr01上执行下面操作
[root@GitLabCESvr02 /]# drbdadm connect r0
这样GitLabCESvr01就能和GitLabCESvr02开始连接上了,并且保证数据不会丢失:
[root@GitLabCESvr01 ~]# cat /proc/drbd
version: 8.3.8 (api:88/proto:86-94)
srcversion: 299AFE04D7AFD98B3CA0AF9
0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r----
ns:0 nr:20592 dw:20592 dr:0 al:0 bm:4 lo:0 pe:0 ua:0