【Mongo高可用】MongoDB 4.4.12副本集部署实践

MongoDB 4.4.12副本集部署实践
一、主机信息
10.172.132.15
10.172.132.16
10.172.132.17
$ cat /etc/redhat-release
CentOS Linux release 7.8.2003 (Core)
$ cat /proc/cpuinfo |grep flags
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm rep_good nopl xtopology eagerfpu pni cx16 x2apic hypervisor lahf_lm

# echo "never" > /sys/kernel/mm/transparent_hugepage/enabled
# echo "never" > /sys/kernel/mm/transparent_hugepage/defrag

$ df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 2.0G 0 2.0G 0% /dev
tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs 2.0G 190M 1.8G 10% /run
tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
/dev/vda1 200G 2.6G 198G 2% /
/dev/mapper/datavg-datalv01 100G 33M 100G 1% /data
tmpfs 396M 0 396M 0% /run/user/4200

根据系统版本、cpu 确定mongodb版本；根据文件系统规划mongodb的相关目录配置，数据目录、日志目录等

软件包下载：https://www.mongodb.com/download-center/community/releases
本系统安装MongoDB 4.4.12，mongodb-linux-x86_64-rhel70-4.4.12.tgz

二、创建mongodb用户、创建配置文件
$ groupadd -g 27017 mongodb
$ useradd -u 27017 -g 27017 mongodb
$ su - mongodb
$ tar -zxvf mongodb-linux-x86_64-rhel70-4.4.12.tgz
$ chown -R mongodb:mongodb /home/mongodb/
创建mongodb数据目录、日志目录、配置文件目录
$ mkdir -p {log,run,init.d}
$ mkdir -p /data/replica-set
$ mv mongodb-linux-x86_64-rhel70-5.0.6 base
$ ll
total 70672
drwxrwxr-x 3 mongodb mongodb 100 Mar 2 18:17 base
drwxrwxr-x 2 mongodb mongodb 30 Mar 2 18:18 init.d
drwxrwxr-x 2 mongodb mongodb 25 Mar 2 18:30 log
-rw-r--r-- 1 mongodb mongodb 72367249 Mar 2 18:16 mongodb-linux-x86_64-rhel70-4.4.12.tgz
drwxrwxr-x 2 mongodb mongodb 26 Mar 2 18:34 run
创建配置文件：
#10.172.132.15
$ vi /home/mongodb/run/mongodb.conf
dbpath=/data/replica-set
logpath=/home/mongodb/log/mongodb.log
fork=true
logappend=true
bind_ip=10.172.132.15
port=27017
replSet=test-replset

#10.172.132.16
$ vi /home/mongodb/run/mongodb.conf
dbpath=/data/replica-set
logpath=/home/mongodb/log/mongodb.log
fork=true
logappend=true
bind_ip=10.172.132.16
port=27017
replSet=test-replset
#10.172.132.17
$ vi /home/mongodb/run/mongodb.conf
dbpath=/data/replica-set
logpath=/home/mongodb/log/mongodb.log
fork=true
logappend=true
bind_ip=10.172.132.17
port=27017
replSet=test-replset

$ cat init.d/mongodb_start.sh
三、三个节点启动mongodb数据库
编辑脚本
$ vi init.d/mongodb_start.sh #启动mongodb脚本
/home/mongodb/base/bin/mongod --config /home/mongodb/run/mongodb.conf
$ vi init.d/mongodb_stop.sh #停止mongodb脚本
/home/mongodb/base/bin/mongod --shutdown -f /home/mongodb/run/mongodb.conf
$ vi init.d/mongodb_shell.sh #登录mongodb shell脚本
/home/mongodb/base/bin/mongo 10.172.132.15:27017
对三个节点启动mongodb实例
10.172.132.15
10.172.132.16
10.172.132.17
/home/mongodb/base/bin/mongod -config /home/mongodb/run/mongodb.conf

登录节点一10.172.132.15，并初始化副本集
$ ./base/bin/mongo 10.172.132.15:27017

> conf=
{
"_id" : "test-replset",
"members" : [
{ "_id" : 0, "host" : "10.172.132.15:27017" },
{ "_id" : 1, "host" : "10.172.132.16:27017" },
{ "_id" : 2, "host" : "10.172.132.17:27017" }
]
}
> rs.initiate(conf)
{ "ok" : 1 }
test-replset:OTHER> rs.initiate(conf)
{
"operationTime" : Timestamp(1646217736, 1),
"ok" : 0,
"errmsg" : "already initialized",
"code" : 23,
"codeName" : "AlreadyInitialized",
"$clusterTime" : {
"clusterTime" : Timestamp(1646217736, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
test-replset:PRIMARY>

test-replset:PRIMARY> use admin;
test-replset:PRIMARY> db.createUser (
{
user:"admin",
pwd:"xxxxxx",
roles:[{role:"root",db:"admin"}]
}
)
Successfully added user: {
"user" : "admin",
"roles" : [
{
"role" : "root",
"db" : "admin"
}
]
}

test-replset:PRIMARY>use admin;
test-replset:PRIMARY>db.createUser(
{
user: "root",
pwd: "xxxxxx",
roles: [ { role: "root", db: "admin" } ]
}
)
四、根据业务需求创建数据库、创建数据库用户
创建数据库用户

test-replset:PRIMARY> db.createUser(
... {
... user: "srv_test",
... pwd: "xxxxxx",
... roles: [ { role: "readWrite", db: "testdb" } ]
... }
... )
Successfully added user: {
"user" : "srv_test",
"roles" : [
{
"role" : "readWrite",
"db" : "testdb"
}
]
}
test-replset:PRIMARY>
$ ./base/bin/mongo 10.172.132.15:27017/testdb -u srv_test -p xxxxxx
MongoDB shell version v4.4.12
connecting to: mongodb://10.172.132.15:27017/testdb?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("4d619082-236e-4ab0-9acd-beffbc36847a") }
MongoDB server version: 4.4.12
test-replset:PRIMARY>

五、配置初始化参数、并配置keyfile安全认证
#admin帐号连接admin库
./base/bin/mongo -uadmin -p xxxxxx 10.172.132.17:27017/admin

1、创建keyfile文件: 三个节点要用同一份keyfile，在10.172.132.15生成，拷贝到另外两个节点，并修改成 600 的文件属性

openssl rand -base64 745 > /home/mongodb/run/keyfile
chmod 600 /home/mongodb/run/keyfile
2、关闭副本集：分别关闭每个节点的mongod
# mongo 10.172.132.15:27017
replSet:PRIMARY> use admin
replSet:PRIMARY> db.shutdownServer()
3、每个节点的配置文件mongo.conf中的新增内容项：
auth=true
keyFile=/home/mongodb/run/keyfile

4、重新启动副本集
mongod --config /home/mongodb/run/mongodb.conf
业务用户进行验证登录Mongodb shell
mongodb:
/home/mongodb/base/bin/mongo 10.172.132.15:27017/testdb -u srv_test -p xxxxxx
六、问题处理：
1、$ ./mongo --version
Illegal instruction
更换数据库版本4.4.12
https://blog.csdn.net/ActionTech/article/details/122927914
2、mongodb.log错误信息 errmsg":"permissions on /home/mongodb/run/keyfile are too open"

附录：

1、创建数据库的用户角色：
role角色
数据库用户角色：read、readWrite;
数据库管理角色：dbAdmin、dbOwner、userAdmin；
集群管理角色：clusterAdmin、clusterManager、clusterMonitor、hostManager；
备份恢复角色：backup、restore；
所有数据库角色：readAnyDatabase、readWriteAnyDatabase、userAdminAnyDatabase、dbAdminAnyDatabase
超级用户角色：root
内部角色：__system
角色说明
read：允许用户读取指定数据库
readWrite：允许用户读写指定数据库
dbAdmin：允许用户在指定数据库中执行管理函数，如索引创建、删除，查看统计或访问system.profile
userAdmin：允许用户向system.users集合写入，可以找指定数据库里创建、删除和管理用户
clusterAdmin：只在admin数据库中可用，赋予用户所有分片和复制集相关函数的管理权限。
readAnyDatabase：只在admin数据库中可用，赋予用户所有数据库的读权限
readWriteAnyDatabase：只在admin数据库中可用，赋予用户所有数据库的读写权限
userAdminAnyDatabase：只在admin数据库中可用，赋予用户所有数据库的userAdmin权限
dbAdminAnyDatabase：只在admin数据库中可用，赋予用户所有数据库的dbAdmin权限。
root：只在admin数据库中可用。超级账号，超级权限
dbOwner: readWrite + dbAdmin + dbAdmin

2、参数文件
节点1：
dbpath=/data/replica-set
logpath=/home/mongodb/log/mongodb.log
pidfilepath=/home/mongodb/log/mongodb.pid
unixSocketPrefix=/home/mongodb/run
journal=true
fork = true
logappend=true
bind_ip=10.172.132.15
port=27017
directoryperdb=true
auth=true
keyFile=/home/mongodb/run/keyfile
replSet=test-replset
节点2：
dbpath=/data/replica-set
logpath=/home/mongodb/log/mongodb.log
pidfilepath=/home/mongodb/log/mongodb.pid
unixSocketPrefix=/home/mongodb/run
journal=true
fork = true
logappend=true
bind_ip=10.172.132.16
port=27017
directoryperdb=true
auth=true
keyFile=/home/mongodb/run/keyfile
replSet=test-replset

节点3：
dbpath=/data/replica-set
logpath=/home/mongodb/log/mongodb.log
pidfilepath=/home/mongodb/log/mongodb.pid
unixSocketPrefix=/home/mongodb/run
journal=true
fork = true
logappend=true
bind_ip=10.172.132.17
port=27017
directoryperdb=true
auth=true
keyFile=/home/mongodb/run/keyfile
replSet=test-replset