Docker 搭建 etcd 集群
阅读目录:
- 主机安装
- 集群搭建
- API 操作
- API 说明和 etcdctl 命令说明
etcd 是 CoreOS 团队发起的一个开源项目(Go 语言,其实很多这类项目都是 Go 语言实现的,只能说很强大),实现了分布式键值存储和服务发现,etcd 和 ZooKeeper/Consul 非常相似,都提供了类似的功能,以及 REST API 的访问操作,具有以下特点:
- 简单:安装和使用简单,提供了 REST API 进行操作交互
- 安全:支持 HTTPS SSL 证书
- 快速:支持并发 10 k/s 的读写操作
- 可靠:采用 raft 算法,实现分布式系统数据的可用性和一致性
etcd 可以单个实例使用,也可以进行集群配置,因为很多项目都是以 etcd 作为服务发现,比如 CoreOS 和 Kubernetes,所以,下面我们使用 Docker 简单搭建一下 etcd 集群。
1. 主机安装
如果不使用 Docker 的话,etcd 在主机上安装,也非常简单。
Linux 安装命令:
$ curl -L https://github.com/coreos/etcd/releases/download/v3.3.0-rc.0/etcd-v3.3.0-rc.0-linux-amd64.tar.gz -o etcd-v3.3.0-rc.0-linux-amd64.tar.gz &&
sudo tar xzvf etcd-v3.3.0-rc.0-linux-amd64.tar.gz &&
cd etcd-v3.3.0-rc.0-linux-amd64 &&
sudo cp etcd* /usr/local/bin/
其实就是将编译后的二进制文件,拷贝到/usr/local/bin/
目录,各个版本的二进制文件,可以从 https://github.com/coreos/etcd/releases/ 中查找下载。
Mac OS 安装命令:
$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" < /dev/null 2> /dev/null
$ brew install etcd
执行下面命令,查看 etcd 是否安装成功:
$ etcd --version
etcd Version: 3.2.12
Git SHA: GitNotFound
Go Version: go1.9.2
Go OS/Arch: darwin/amd64
2. 集群搭建
搭建 etcd 集群,需要借助下 Docker Machine 创建三个 Docker 主机,命令:
$ docker-machine create -d virtualbox manager1 &&
docker-machine create -d virtualbox worker1 &&
docker-machine create -d virtualbox worker2
$ docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
manager1 - virtualbox Running tcp://192.168.99.100:2376 v17.11.0-ce
worker1 - virtualbox Running tcp://192.168.99.101:2376 v17.11.0-ce
worker2 - virtualbox Running tcp://192.168.99.102:2376 v17.11.0-ce
为防止 Docker 主机中垃取官方镜像,速度慢的问题,我们还需要将 etcd 镜像打包推送到私有仓库中,命令:
$ docker tag quay.io/coreos/etcd 192.168.99.1:5000/quay.io/coreos/etcd:latest &&
docker push 192.168.99.1:5000/quay.io/coreos/etcd:latest &&
docker pull 192.168.99.1:5000/quay.io/coreos/etcd:latest
另外,还需要将私有仓库地址配置在 Docker 主机中,并重启三个 Docker 主机,具体配置参考:Docker 三剑客之 Docker Swarm
Docker 主机配置好之后,我们需要使用docker-machine ssh
命令,分别进入三个 Docker 主机中,执行 Docker etcd 配置命令。
manager1 主机(node1 192.168.99.100
):
$ docker run -d --name etcd \
-p 2379:2379 \
-p 2380:2380 \
--volume=etcd-data:/etcd-data \
192.168.99.1:5000/quay.io/coreos/etcd \
/usr/local/bin/etcd \
--data-dir=/etcd-data --name node1 \
--initial-advertise-peer-urls http://192.168.99.100:2380 --listen-peer-urls http://0.0.0.0:2380 \
--advertise-client-urls http://192.168.99.100:2379 --listen-client-urls http://0.0.0.0:2379 \
--initial-cluster-state new \
--initial-cluster-token docker-etcd \
--initial-cluster node1=http://192.168.99.100:2380,node2=http://192.168.99.101:2380,node3=http://192.168.99.102:2380
worker1 主机(node2 192.168.99.101
):
$ docker run -d --name etcd \
-p 2379:2379 \
-p 2380:2380 \
--volume=etcd-data:/etcd-data \
192.168.99.1:5000/quay.io/coreos/etcd \
/usr/local/bin/etcd \
--data-dir=/etcd-data --name node2 \
--initial-advertise-peer-urls http://192.168.99.101:2380 --listen-peer-urls http://0.0.0.0:2380 \
--advertise-client-urls http://192.168.99.101:2379 --listen-client-urls http://0.0.0.0:2379 \
--initial-cluster-state new \
--initial-cluster-token docker-etcd \
--initial-cluster node1=http://192.168.99.100:2380,node2=http://192.168.99.101:2380,node3=http://192.168.99.102:2380
worker2 主机(node1 192.168.99.102
):
$ docker run -d --name etcd \
-p 2379:2379 \
-p 2380:2380 \
--volume=etcd-data:/etcd-data \
192.168.99.1:5000/quay.io/coreos/etcd \
/usr/local/bin/etcd \
--data-dir=/etcd-data --name node3 \
--initial-advertise-peer-urls http://192.168.99.102:2380 --listen-peer-urls http://0.0.0.0:2380 \
--advertise-client-urls http://192.168.99.102:2379 --listen-client-urls http://0.0.0.0:2379 \
--initial-cluster-state existing \
--initial-cluster-token docker-etcd \
--initial-cluster node1=http://192.168.99.100:2380,node2=http://192.168.99.101:2380,node3=http://192.168.99.102:2380
先来说明下 etcd 各个配置参数的意思(参考自 etcd 使用入门):
--name
:节点名称,默认为 default。--data-dir
:服务运行数据保存的路径,默认为${name}.etcd
。--snapshot-count
:指定有多少事务(transaction)被提交时,触发截取快照保存到磁盘。--heartbeat-interval
:leader 多久发送一次心跳到 followers。默认值是 100ms。--eletion-timeout
:重新投票的超时时间,如果 follow 在该时间间隔没有收到心跳包,会触发重新投票,默认为 1000 ms。--listen-peer-urls
:和同伴通信的地址,比如http://ip:2380
,如果有多个,使用逗号分隔。需要所有节点都能够访问,所以不要使用 localhost!--listen-client-urls
:对外提供服务的地址:比如http://ip:2379,http://127.0.0.1:2379
,客户端会连接到这里和 etcd 交互。--advertise-client-urls
:对外公告的该节点客户端监听地址,这个值会告诉集群中其他节点。--initial-advertise-peer-urls
:该节点同伴监听地址,这个值会告诉集群中其他节点。--initial-cluster
:集群中所有节点的信息,格式为node1=http://ip1:2380,node2=http://ip2:2380,…
,注意:这里的 node1 是节点的 --name 指定的名字;后面的 ip1:2380 是 --initial-advertise-peer-urls 指定的值。--initial-cluster-state
:新建集群的时候,这个值为 new;假如已经存在的集群,这个值为 existing。--initial-cluster-token
:创建集群的 token,这个值每个集群保持唯一。这样的话,如果你要重新创建集群,即使配置和之前一样,也会再次生成新的集群和节点 uuid;否则会导致多个集群之间的冲突,造成未知的错误。
上述配置也可以设置配置文件,默认为/etc/etcd/etcd.conf
。
我们可以使用docker ps
,查看 Docker etcd 是否配置成功:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
463380d23dfe 192.168.99.1:5000/quay.io/coreos/etcd "/usr/local/bin/et..." 2 hours ago Up 2 hours 0.0.0.0:2379-2380->2379-2380/tcp etcd
然后进入其中一个 Docker 主机:
$ docker exec -it etcd bin/sh
执行下面命令(查看集群成员):
$ etcdctl member list
773d30c9fc6640b4: name=node2 peerURLs=http://192.168.99.101:2380 clientURLs=http://192.168.99.101:2379 isLeader=true
b2b0bca2e0cfcc19: name=node3 peerURLs=http://192.168.99.102:2380 clientURLs=http://192.168.99.102:2379 isLeader=false
c88e2cccbb287a01: name=node1 peerURLs=http://192.168.99.100:2380 clientURLs=http://192.168.99.100:2379 isLeader=false
可以看到,集群里面有三个成员,并且node2
为管理员,node1
和node3
为普通成员。
etcdctl 是 ectd 的客户端命令工具(也是 go 语言实现),里面封装了 etcd 的 REST API 执行命令,方便我们进行操作 etcd,后面再列出 etcdctl 的命令详细说明。
上面命令的 etcd API 版本为 2.0,我们可以手动设置版本为 3.0,命令:
$ export ETCDCTL_API=3 && /usr/local/bin/etcdctl put foo bar
OK
部分命令和执行结果还是和 2.0 版本,有很多不同的,比如同是查看集群成员,3.0 版本的执行结果:
$ etcdctl member list
773d30c9fc6640b4, started, node2, http://192.168.99.101:2380, http://192.168.99.101:2379
b2b0bca2e0cfcc19, started, node3, http://192.168.99.102:2380, http://192.168.99.102:2379
c88e2cccbb287a01, started, node1, http://192.168.99.100:2380, http://192.168.99.100:2379
好了,我们现在再演示一种情况,就是从集群中移除一个节点,然后再把它添加到集群中,为演示 etcd 中使用 Raft 算法,我们将node2
管理节点,作为操作对象。
我们在随便一个主机 etcd 容器中(node2
除外),执行成员移除集群命令(必须使用 ID,使用别名会报错):
$ etcdctl member remove 773d30c9fc6640b4
Member 773d30c9fc6640b4 removed from cluster f84185fa5f91bdf6
我们再执行下查看集群成员命令(v2 版本):
$ etcdctl member list
b2b0bca2e0cfcc19: name=node3 peerURLs=http://192.168.99.102:2380 clientURLs=http://192.168.99.102:2379 isLeader=true
c88e2cccbb287a01: name=node1 peerURLs=http://192.168.99.100:2380 clientURLs=http://192.168.99.100:2379 isLeader=false
会发现node2
管理节点被移除集群了,并且通过 Raft 算法,node3
被推举为管理节点。
在将node2
节点重新加入集群之前,我们需要执行下面命令:
$ etcdctl member add node2 --peer-urls="http://192.168.99.101:2380"
Member 22b0de6ffcd98f00 added to cluster f84185fa5f91bdf6
ETCD_NAME="node2"
ETCD_INITIAL_CLUSTER="node2=http://192.168.99.101:2380,node3=http://192.168.99.102:2380,node1=http://192.168.99.100:2380"
ETCD_INITIAL_CLUSTER_STATE="existing"
可以看到,ETCD_INITIAL_CLUSTER_STATE 值为existing
,也就是我们配置的--initial-cluster-state
参数。
我们再执行下查看集群成员命令(v2 版本):
$ etcdctl member list
22b0de6ffcd98f00[unstarted]: peerURLs=http://192.168.99.101:2380
b2b0bca2e0cfcc19: name=node3 peerURLs=http://192.168.99.102:2380 clientURLs=http://192.168.99.102:2379 isLeader=true
c88e2cccbb287a01: name=node1 peerURLs=http://192.168.99.100:2380 clientURLs=http://192.168.99.100:2379 isLeader=false
会发现22b0de6ffcd98f00
成员状态变为了unstarted
。
我们在node2
节点,执行 Docker etcd 集群配置命令:
$ docker run -d --name etcd \
-p 2379:2379 \
-p 2380:2380 \
--volume=etcd-data:/etcd-data \
192.168.99.1:5000/quay.io/coreos/etcd \
/usr/local/bin/etcd \
--data-dir=/etcd-data --name node2 \
--initial-advertise-peer-urls http://192.168.99.101:2380 --listen-peer-urls http://0.0.0.0:2380 \
--advertise-client-urls http://192.168.99.101:2379 --listen-client-urls http://0.0.0.0:2379 \
--initial-cluster-state existing \
--initial-cluster-token docker-etcd \
--initial-cluster node1=http://192.168.99.100:2380,node2=http://192.168.99.101:2380,node3=http://192.168.99.102:2380
结果并不像我们想要的那样成功,执行查看日志:
$ docker logs etcd
2017-12-25 08:19:30.160967 I | etcdmain: etcd Version: 3.2.12
2017-12-25 08:19:30.161062 I | etcdmain: Git SHA: b19dae0
2017-12-25 08:19:30.161082 I | etcdmain: Go Version: go1.8.5
2017-12-25 08:19:30.161092 I | etcdmain: Go OS/Arch: linux/amd64
2017-12-25 08:19:30.161105 I | etcdmain: setting maximum number of CPUs to 1, total number of available CPUs is 1
2017-12-25 08:19:30.161144 N | etcdmain: the server is already initialized as member before, starting as etcd member...
2017-12-25 08:19:30.161195 I | embed: listening for peers on http://0.0.0.0:2380
2017-12-25 08:19:30.161232 I | embed: listening for client requests on 0.0.0.0:2379
2017-12-25 08:19:30.165269 I | etcdserver: name = node2
2017-12-25 08:19:30.165317 I | etcdserver: data dir = /etcd-data
2017-12-25 08:19:30.165335 I | etcdserver: member dir = /etcd-data/member
2017-12-25 08:19:30.165347 I | etcdserver: heartbeat = 100ms
2017-12-25 08:19:30.165358 I | etcdserver: election = 1000ms
2017-12-25 08:19:30.165369 I | etcdserver: snapshot count = 100000
2017-12-25 08:19:30.165385 I | etcdserver: advertise client URLs = http://192.168.99.101:2379
2017-12-25 08:19:30.165593 I | etcdserver: restarting member 773d30c9fc6640b4 in cluster f84185fa5f91bdf6 at commit index 14
2017-12-25 08:19:30.165627 I | raft: 773d30c9fc6640b4 became follower at term 11
2017-12-25 08:19:30.165647 I | raft: newRaft 773d30c9fc6640b4 [peers: [], term: 11, commit: 14, applied: 0, lastindex: 14, lastterm: 11]
2017-12-25 08:19:30.169277 W | auth: simple token is not cryptographically signed
2017-12-25 08:19:30.170424 I | etcdserver: starting server... [version: 3.2.12, cluster version: to_be_decided]
2017-12-25 08:19:30.171732 I | etcdserver/membership: added member 773d30c9fc6640b4 [http://192.168.99.101:2380] to cluster f84185fa5f91bdf6
2017-12-25 08:19:30.171845 I | etcdserver/membership: added member c88e2cccbb287a01 [http://192.168.99.100:2380] to cluster f84185fa5f91bdf6
2017-12-25 08:19:30.171877 I | rafthttp: starting peer c88e2cccbb287a01...
2017-12-25 08:19:30.171902 I | rafthttp: started HTTP pipelining with peer c88e2cccbb287a01
2017-12-25 08:19:30.175264 I | rafthttp: started peer c88e2cccbb287a01
2017-12-25 08:19:30.175339 I | rafthttp: added peer c88e2cccbb287a01
2017-12-25 08:19:30.178326 I | etcdserver/membership: added member cbd7fa8d01297113 [http://192.168.99.102:2380] to cluster f84185fa5f91bdf6
2017-12-25 08:19:30.178383 I | rafthttp: starting peer cbd7fa8d01297113...
2017-12-25 08:19:30.178410 I | rafthttp: started HTTP pipelining with peer cbd7fa8d01297113
2017-12-25 08:19:30.179794 I | rafthttp: started peer cbd7fa8d01297113
2017-12-25 08:19:30.179835 I | rafthttp: added peer cbd7fa8d01297113
2017-12-25 08:19:30.180062 N | etcdserver/membership: set the initial cluster version to 3.0
2017-12-25 08:19:30.180132 I | etcdserver/api: enabled capabilities for version 3.0
2017-12-25 08:19:30.180255 N | etcdserver/membership: updated the cluster version from 3.0 to 3.2
2017-12-25 08:19:30.180430 I | etcdserver/api: enabled capabilities for version 3.2
2017-12-25 08:19:30.183979 I | rafthttp: started streaming with peer c88e2cccbb287a01 (writer)
2017-12-25 08:19:30.184139 I | rafthttp: started streaming with peer c88e2cccbb287a01 (writer)
2017-12-25 08:19:30.184232 I | rafthttp: started streaming with peer c88e2cccbb287a01 (stream MsgApp v2 reader)
2017-12-25 08:19:30.185142 I | rafthttp: started streaming with peer c88e2cccbb287a01 (stream Message reader)
2017-12-25 08:19:30.186518 I | etcdserver/membership: removed member cbd7fa8d01297113 from cluster f84185fa5f91bdf6
2017-12-25 08:19:30.186573 I | rafthttp: stopping peer cbd7fa8d01297113...
2017-12-25 08:19:30.186614 I | rafthttp: started streaming with peer cbd7fa8d01297113 (writer)
2017-12-25 08:19:30.186786 I | rafthttp: stopped streaming with peer cbd7fa8d01297113 (writer)
2017-12-25 08:19:30.186815 I | rafthttp: started streaming with peer cbd7fa8d01297113 (writer)
2017-12-25 08:19:30.186831 I | rafthttp: stopped streaming with peer cbd7fa8d01297113 (writer)
2017-12-25 08:19:30.186876 I | rafthttp: started streaming with peer cbd7fa8d01297113 (stream MsgApp v2 reader)
2017-12-25 08:19:30.187224 I | rafthttp: started streaming with peer cbd7fa8d01297113 (stream Message reader)
2017-12-25 08:19:30.187647 I | rafthttp: stopped HTTP pipelining with peer cbd7fa8d01297113
2017-12-25 08:19:30.187682 I | rafthttp: stopped streaming with peer cbd7fa8d01297113 (stream MsgApp v2 reader)
2017-12-25 08:19:30.187873 I | rafthttp: stopped streaming with peer cbd7fa8d01297113 (stream Message reader)
2017-12-25 08:19:30.187895 I | rafthttp: stopped peer cbd7fa8d01297113
2017-12-25 08:19:30.187911 I | rafthttp: removed peer cbd7fa8d01297113
2017-12-25 08:19:30.188034 I | etcdserver/membership: added member b2b0bca2e0cfcc19 [http://192.168.99.102:2380] to cluster f84185fa5f91bdf6
2017-12-25 08:19:30.188059 I | rafthttp: starting peer b2b0bca2e0cfcc19...
2017-12-25 08:19:30.188075 I | rafthttp: started HTTP pipelining with peer b2b0bca2e0cfcc19
2017-12-25 08:19:30.188510 I | rafthttp: started peer b2b0bca2e0cfcc19
2017-12-25 08:19:30.188533 I | rafthttp: added peer b2b0bca2e0cfcc19
2017-12-25 08:19:30.188795 I | etcdserver/membership: removed member 773d30c9fc6640b4 from cluster f84185fa5f91bdf6
2017-12-25 08:19:30.193643 I | rafthttp: started streaming with peer b2b0bca2e0cfcc19 (writer)
2017-12-25 08:19:30.193730 I | rafthttp: started streaming with peer b2b0bca2e0cfcc19 (writer)
2017-12-25 08:19:30.193797 I | rafthttp: started streaming with peer b2b0bca2e0cfcc19 (stream MsgApp v2 reader)
2017-12-25 08:19:30.194782 I | rafthttp: started streaming with peer b2b0bca2e0cfcc19 (stream Message reader)
2017-12-25 08:19:30.195663 I | raft: 773d30c9fc6640b4 [term: 11] received a MsgHeartbeat message with higher term from b2b0bca2e0cfcc19 [term: 12]
2017-12-25 08:19:30.195716 I | raft: 773d30c9fc6640b4 became follower at term 12
2017-12-25 08:19:30.195736 I | raft: raft.node: 773d30c9fc6640b4 elected leader b2b0bca2e0cfcc19 at term 12
2017-12-25 08:19:30.196617 E | rafthttp: streaming request ignored (ID mismatch got 22b0de6ffcd98f00 want 773d30c9fc6640b4)
2017-12-25 08:19:30.197064 E | rafthttp: streaming request ignored (ID mismatch got 22b0de6ffcd98f00 want 773d30c9fc6640b4)
2017-12-25 08:19:30.197846 E | rafthttp: streaming request ignored (ID mismatch got 22b0de6ffcd98f00 want 773d30c9fc6640b4)
2017-12-25 08:19:30.198242 E | rafthttp: streaming request ignored (ID mismatch got 22b0de6ffcd98f00 want 773d30c9fc6640b4)
2017-12-25 08:19:30.201771 E | etcdserver: the member has been permanently removed from the cluster
2017-12-25 08:19:30.202060 I | etcdserver: the data-dir used by this member must be removed.
2017-12-25 08:19:30.202307 E | etcdserver: publish error: etcdserver: request cancelled
2017-12-25 08:19:30.202338 I | etcdserver: aborting publish because server is stopped
2017-12-25 08:19:30.202364 I | rafthttp: stopping peer b2b0bca2e0cfcc19...
2017-12-25 08:19:30.202482 I | rafthttp: stopped streaming with peer b2b0bca2e0cfcc19 (writer)
2017-12-25 08:19:30.202504 I | rafthttp: stopped streaming with peer b2b0bca2e0cfcc19 (writer)
2017-12-25 08:19:30.204143 I | rafthttp: stopped HTTP pipelining with peer b2b0bca2e0cfcc19
2017-12-25 08:19:30.204186 I | rafthttp: stopped streaming with peer b2b0bca2e0cfcc19 (stream MsgApp v2 reader)
2017-12-25 08:19:30.204205 I | rafthttp: stopped streaming with peer b2b0bca2e0cfcc19 (stream Message reader)
2017-12-25 08:19:30.204217 I | rafthttp: stopped peer b2b0bca2e0cfcc19
2017-12-25 08:19:30.204228 I | rafthttp: stopping peer c88e2cccbb287a01...
2017-12-25 08:19:30.204241 I | rafthttp: stopped streaming with peer c88e2cccbb287a01 (writer)
2017-12-25 08:19:30.204255 I | rafthttp: stopped streaming with peer c88e2cccbb287a01 (writer)
2017-12-25 08:19:30.204824 I | rafthttp: stopped HTTP pipelining with peer c88e2cccbb287a01
2017-12-25 08:19:30.204860 I | rafthttp: stopped streaming with peer c88e2cccbb287a01 (stream MsgApp v2 reader)
2017-12-25 08:19:30.204878 I | rafthttp: stopped streaming with peer c88e2cccbb287a01 (stream Message reader)
2017-12-25 08:19:30.204891 I | rafthttp: stopped peer c88e2cccbb287a01
这么长的日志,说明啥问题呢,就是说我们虽然重新执行的 etcd 创建命令,但因为读取之前配置文件的关系,etcd 会恢复之前的集群成员,但之前的集群节点已经被移除了,所以集群节点就一直处于停止状态。
怎么解决呢?很简单,就是将我们之前创建的etcd-data
数据卷轴删掉,命令:
$ docker volume ls
DRIVER VOLUME NAME
local etcd-data
$ docker volume rm etcd-data
etcd-data
然后,再在node2
节点,重新执行 Docker etcd 集群配置命令(上面),会发现执行是成功的。
我们再执行下查看集群成员命令(v2 版本):
$ etcdctl member list
22b0de6ffcd98f00: name=node2 peerURLs=http://192.168.99.101:2380 clientURLs=http://192.168.99.101:2379 isLeader=false
b2b0bca2e0cfcc19: name=node3 peerURLs=http://192.168.99.102:2380 clientURLs=http://192.168.99.102:2379 isLeader=true
c88e2cccbb287a01: name=node1 peerURLs=http://192.168.99.100:2380 clientURLs=http://192.168.99.100:2379 isLeader=false
3. API 操作
etcd REST API 被用于键值操作和集群成员操作,这边就简单说几个,详细的 API 查看附录说明。
1. 键值管理
设置键值命令:
$ curl http://127.0.0.1:2379/v2/keys/hello -XPUT -d value="hello world"
{"action":"set","node":{"key":"/hello","value":"hello world","modifiedIndex":17,"createdIndex":17}}
查看键值命令:
$ curl http://127.0.0.1:2379/v2/keys/hello
{"action":"get","node":{"key":"/hello","value":"hello world","modifiedIndex":17,"createdIndex":17}}
删除键值命令:
$ curl http://127.0.0.1:2379/v2/keys/hello -XDELETE
{"action":"delete","node":{"key":"/hello","modifiedIndex":19,"createdIndex":17},"prevNode":{"key":"/hello","value":"hello world","modifiedIndex":17,"createdIndex":17}}
2. 成员管理
列出集群中的所有成员:
$ curl http://127.0.0.1:2379/v2/members
{"members":[{"id":"22b0de6ffcd98f00","name":"node2","peerURLs":["http://192.168.99.101:2380"],"clientURLs":["http://192.168.99.101:2379"]},{"id":"b2b0bca2e0cfcc19","name":"node3","peerURLs":["http://192.168.99.102:2380"],"clientURLs":["http://192.168.99.102:2379"]},{"id":"c88e2cccbb287a01","name":"node1","peerURLs":["http://192.168.99.100:2380"],"clientURLs":["http://192.168.99.100:2379"]}]}
查看当前节点是否为管理节点:
$ curl http://127.0.0.1:2379/v2/stats/leader
{"leader":"b2b0bca2e0cfcc19","followers":{"22b0de6ffcd98f00":{"latency":{"current":0.001051,"average":0.0029195000000000002,"standardDeviation":0.001646769458667484,"minimum":0.001051,"maximum":0.006367},"counts":{"fail":0,"success":10}},"c88e2cccbb287a01":{"latency":{"current":0.000868,"average":0.0022389999999999997,"standardDeviation":0.0011402923601720172,"minimum":0.000868,"maximum":0.004725},"counts":{"fail":0,"success":12}}}}
查看当前节点信息:
$ curl http://127.0.0.1:2379/v2/stats/self
{"name":"node3","id":"b2b0bca2e0cfcc19","state":"StateLeader","startTime":"2017-12-25T06:00:28.803429523Z","leaderInfo":{"leader":"b2b0bca2e0cfcc19","uptime":"36m45.45263851s","startTime":"2017-12-25T08:13:02.103896843Z"},"recvAppendRequestCnt":6,"sendAppendRequestCnt":22}
查看集群状态:
$ curl http://127.0.0.1:2379/v2/stats/store
{"getsSuccess":9,"getsFail":4,"setsSuccess":9,"setsFail":0,"deleteSuccess":3,"deleteFail":0,"updateSuccess":0,"updateFail":0,"createSuccess":7,"createFail":0,"compareAndSwapSuccess":0,"compareAndSwapFail":0,"compareAndDeleteSuccess":0,"compareAndDeleteFail":0,"expireCount":0,"watchers":0}
当然也可以通过 API 添加和删除集群成员。
4. API 说明和 etcdctl 命令说明
etcd REST API 说明(v2 版本):
命令 | 说明 |
---|---|
curl -L http://127.0.0.1:2379/version | 查看版本 |
curl http://127.0.0.1:2379/v2/keys/message -XPUT -d value="Hello world" | 添加键值 |
curl http://127.0.0.1:2379/v2/keys/message | 获取键值 |
curl http://127.0.0.1:2379/v2/keys/message -XPUT -d value="Hello etcd" | 更新键值 |
curl http://127.0.0.1:2379/v2/keys/message -XDELETE | 删除键值 |
curl http://127.0.0.1:2379/v2/keys/foo -XPUT -d value=bar -d ttl=5 | 添加 TTL 键值(过期时间) |
curl http://127.0.0.1:2379/v2/keys/foo | 获取 TTL 键值 |
curl http://127.0.0.1:2379/v2/keys/foo -XPUT -d value=bar -d ttl= -d prevExist=true | 更新 TTL 键值 |
curl http://127.0.0.1:2379/v2/keys/foo?wait=true | |
curl http://127.0.0.1:2379/v2/keys/foo -XPUT -d value=bar | |
curl 'http://127.0.0.1:2379/v2/keys/foo?wait=true&waitIndex=7' | |
curl http://127.0.0.1:2379/v2/keys/queue -XPOST -d value=Job1 | |
curl -s 'http://127.0.0.1:2379/v2/keys/queue?recursive=true&sorted=true' | |
curl http://127.0.0.1:2379/v2/keys/dir -XPUT -d ttl=30 -d dir=true | |
curl http://127.0.0.1:2379/v2/keys/dir -XPUT -d ttl=30 -d dir=true -d prevExist=true | |
curl 'http://127.0.0.1:2379/v2/keys/dir?wait=true' | |
curl http://127.0.0.1:2379/v2/keys/dir -XPUT -d dir=true | 创建目录 |
curl http://127.0.0.1:2379/v2/keys/foo_dir/foo -XPUT -d value=bar | 在目录下添加键值 |
curl http://127.0.0.1:2379/v2/keys/?recursive=true | 获取目录下的键值 |
curl 'http://127.0.0.1:2379/v2/keys/foo_dir?dir=true' -XDELETE | 删除目录 |
curl http://10.0.0.10:2379/v2/members | 查看集群成员 |
curl http://10.0.0.10:2379/v2/members -XPOST -H "Content-Type: application/json" -d '{"peerURLs":["http://10.0.0.10:2380"]}' | 添加集群成员 |
curl http://10.0.0.10:2379/v2/members/272e204152 -XDELETE | 删除集群成员 |
curl http://10.0.0.10:2379/v2/members/272e204152 -XPUT -H "Content-Type: application/json" -d '{"peerURLs":["http://10.0.0.10:2380"]}' | 更新集群成员 |
更多 API 请查看:etcd API 和 Members API
etcdctl 命令说明:
命令 | 说明 |
---|---|
etcdctl set key value | 添加键值 |
etcdctl get key | 获取键值 |
etcdctl update key value | 更新键值 |
etcdctl rm key | 删除键值 |
etcdctl mkdir dirname | 添加目录(不存在的话创建) |
etcdctl setdir | 添加目录(都创建) |
etcdctl updatedir | 更新目录 |
etcdctl rmdir | 删除目录 |
etcdctl ls | 列出目录 |
etcdctl watch | 监控键值 |
etcdctl exec-watch | 监控键值(执行命令) |
etcdctl list | 查看集群成员 |
etcdctl member add | 添加集群成员 |
etcdctl remove | 移除集群成员 |
参考资料:
- Install etcd on Mac with Brew
- linux 基于 docker 部署 etcd 集群
- Docker 搭建 etcd 集群及管理
- DOCKER 服务发现 - ETCD 集群
- docker-machine 使用 discovery 模式搭建 etcd 集群
- 在容器内运行 etcd 集群
- etcd rest api 基本操作
- etcd 使用入门(推荐)
- 创建高可用 etcd 集群
- 初试 ETCD(推荐)
- Run etcd clusters inside containers(官方)