• 欢迎访问蜷缩的蜗牛博客 蜷缩的蜗牛
  • 微信搜索: 蜷缩的蜗牛 | 联系站长 kbsonlong@qq.com
  • 如果您觉得本站非常有看点,那么赶紧使用Ctrl+D 收藏吧

etcd 集群管理维护

ELK 蜷缩的蜗牛 8个月前 (01-16) 58次浏览 已收录

官方网站:

https://github.com/coreos/etcd/

环境:

CentOS7

etcd-3.0.4

3 节点集群示例

etcd1:192.168.8.101

etcd2:192.168.8.102

etcd3:192.168.8.103

一.安装 etcd(所有节点)

curl -L https://github.com/coreos/etcd/releases/download/v3.0.4/etcd-v3.0.4-linux-amd64.tar.gz -o etcd-v3.0.4-linux-amd64.tar.gz

tar xzvf etcd-v3.0.4-linux-amd64.tar.gz

cp -af etcd-v3.0.4-linux-amd64/{etcd,etcdctl} /usr/local/bin

chmod +x /usr/local/bin/{etcd,etcdctl}

二.配置 etcd 集群

https://github.com/coreos/etcd/blob/master/Documentation/op-guide/clustering.md

cluster 帮助文档 etcd-v3.0.4-linux-amd64/Documentation/op-guide/clustering.md

This guide will cover the following mechanisms for bootstrapping an etcd cluster:

* [Static](#static)

* [etcd Discovery](#etcd-discovery)

* [DNS Discovery](#dns-discovery)

目前支持三种发现方式,Static 适用于有固定 IP 的主机节点,etcd Discovery 适用于 DHCP 环境,DNS Discovery 依赖 DNS SRV 记录

Static 方式

提示:etcd 支持 ssl/tls,详见官方文档

https://github.com/coreos/etcd/blob/master/Documentation/op-guide/security.md

节点一:etcd1:192.168.8.101

etcd –name etcd1 –data-dir /opt/etcd \

–initial-advertise-peer-urls http://192.168.8.101:2380 \

–listen-peer-urls http://192.168.8.101:2380 \

–listen-client-urls http://192.168.8.101:2379,http://127.0.0.1:2379 \

–advertise-client-urls http://192.168.8.101:2379 \

–initial-cluster-token etcd-cluster-1 \

–initial-cluster etcd1=http://192.168.8.101:2380,etcd2=http://192.168.8.102:2380,etcd3=http://192.168.8.103:2380 \

–initial-cluster-state new

节点二:etcd2:192.168.8.102

etcd –name etcd2 –data-dir /opt/etcd \

–initial-advertise-peer-urls http://192.168.8.102:2380 \

–listen-peer-urls http://192.168.8.102:2380 \

–listen-client-urls http://192.168.8.102:2379,http://127.0.0.1:2379 \

–advertise-client-urls http://192.168.8.102:2379 \

–initial-cluster-token etcd-cluster-1 \

–initial-cluster etcd1=http://192.168.8.101:2380,etcd2=http://192.168.8.102:2380,etcd3=http://192.168.8.103:2380 \

–initial-cluster-state new

节点三:etcd1:192.168.8.103

etcd –name etcd3 –data-dir /opt/etcd \

–initial-advertise-peer-urls http://192.168.8.103:2380 \

–listen-peer-urls http://192.168.8.103:2380 \

–listen-client-urls http://192.168.8.103:2379,http://127.0.0.1:2379 \

–advertise-client-urls http://192.168.8.103:2379 \

–initial-cluster-token etcd-cluster-1 \

–initial-cluster etcd1=http://192.168.8.101:2380,etcd2=http://192.168.8.102:2380,etcd3=http://192.168.8.103:2380 \

–initial-cluster-state new

2379 是用于监听客户端请求,2380 用于集群通信,可以通过–data-dir 指定数据存放目录,不指定则默认为当前工作目录

[root@node3 ~]# netstat -tunlp|grep etcd

tcp        0      0 192.168.8.103:2379      0.0.0.0:*               LISTEN      11103/etcd

tcp        0      0 127.0.0.1:2379          0.0.0.0:*               LISTEN      11103/etcd

tcp        0      0 192.168.8.103:2380      0.0.0.0:*               LISTEN      11103/etcd

[root@node3 ~]# ls

etcd3.etcd

[root@node3 ~]# ls etcd3.etcd/

fixtures/ member/

[root@node3 ~]# ls etcd3.etcd/fixtures/

client/ peer/

[root@node3 ~]# ls etcd3.etcd/fixtures/peer/

cert.pem  key.pem

注意:上面的初始化只是在集群初始化时运行一次,之后服务有重启,必须要去除掉 initial 参数,否则报错

请使用如下类似命令

etcd –name etcd3   –data-dir /opt/etcd \

–listen-peer-urls http://192.168.8.103:2380 \

–listen-client-urls http://192.168.8.103:2379,http://127.0.0.1:2379 \

–advertise-client-urls http://192.168.8.103:2379

三.管理集群

etcdctl

https://github.com/coreos/etcd/blob/master/Documentation/op-guide/maintenance.md

[root@node3 ~]# etcdctl –version

etcdctl version: 3.0.4

API version: 2

COMMANDS:

backup          backup an etcd directory

cluster-health  check the health of the etcd cluster

mk              make a new key with a given value

mkdir           make a new directory

rm              remove a key or a directory

rmdir           removes the key if it is an empty directory or a key-value pair

get             retrieve the value of a key

ls              retrieve a directory

set             set the value of a key

setdir          create a new directory or update an existing directory TTL

update          update an existing key with a given value

updatedir       update an existing directory

watch           watch a key for changes

exec-watch      watch a key for changes and exec an executable

member          member add, remove and list subcommands

import          import a snapshot to a cluster

user            user add, grant and revoke subcommands

role            role add, grant and revoke subcommands

auth            overall auth controls

集群健康状态

[root@node3 ~]# etcdctl cluster-health

member 2947dd07df9e44da is healthy: got healthy result from http://192.168.8.102:2379

member 571bf93ce7760601 is healthy: got healthy result from http://192.168.8.101:2379

member b200a8bec19bd22e is healthy: got healthy result from http://192.168.8.103:2379

cluster is healthy

集群成员查看

[root@node3 ~]# etcdctl member list

2947dd07df9e44da: name=etcd2 peerURLs=http://192.168.8.102:2380 clientURLs=http://192.168.8.102:2379 isLeader=false

571bf93ce7760601: name=etcd1 peerURLs=http://192.168.8.101:2380 clientURLs=http://192.168.8.101:2379 isLeader=true

b200a8bec19bd22e: name=etcd3 peerURLs=http://192.168.8.103:2380 clientURLs=http://192.168.8.103:2379 isLeader=false

删除集群成员

[root@node2 ~]# etcdctl member remove b200a8bec19bd22e

Removed member 4d11141f72b2744c from cluster

[root@node2 ~]# etcdctl member list

2947dd07df9e44da: name=etcd2 peerURLs=http://192.168.8.102:2380 clientURLs=http://192.168.8.102:2379 isLeader=false

571bf93ce7760601: name=etcd1 peerURLs=http://192.168.8.101:2380 clientURLs=http://192.168.8.101:2379 isLeader=true

添加集群成员

https://github.com/coreos/etcd/blob/master/Documentation/op-guide/runtime-configuration.md

注意:步骤很重要,不然会报集群 ID 不匹配

[root@node2 ~]# etcdctl member add –help

NAME:

etcdctl member add – add a new member to the etcd cluster

USAGE:

etcdctl member add

1.将目标节点添加到集群

[root@node2 ~]# etcdctl member add etcd3 http://192.168.8.103:2380

Added member named etcd3 with ID 28e0d98e7ec15cd4 to cluster

ETCD_NAME=”etcd3″

ETCD_INITIAL_CLUSTER=”etcd3=http://192.168.8.103:2380,etcd2=http://192.168.8.102:2380,etcd1=http://192.168.8.101:2380″

ETCD_INITIAL_CLUSTER_STATE=”existing”

[root@node2 ~]# etcdctl member list

2947dd07df9e44da: name=etcd2 peerURLs=http://192.168.8.102:2380 clientURLs=http://192.168.8.102:2379 isLeader=false

571bf93ce7760601: name=etcd1 peerURLs=http://192.168.8.101:2380 clientURLs=http://192.168.8.101:2379 isLeader=true

d4f257d2b5f99b64[unstarted]: peerURLs=http://192.168.8.103:2380

此时,集群会为目标节点生成一个唯一的 member ID

2.清空目标节点的 data-dir

[root@node3 ~]#rm -rf /opt/etcd

注意:节点删除后,集群中的成员信息会更新,新节点加入集群是作为一个全新的节点加入,如果 data-dir 有数据,etcd 启动时会读取己经存在的数据,启动时仍然用的老 member ID,也会造成,集群不无法加入,所以一定要清空新节点的 data-dir

2016-08-12 01:59:41.084928 E | rafthttp: failed to find member 2947dd07df9e44da in cluster ce2f2517679629de

2016-08-12 01:59:41.133698 W | rafthttp: failed to process raft message (raft: stopped)

2016-08-12 01:59:41.135746 W | rafthttp: failed to process raft message (raft: stopped)

2016-08-12 01:59:41.170915 E | rafthttp: failed to find member 2947dd07df9e44da in cluster ce2f2517679629de

3.在目标节点上启动 etcd

etcd –name etcd3 –data-dir /opt/etcd \

–initial-advertise-peer-urls http://192.168.8.103:2380 \

–listen-peer-urls http://192.168.8.103:2380 \

–listen-client-urls http://192.168.8.103:2379,http://127.0.0.1:2379 \

–advertise-client-urls http://192.168.8.103:2379 \

–initial-cluster-token etcd-cluster-1 \

–initial-cluster etcd1=http://192.168.8.101:2380,etcd2=http://192.168.8.102:2380,etcd3=http://192.168.8.103:2380 \

–initial-cluster-state existing

注意: 这里的 initial 标记一定要指定为 existing,如果为 new 则会自动生成一个新的 member ID,和前面添加节点时生成的 ID 不一致,故日志中会报节点 ID 不匹配的错

[root@node2 ~]# etcdctl member list

28e0d98e7ec15cd4: name=etcd3 peerURLs=http://192.168.8.103:2380 clientURLs=http://192.168.8.103:2379 isLeader=false

2947dd07df9e44da: name=etcd2 peerURLs=http://192.168.8.102:2380 clientURLs=http://192.168.8.102:2379 isLeader=false

571bf93ce7760601: name=etcd1 peerURLs=http://192.168.8.101:2380 clientURLs=http://192.168.8.101:2379 isLeader=true

删改查

[root@node3 ~]# etcdctl set foo “bar”

bar

[root@node3 ~]# etcdctl get foo

bar

[root@node3 ~]# etcdctl mkdir hello

[root@node3 ~]# etcdctl ls

/foo

/hello

[root@node3 ~]# etcdctl –output extended get foo

Key: /foo

Created-Index: 9

Modified-Index: 9

TTL: 0

Index: 10

bar

[root@node3 ~]# etcdctl –output json get foo

{“action”:”get”,”node”:{“key”:”/foo”,”value”:”bar”,”nodes”:null,”createdIndex”:9,”modifiedIndex”:9},”prevNode”:null}

[root@node2 ~]# etcdctl update foo “etcd cluster is ok”

etcd cluster is ok

[root@node2 ~]# etcdctl get foo

etcd cluster is ok

[root@node3 ~]# etcdctl import –snap /opt/etcd/member/snap/db

starting to import snapshot /opt/etcd/member/snap/db with 10 clients

2016-08-12 01:18:17.281921 I | entering dir: /

finished importing 0 keys

REST API

https://github.com/coreos/etcd/tree/master/Documentation/learning

[root@node1 ~]# curl 192.168.8.101:2379/v2/keys

{“action”:”get”,”node”:{“dir”:true,”nodes”:[{“key”:”/foo”,”value”:”etcd cluster is ok”,”modifiedIndex”:28,”createdIndex”:9},{“key”:”/hello”,”dir”:true,”modifiedIndex”:10,”createdIndex”:10},{“key”:”/registry”,”dir”:true,”modifiedIndex”:47,”createdIndex”:47}]}}

[root@node1 ~]# curl -fs -X PUT 192.168.8.101:2379/v2/keys/_test

{“action”:”set”,”node”:{“key”:”/_test”,”value”:””,”modifiedIndex”:1439,”createdIndex”:1439}}

[root@node1 ~]# curl -X GET 192.168.8.101:2379/v2/keys/_test

{“action”:”get”,”node”:{“key”:”/_test”,”value”:””,”modifiedIndex”:1439,”createdIndex”:1439}}

四.systemd 管控

1.建用户 etcd

useradd -r -s /sbin/nologin etcd

chown -R etcd: /opt/etcd

2.创建 systemd 服务脚本 etcd.service

cat >/lib/systemd/system/etcd.service <<HERE

[Unit]

Description=Etcd Server

After=network.target

After=network-online.target

Wants=network-online.target

[Service]

Type=notify

WorkingDirectory=/opt/etcd/

User=etcd

ExecStart=/usr/local/bin/etcd –config-file /etc/etcd.conf

Restart=on-failure

LimitNOFILE=1000000

[Install]

WantedBy=multi-user.target

HERE

3.创建主配置文件 etcd.conf

cat >/etc/etcd.conf <<HERE

name: etcd2

data-dir: “/opt/etcd”

listen-peer-urls: “http://192.168.8.102:2380&#8221;

listen-client-urls: “http://192.168.8.102:2379,http://127.0.0.1:2379&#8221;

advertise-client-urls: “http://192.168.8.102:2379&#8221;

HERE

提示:etcd-3.x 版本支持yamljson两种配置文件格式,配置模板见https://github.com/coreos/etcd/blob/master/etcd.conf.yml.sample

不同节点的配置文件不同,如上是 etcd2 的范本

4.测试 systemd 启动

[root@node2 ~]# systemctl enable etcd

Created symlink from /etc/systemd/system/multi-user.target.wants/etcd.service to /usr/lib/systemd/system/etcd.service.

[root@node2 ~]# systemctl start etcd

[root@node2 ~]# systemctl status etcd

etcd.service – Etcd Server

Loaded: loaded (/usr/lib/systemd/system/etcd.service; enabled; vendor preset: disabled)

Active: active (running) since 五 2016-08-12 03:06:30 CST; 8min ago

Main PID: 12099 (etcd)

CGroup: /system.slice/etcd.service

└─12099 /usr/local/bin/etcd –config-file /etc/etcd.conf

8 月 12 03:10:30 node2.example.com etcd[12099]: the clock difference against peer 571bf93ce776…1s]

8 月 12 03:11:00 node2.example.com etcd[12099]: the clock difference against peer 571bf93ce776…1s]

8 月 12 03:11:30 node2.example.com etcd[12099]: the clock difference against peer 571bf93ce776…1s]

8 月 12 03:12:00 node2.example.com etcd[12099]: the clock difference against peer 571bf93ce776…1s]

8 月 12 03:12:30 node2.example.com etcd[12099]: the clock difference against peer 571bf93ce776…1s]

8 月 12 03:13:00 node2.example.com etcd[12099]: the clock difference against peer 571bf93ce776…1s]

8 月 12 03:13:30 node2.example.com etcd[12099]: the clock difference against peer 571bf93ce776…1s]

8 月 12 03:14:00 node2.example.com etcd[12099]: the clock difference against peer 571bf93ce776…1s]

8 月 12 03:14:30 node2.example.com etcd[12099]: the clock difference against peer 571bf93ce776…1s]

8 月 12 03:15:00 node2.example.com etcd[12099]: the clock difference against peer 571bf93ce776…1s]

Hint: Some lines were ellipsized, use -l to show in full.

本文转载自 etcd 集群管理维护


蜷缩的蜗牛 , 版权所有丨如未注明 , 均为原创丨 转载请注明etcd 集群管理维护
喜欢 (0)
[]
分享 (0)