ETCD 的数据压缩

先看下错误:

1
2
3
level=error ts=2020-11-05T08:19:55.560476348Z caller=etcd.go:124 msg="error CASing" key=collectors/ring err="etcdserver: mvcc: database space exceeded"

level=error ts=2020-11-05T08:19:55.560509943Z caller=lifecycler.go:409 msg="failed to write to the KV store, sleeping" ring=ingester err="etcdserver: mvcc: database space exceeded"
1
{"level":"warn","ts":"2020-11-05T08:20:00.524Z","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-5672ee3e-005f-4100-aef5-a4e3dd2ae592/192.168.48.32:23791","attempt":0,"error":"rpc error: code = ResourceExhausted desc = etcdserver: mvcc: database space exceeded"}

这里我们需要做的是 数据压缩。

按照以下的步骤一步一步来:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# 使用 ETCDCTL_API = 3
export ETCDCTL_API=3

# 查看告警信息,告警信息一般 memberID:8630161756594109333 alarm:NOSPACE
etcdctl --endpoints=http://10.50.0.108:23791,http://10.50.0.108:23792,http://10.50.0.108:23793 alarm list

# 获取当前版本
rev=$(etcdctl --endpoints=http://10.50.0.108:23791 endpoint status --write-out="json" | egrep -o '"revision":[0-9]*' | egrep -o '[0-9].*')

# 压缩掉旧的版本
etcdctl --endpoints=http://10.50.0.108:23791 compact $rev

# 整理 多余空间
etcdctl --endpoints=http://10.50.0.108:23791 defrag

# 取消告警信息
etcdctl --endpoints=http://10.50.0.108:23791,http://10.50.0.108:23792,http://10.50.0.108:23793 alarm disarm


command: etcd -name etcd1 -data-dir=/var/lib/etcd/ -auto-compaction-retention ‘1’ -max-request-bytes ‘33554432’ –quota-backend-bytes ‘8589934592’ -advertise-client-urls http://0.0.0.0:2379 -listen-client-urls http://0.0.0.0:2379 -listen-peer-urls http://0.0.0.0:2380 -initial-cluster-token etcd-cluster -initial-cluster “etcd1=http://etcd1:2380,etcd2=http://etcd2:2380,etcd3=http://etcd3:2380" -initial-cluster-state new

etcd –name etcd2 –data-dir=/var/lib/etcd/ –auto-compaction-retention ‘1’ –max-request-bytes ‘33554432’ –quota-backend-bytes ‘8589934592’ –listen-client-urls http://0.0.0.0:2379 –advertise-client-urls http://0.0.0.0:2379 –initial-advertise-peer-urls http://0.0.0.0:2380 –listen-peer-urls http://0.0.0.0:2380 –initial-cluster-token etcd-cluster –initial-cluster “etcd1=http://etcd1:2380,etcd2=http://etcd2:2380,etcd3=http://etcd3:2380" –initial-cluster-state new