ES配置生命周期策略(二)
上篇,ES配置生命周期策略(一)只是根据官网验证了demo,
本篇给出实际工作中的使用,供参考
步骤是四步(前三步相同):
1、配置策略(policy)
PUT _ilm/policy/ehr_prod_log_policy { "policy": { "phases": { "hot": { "actions": { "rollover": { "max_size": "30gb" } } }, "delete": { "min_age": "30d", "actions": { "delete": {} } } } } }
两个阶段,hot和delete;
索引超过30G,则执滚动;
delete阶段,超过30天执行删除动作。
2、索引模版(template)
PUT _template/ehr_prod_log_template { "index_patterns": ["ehr-prod-log-rollover-*"], "settings": { "number_of_shards": 1, "number_of_replicas": 1, "index.lifecycle.name": "ehr_prod_log_policy", "index.lifecycle.rollover_alias": "ehr-prod-log-rollover" } }
kafka中topic名称ehr-prod-log,这里对应定义别名“ehr-prod-log-rollover”,logstash配置会用到。
3、索引(index)
PUT ehr-prod-log-rollover-00000001 { "aliases": { "ehr-prod-log-rollover": { "is_write_index": true } } }
4、Logstash配置
output { elasticsearch { hosts => ["**"] index => "ehr-prod-log-rollover" user => "ehr-prod" password => "***" } }
实战中,通过logstash将日志从kafka采集到es,index => "ehr-prod-log-rollover",这里使用别名,与之对应。
则在es中对应生成对应索引,ehr-prod-log-rollover-0000000n,而对外使用别名即可。