elasticsearch的一些运维和开发常见问题
elasticsearch missing authentication credentials for REST request
现在网上错误的、没有经过验证的技术贴漫天飞，很容易给读者造成各种误导，由于这部分关系，笔者也开始认真地进行技术贴的撰写，同时也mark下一些知识点，特别是在这种由于公司内网环境导致无法把资料迁出的情况。
场景
elasticsearch 在开启xpack认证之后, 直接通过curl 访问接口会报错
解决
curl带上认证
在ECS终端访问REST API的时候
curl --user elastic:yourpassword -XGET ‘localhost:9200’
elastic是默认的账号
权限不够或 can not run elasticsearch as root
问题：
es安装好之后，使用root启动会报错：can not run elasticsearch as root
[root@iZbp1bb2egi7w0ueys548pZ bin]# ./elasticsearch
[2019-01-21T09:50:59,387][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:134) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:121) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:69) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:134) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) ~[elasticsearch-6.0.0.jar:6.0.0]
Caused by: java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:104) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:171) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:322) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:130) ~[elasticsearch-6.0.0.jar:6.0.0]
... 6 more
原因：
为了安全不允许使用root用户启动
解决：
es5之后的都不能使用添加启动参数或者修改配置文件等方法启动了，必须要创建用户
1、创建用户：elasticsearch
[root@iZbp1bb2egi7w0ueys548pZ bin]# adduser elasticsearch
2、创建用户密码，需要输入两次
[root@iZbp1bb2egi7w0ueys548pZ bin]# passwd elasticsearch
3、将对应的文件夹权限赋给该用户
[root@iZbp1bb2egi7w0ueys548pZ local]# chown -R elasticsearch elasticsearch-6.0.0
4、切换至elasticsearch用户
[root@iZbp1bb2egi7w0ueys548pZ etc]# su elasticsearch 5、进入启动目录启动 /usr/local/elasticsearch-6.0.0/bin 使用后台启动方式：./elasticsearch -d
[elasticsearch@vmt10003 bin]$ ./elasticsearch -d
6、启动后测试 输入curl ip:9200,如果返回一个json数据说明启动成功
可能遇到的问题
1、启动后访问ip:9200没有显示json
[root@iZbp1bb2egi7w0ueys548qZ ~]# curl 10.132.131.51:9200
curl: (7) Failed connect to 10.132.131.51:9200; Connection refused
解决：
修改elasticsearch.yml文件，添加
network.host: 0.0.0.0
再次启动就可以了
[root@iZbp1bb2egi7w0ueys548qZ ~]# curl 10.132.131.51:9200
{
"name" : "dMD7fZd",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "Cy4a99t2Sw2_hnJ_jtdRgA",
"version" : {
"number" : "6.0.0",
"build_hash" : "8f0685b",
"build_date" : "2017-11-10T18:41:22.859Z",
"build_snapshot" : false,
"lucene_version" : "7.0.1",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
如果开始了密码功能访问发生应该是这样
curl --user elastic:abc-123 -XGET '127.0.0.1:9200'
如果某些es的sh程序不能启动的话 还需要设置这些sh的执行权限 chmod -x xxx.sh
没什么问题的话 帆软报表就可以连接ES使用了
elasticsearch密码设置
7.0以后es把基础的安全模块免费集成了。很棒。现在试试设置吧。
官网安全模块文档
其中包含了一个安全入门教程
下面基本都是以我的操作为例，实际上可以按照官网文档进行其它的尝试。
一、设置账号密码
单节点
在elasticsearch.yml文件里增加配置
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
初始化密码需要在es启动的情况下进行设置，按照提示输入各个内置用户的密码。
[esuser@localhost elasticsearch-7.12.0]$./bin/elasticsearch -d
[esuser@localhost elasticsearch-7.12.0]$./bin/elasticsearch-setup-passwords interactive
主要设置了一下的密码
Changed password for user [apm_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]
开发问题
springboot中依赖
注意 我这边的es服务端是7.4.2
org.elasticsearch.client
transport
6.4.2
org.elasticsearch
elasticsearch
org.elasticsearch
elasticsearch
6.4.2
org.elasticsearch.plugin
transport-netty4-client
6.4.2
es配置代码
package com.ag.dtools.config;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.TransportAddress;
import org.elasticsearch.transport.client.PreBuiltTransportClient;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.net.InetAddress;
@Configuration
public class ElasticSearchConf {
private static final Logger LOGGER = LoggerFactory.getLogger(ElasticSearchConf.class);
/**
* elk集群地址
*/
@Value("${elasticsearch.ip}")
private String hostName;
/**
* 端口
*/
@Value("${elasticsearch.port}")
private String port;
/**
* 集群名称
*/
@Value("${elasticsearch.cluster.name}")
private String clusterName;
/**
* 连接池
*/
@Value("${elasticsearch.pool}")
private String poolSize;
/**
* Bean name default 函数名字
*
* @return
*/
@Bean(name = "transportClient")
public TransportClient transportClient() {
LOGGER.info("Elasticsearch初始化开始。。。。。");
TransportClient transportClient = null;
try {
// 配置信息
Settings esSetting = Settings.builder()
.put("cluster.name", clusterName) //集群名字
.put("client.transport.sniff", true)//增加嗅探机制，找到ES集群
.put("thread_pool.search.size", Integer.parseInt(poolSize))//增加线程池个数，暂时设为5
.build();
//配置信息Settings自定义
transportClient = new PreBuiltTransportClient(esSetting);
TransportAddress transportAddress = new TransportAddress(InetAddress.getByName(hostName), Integer.valueOf(port));
transportClient.addTransportAddresses(transportAddress);
} catch (Exception e) {
LOGGER.error("elasticsearch TransportClient create error!!", e);
}
return transportClient;
}
}
yml
elasticsearch:
ip: 112.124.xx.xx
port: 9200
pool: 5
cluster:
name: my-application