elasticsearch-6.2.4集群安装以及开启集群X-pack密码认证
elasticsearch-6.2.4集群安装以及开启集群X-pack密码认证
一、rpm包方式安装elasticsearch
环境是：
[root@sdk-25 run]# cat /etc/redhat-release CentOS Linux release 7.6.1810 (Core) 关闭selinx, 下面是安装和启动命令 [root@local-216 soft]# rpm -ivh elasticsearch-6.2.4.rpm warning: elasticsearch-6.2.4.rpm: Header V4 RSA/SHA512 Signature, key ID d88e42b4: NOKEY Preparing... ################################# [100%] Creating elasticsearch group... OK Creating elasticsearch user... OK Updating / installing... 1:elasticsearch-0:6.2.4-1 ################################# [100%]
下面是 systemctl启动es命令:
NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd sudo systemctl daemon-reload sudo systemctl enable elasticsearch.service You can start elasticsearch service by executing sudo systemctl start elasticsearch.service
二、采用二进制包elasticsearch-6.2.4.tar.gz 直接解压方式安装
提前对elasticsearch服务器的系统进行下面的参数优化，这样在安装完es启动过程中可以避免好多报错
ES服务器系统环境优化:
最少使用swap内存交换分区， 关于优化，可以参考 https://www.jianshu.com/p/7c163d7e9ecb [root@sdk-25 ~]# tail -2 /etc/sysctl.conf vm.swappiness=1 ##禁止用交换内存vm.max_map_count=262144 ##设置虚拟内存 [root@sdk-25 ~]# cat /etc/security/limits.conf ##文件句柄数* soft nofile 131072* hard nofile 131072##进程线程数* soft nproc 131072* hard nproc 131072##内存锁定交换* soft memlock unlimited * hard memlock unlimited
安装系统环境说明:
[root@sdk-25 run]# cat /etc/redhat-release
CentOS Linux release 7.6.1810 (Core)
具体安装步骤如下：
提前安装好jdk1.8环境：
[root@sdk-25 config]# source /etc/profile
export JAVA_HOME=/usr/local/jdk
export PATH=$JAVA_HOME/bin:$PATH
centos7.6.单实例二进制包安装ES:
下载elasticsearch-6.2.4.tar.gz 二进制安装包：
wget -P /data/soft https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.4.tar.gz
下面准备的是单台机器启动三个elasticsearch实例：
cd /data/soft;tar xf elasticsearch-6.2.4.tar.gz -C /usr/local/;cp -rp elasticsearch-6.2.4 elasticsearch01;cp -rp elasticsearch-6.2.4 elasticsearch02;cp -rp elasticsearch-6.2.4 elasticsearch03;
useradd elasticsearch;passwd elasticsearch
mkdir /data/elasticsearch{01,02,03}/{data,logs,run} -p cd /data
chown -R elasticsearch.elasticsearch elasticsearch0*2.1、启动第一个单实例elasticsearch01
单实例elasticsearch01的配置文件如下:
[root@sdk-25 config]# cat /usr/local/elasticsearch01/config/elasticsearch.yml node.name: node25## Add custom attributes to the node:##node.attr.rack: r1#path.data: /data/elasticsearch01/data path.logs: /data/elasticsearch01/logs#bootstrap.memory_lock: true#network.host: 127.0.0.1#network.host: 192.168.1.25#http.port: 9200 transport.tcp.port: 9300##下面是es7版本的参数#discovery.seed_hosts: ["192.168.1.25:9300"]#cluster.initial_master_nodes: ["192.168.1.25:9300"]
提示：之前在centos7.6 机器上是rpm包方式安装的elasticsearch，所以会有systemctl启动脚本,或者service elasticsearch01 start/status/restart/
[root@sdk-25 ~]# cp /usr/lib/systemd/system/elasticsearch.service /usr/lib/systemd/system/elasticsearch01.service[root@sdk-25 ~]# systemctl enable elasticsearch01.service Created symlink from /etc/systemd/system/multi-user.target.wants/elasticsearch01.service to /usr/lib/systemd/system/elasticsearch01.service.
elasticsearch01启动脚本配置文件elasticsearch 配置文件如下:
[root@sdk-25 run]# cat /usr/lib/systemd/system/elasticsearch01.service[Unit]Description=ElasticsearchDocumentation=http://www.elastic.coWants=network-online.targetAfter=network-online.target[Service]RuntimeDirectory=elasticsearchEnvironment=ES_HOME=/usr/local/elasticsearch01Environment=ES_PATH_CONF=/usr/local/elasticsearch01/configEnvironment=PID_DIR=/data/elasticsearch01/runEnvironmentFile=-/etc/sysconfig/elasticsearch01WorkingDirectory=/usr/local/elasticsearch01User=elasticsearchGroup=elasticsearchExecStart=/usr/local/elasticsearch01/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet# StandardOutput is configured to redirect to journalctl since# some error messages may be logged in standard output before# elasticsearch logging system is initialized. Elasticsearch# stores its logs in /var/log/elasticsearch and does not use# journalctl by default. If you also want to enable journalctl# logging, you can simply remove the "quiet" option from ExecStart.StandardOutput=journalStandardError=inherit# Specifies the maximum file descriptor number that can be opened by this processLimitNOFILE=65536# Specifies the maximum number of processesLimitNPROC=4096# Specifies the maximum size of virtual memoryLimitAS=infinity# Specifies the maximum file sizeLimitFSIZE=infinity# Disable timeout logic and wait until process is stoppedTimeoutStopSec=0# SIGTERM signal is used to stop the Java processKillSignal=SIGTERM# Send the signal only to the JVM rather than its control groupKillMode=process# Java process is never killedSendSIGKILL=no# When a JVM receives a SIGTERM signal it exits with code 143SuccessExitStatus=143[Install]WantedBy=multi-user.target# Built for distribution-6.2.4 (distribution)elasticsearch01环境变量 配置文件如下:
[root@sdk-25 local]# cp /etc/sysconfig/elasticsearch /etc/sysconfig/elasticsearch01 [root@sdk-25 run]# cat /etc/sysconfig/elasticsearch01#################################Elasticsearch#################################Elasticsearch home directoryES_HOME=/usr/local/elasticsearch01#Elasticsearch Java pathJAVA_HOME=/usr/local/jdk Elasticsearch configuration directory ES_PATH_CONF=/usr/local/elasticsearch01/config#Elasticsearch PID directoryPID_DIR=/data/elasticsearch01/run#Additional Java OPTS#ES_JAVA_OPTS=#Configure restart on package upgrade (true, every other setting will lead to not restarting)#RESTART_ON_UPGRADE=true#################################Elasticsearch service#################################SysV init.d#The number of seconds to wait before checking if Elasticsearch started successfully as a daemon processES_STARTUP_SLEEP_TIME=5#################################System properties#################################Specifies the maximum file descriptor number that can be opened by this process#When using Systemd, this setting is ignored and the LimitNOFILE defined in#/usr/lib/systemd/system/elasticsearch.service takes precedence#MAX_OPEN_FILES=65536#The maximum number of bytes of memory that may be locked into RAM#Set to "unlimited" if you use the 'bootstrap.memory_lock: true' option#in elasticsearch.yml.#When using systemd, LimitMEMLOCK must be set in a unit file such as#/etc/systemd/system/elasticsearch.service.d/override.conf.#MAX_LOCKED_MEMORY=unlimited#Maximum number of VMA (Virtual Memory Areas) a process can own#When using Systemd, this setting is ignored and the 'vm.max_map_count'#property is set at boot time in /usr/lib/sysctl.d/elasticsearch.conf#MAX_MAP_COUNT=262144
[root@sdk-25 run]# systemctl status elasticsearch.service● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
Active: active (running) since 五 2020-07-24 23:44:12 CST; 3min 32s ago
Docs: http://www.elastic.co
Main PID: 18141 (java)
CGroup: /system.slice/elasticsearch.service
└─18141 /usr/local/jdk/bin/java -Xms4g -Xmx4g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -Xss1m -Djava....7月 24 23:44:12 sdk-25 systemd[1]: Started Elasticsearch.
[root@sdk-25 local]# ss -lntup|grep javatcp LISTEN 0 128 ::ffff:127.0.0.1:9200 :::* users:(("java",pid=7245,fd=750))
tcp LISTEN 0 128 ::ffff:127.0.0.1:9300 :::* users:(("java",pid=7245,fd=556))
[root@sdk-25 local]# service elasticsearch01 stopRedirecting to /bin/systemctl stop elasticsearch01.service
[root@sdk-25 local]# service elasticsearch01 startRedirecting to /bin/systemctl start elasticsearch01.service
[root@sdk-25 local]# ss -lntup|grep javatcp LISTEN 0 128 ::ffff:127.0.0.1:9200 :::* users:(("java",pid=8591,fd=750))
tcp LISTEN 0 128 ::ffff:127.0.0.1:9300 :::* users:(("java",pid=8591,fd=556))
[root@sdk-25 local]# service elasticsearch01 statusRedirecting to /bin/systemctl status elasticsearch01.service
● elasticsearch01.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch01.service; enabled; vendor preset: disabled)
Active: active (running) since 日 2020-07-26 12:03:03 CST; 54s ago
Docs: http://www.elastic.co
Main PID: 7245 (java)
CGroup: /system.slice/elasticsearch01.service
├─7245 /usr/local/jdk/bin/java -Xms4g -Xmx4g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -Xss1m -Djava.a...
└─7409 /usr/local/elasticsearch01/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/controller7月 26 12:03:03 sdk-25 systemd[1]: Started Elasticsearch.2.2、启动第二个单实例elasticsearch02
操作过程同第一个单实例elasticsearch01一样
单实例elasticsearch02配置文件内容如下:
[root@sdk-25 local]# cat /usr/local/elasticsearch02/config/elasticsearch.ymlnode.name: node25-1#Add custom attributes to the node:#node.attr.rack: r1path.data: /data/elasticsearch02/datapath.logs: /data/elasticsearch02/logsbootstrap.memory_lock: truenetwork.host: 127.0.0.1#network.host: 192.168.1.25http.port: 9201transport.tcp.port: 9301##下面是es7版本的参数#discovery.seed_hosts: ["192.168.1.25:9300"]#cluster.initial_master_nodes: ["192.168.1.25:9300"]
准备systemctl启动的配置文件和启动加载的环境变量文件：
[root@sdk-25 local]# cat /usr/lib/systemd/system/elasticsearch02.service[Unit]Description=ElasticsearchDocumentation=http://www.elastic.coWants=network-online.targetAfter=network-online.target[Service]RuntimeDirectory=elasticsearchEnvironment=ES_HOME=/usr/local/elasticsearch02Environment=ES_PATH_CONF=/usr/local/elasticsearch02/configEnvironment=PID_DIR=/data/elasticsearch02/runEnvironmentFile=-/etc/sysconfig/elasticsearch02WorkingDirectory=/usr/local/elasticsearch02User=elasticsearchGroup=elasticsearchExecStart=/usr/local/elasticsearch02/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet#StandardOutput is configured to redirect to journalctl since#some error messages may be logged in standard output before#elasticsearch logging system is initialized. Elasticsearch#stores its logs in /var/log/elasticsearch and does not use#journalctl by default. If you also want to enable journalctl#logging, you can simply remove the "quiet" option from ExecStart.StandardOutput=journalStandardError=inherit#Specifies the maximum file descriptor number that can be opened by this processLimitNOFILE=65536#Specifies the maximum number of processesLimitNPROC=4096#Specifies the maximum size of virtual memoryLimitAS=infinity#Specifies the maximum file sizeLimitFSIZE=infinity#Disable timeout logic and wait until process is stoppedTimeoutStopSec=0#SIGTERM signal is used to stop the Java processKillSignal=SIGTERM#Send the signal only to the JVM rather than its control groupKillMode=process#Java process is never killedSendSIGKILL=no#When a JVM receives a SIGTERM signal it exits with code 143SuccessExitStatus=143[Install]WantedBy=multi-user.target#Built for distribution-6.2.4 (distribution)[root@sdk-25 local]# cat /etc/sysconfig/elasticsearch02 #################################Elasticsearch#################################Elasticsearch home directoryES_HOME=/usr/local/elasticsearch02#Elasticsearch Java pathJAVA_HOME=/usr/local/jdk#Elasticsearch configuration directoryES_PATH_CONF=/usr/local/elasticsearch02/config#Elasticsearch PID directoryPID_DIR=/data/elasticsearch02/run#Additional Java OPTS#ES_JAVA_OPTS=#Configure restart on package upgrade (true, every other setting will lead to not restarting)#RESTART_ON_UPGRADE=true#################################Elasticsearch service#################################SysV init.d#The number of seconds to wait before checking if Elasticsearch started successfully as a daemon processES_STARTUP_SLEEP_TIME=5#################################System properties#################################Specifies the maximum file descriptor number that can be opened by this process#When using Systemd, this setting is ignored and the LimitNOFILE defined in#/usr/lib/systemd/system/elasticsearch.service takes precedence#MAX_OPEN_FILES=65536#The maximum number of bytes of memory that may be locked into RAM#Set to "unlimited" if you use the 'bootstrap.memory_lock: true' option#in elasticsearch.yml.#When using systemd, LimitMEMLOCK must be set in a unit file such as#/etc/systemd/system/elasticsearch.service.d/override.conf.#MAX_LOCKED_MEMORY=unlimited#Maximum number of VMA (Virtual Memory Areas) a process can own#When using Systemd, this setting is ignored and the 'vm.max_map_count'#property is set at boot time in /usr/lib/sysctl.d/elasticsearch.conf#MAX_MAP_COUNT=262144
systemctl 相关的es启动命令如下:
保证es开机自启动:
[root@sdk-25 system]# systemctl enable elasticsearch02.serviceCreated symlink from /etc/systemd/system/multi-user.target.wants/elasticsearch02.service to /usr/lib/systemd/system/elasticsearch02.service.
[root@sdk-25 local]# service elasticsearch02 status/stop/restart[root@sdk-25 local]# systemctl status elasticsearch02.service ● elasticsearch02.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch02.service; enabled; vendor preset: disabled)
Active: inactive (dead) since 日 2020-07-26 12:21:49 CST; 6s ago
Docs: http://www.elastic.co
Process: 4255 ExecStart=/usr/local/elasticsearch02/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=143)
Main PID: 4255 (code=exited, status=143)7月 25 00:02:22 sdk-25 systemd[1]: Started Elasticsearch.7月 26 12:21:49 sdk-25 systemd[1]: Stopping Elasticsearch...7月 26 12:21:49 sdk-25 systemd[1]: Stopped Elasticsearch.
[root@sdk-25 local]# service elasticsearch02 startRedirecting to /bin/systemctl start elasticsearch02.service
[root@sdk-25 local]# ss -lntup|egrep "9201|9301"tcp LISTEN 0 128 ::ffff:127.0.0.1:9201 :::* users:(("java",pid=11387,fd=685))
tcp LISTEN 0 128 ::ffff:127.0.0.1:9301 :::* users:(("java",pid=11387,fd=491))[root@sdk-25 system]# curl http://127.0.0.1:9201{ "name" : "node25-1", "cluster_name" : "elasticsearch", "cluster_uuid" : "6qPnIoTCRn2fACH4CENyPA", "version" : { "number" : "6.2.4", "build_hash" : "ccec39f", "build_date" : "2018-04-12T20:37:28.497551Z", "build_snapshot" : false, "lucene_version" : "7.2.1", "minimum_wire_compatibility_version" : "5.6.0", "minimum_index_compatibility_version" : "5.0.0"
}, "tagline" : "You Know, for Search"}2.3启动第三个单实例elasticsearch03
操作步骤和上面的方法一样，此处不再过多的描述了
三、elasticsearch01 实例一上安装IK插件和pinyin插件,以及x-pack插件
这些插件的版本要和实例elasticsearch01的版本严格保持一致，否则会出错（不兼容）
安装IK插件和pinyin插件 直接在github上下下载，解压到/usr/local/elasticsearch01/plugins ,同时一定要注意插件的权限必须为elasticsearch，否则到期es重启失败。同时安装完插件要重启下es服务
[root@sdk-25 plugins]# pwd/usr/local/elasticsearch01/plugins [root@sdk-25 plugins]# lsik pinyin x-pack[root@sdk-25 plugins]# ll总用量 0drwxrwxrwx 3 elasticsearch elasticsearch 213 5月 6 2018 ik drwxrwxrwx 2 elasticsearch elasticsearch 113 5月 6 2018 pinyin drwxr-xr-x 11 elasticsearch elasticsearch 244 7月 25 12:07 x-pack
下面详细的介绍下x-pack插件的安装和简单的应用：
安装x-pack插件
参考文档:
https://www.jianshu.com/p/802c5d803a95
查看已经安装的插件:
[root@sdk-25 vhost]# /usr/local/elasticsearch01/bin/elasticsearch-plugin listik pinyi
注册x-pack插件，两种安装方式：
在线安装方式：（g国内的话，基于网络环境，，非常慢，基本安装不上）
下面是在国外的服务器进行现在安装的，非常的快 [root@192-200-102-74 plugins]# /usr/share/elasticsearch/bin/elasticsearch-plugin install x-pack -> Downloading x-pack from elastic [=================================================] 100%?? @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: plugin requires additional permissions @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ * java.io.FilePermission \\.\pipe\* read,write* java.lang.RuntimePermission accessClassInPackage.com.sun.activation.registries * java.lang.RuntimePermission getClassLoader * java.lang.RuntimePermission setContextClassLoader * java.lang.RuntimePermission setFactory * java.net.SocketPermission * connect,accept,resolve * java.security.SecurityPermission createPolicy.JavaPolicy * java.security.SecurityPermission getPolicy * java.security.SecurityPermission putProviderProperty.BC * java.security.SecurityPermission setPolicy * java.util.PropertyPermission * read,writeSee http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.htmlfor descriptions of what these permissions allow and the associated risks. Continue with installation? [y/N]y@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: plugin forks a native controller @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ This plugin launches a native controller that is not subject to the Java security manager nor to system call filters. Continue with installation? [y/N]yElasticsearch keystore is required by plugin [x-pack-security], creating... -> Installed x-pack with: x-pack-core,x-pack-deprecation,x-pack-graph,x-pack-logstash,x-pack-ml,x-pack-monitoring,x-pack-security,x-pack-upgrade,x-pack-watcher [root@192-200-102-74 plugins]# echo $?0
第二种就是离线安装方式:国内的服务器建议就离线安装:
**提前下载多对应的x-pack插件的版本。
我线上用的是elasticsearch.6.2.4.tar.gz 二进制包安装的，所以要下载对应的离线插件版本x-pack.2.6.4.zip包
官方的下载地址，当然也得搭*下载
https://artifacts.elastic.co/downloads/packs/x-pack/x-pack-6.2.4.zip
下载完，上传到香港的阿里ECS服务器
du -sh x-pack-6.2.4.zip
296M x-pack-6.2.4.zip
从HK的es服务器推送到阿里的OSS,然后通过阿里CDN域名来下载：
[root@hk-cj01 ~]# /usr/local/sbin/ossutil64 --config-file=/data/soft/ossconfig cp x-pack-6.2.4.zip oss://lanhu-static/zy01baodown/ --update Succeed: Total num: 1, size: 309,419,696. OK num: 1(upload 1 files). 79.701253(s) elapsed wget https://va1.j7lf.cn/zy01baodown/x-pack-6.2.4.zip [root@sdk-25 vhost]# /usr/local/elasticsearch01/bin/elasticsearch-plugin install file:///data/soft/x-pack-6.2.4.zip -> Downloading file:///data/soft/x-pack-6.2.4.zip [=================================================] 100% @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: plugin requires additional permissions @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ * java.io.FilePermission \\.\pipe\* read,write * java.lang.RuntimePermission accessClassInPackage.com.sun.activation.registries * java.lang.RuntimePermission getClassLoader * java.lang.RuntimePermission setContextClassLoader * java.lang.RuntimePermission setFactory * java.net.SocketPermission * connect,accept,resolve * java.security.SecurityPermission createPolicy.JavaPolicy * java.security.SecurityPermission getPolicy * java.security.SecurityPermission putProviderProperty.BC * java.security.SecurityPermission setPolicy * java.util.PropertyPermission * read,write See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.htmlfor descriptions of what these permissions allow and the associated risks. Continue with installation? [y/N]y@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: plugin forks a native controller @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ This plugin launches a native controller that is not subject to the Java security manager nor to system call filters. Continue with installation? [y/N]yElasticsearch keystore is required by plugin [x-pack-security], creating... -> Installed x-pack with: x-pack-core,x-pack-deprecation,x-pack-graph,x-pack-logstash,x-pack-ml,x-pack-monitoring,x-pack-security,x-pack-upgrade,x-pack-watcher
确定插件是否安装成功:
[root@sdk-25 vhost]# /usr/local/elasticsearch01/bin/elasticsearch-plugin listik pinyinx-pack x-pack-core x-pack-deprecation x-pack-graph x-pack-logstash x-pack-ml x-pack-monitoring x-pack-security x-pack-upgrade x-pack-watcher
卸载x-pack插件
bin/elasticsearch-plugin remove x-pack
es默认生成的密码如下:
/usr/local/elasticsearch01/bin/x-pack/setup-passwords auto
[root@sdk-25 vhost]# /usr/local/elasticsearch01/bin/x-pack/setup-passwords auto Unexpected response code [404] from calling GET http://127.0.0.1:9200/_xpack/security/_authenticate?pretty
Possible causes include:
* The relative path of the URL is incorrect. Is there a proxy in-between?
* The protocol (http/https) does not match the port.
* Is this really an Elasticsearch server?
ERROR: Uknown error
报错，因为安装完插件需要重启下es,
[root@sdk-25 vhost]# systemctl restart elasticsearch.service**重启完接着报错：**
[root@sdk-25 vhost]# systemctl status elasticsearch.service● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since 六 2020-07-25 12:27:59 CST; 7s ago
Docs: http://www.elastic.co
Process: 32419 ExecStart=/usr/local/elasticsearch01/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=1/FAILURE)
Main PID: 32419 (code=exited, status=1/FAILURE)7月 25 12:27:57 sdk-25 systemd[1]: Started Elasticsearch.7月 25 12:27:59 sdk-25 elasticsearch[32419]: Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /usr/local/elastic...rch.keystore7月 25 12:27:59 sdk-25 systemd[1]: elasticsearch.service: main process exited, code=exited, status=1/FAILURE7月 25 12:27:59 sdk-25 systemd[1]: Unit elasticsearch.service entered failed state.7月 25 12:27:59 sdk-25 systemd[1]: elasticsearch.service failed.
Hint: Some lines were ellipsized, use -l to show in full.查看安装上的插件权限，发现是权限不对导致的
[root@sdk-25 plugins]# ll总用量 0drwxrwxrwx 3 root root 213 5月 6 2018 ik drwxrwxrwx 2 root root 113 5月 6 2018 pinyin drwxr-xr-x 11 root root 244 7月 25 12:07 x-pack
授权elasticsearch：
[root@sdk-25 elasticsearch01]# chown -R elasticsearch.elasticsearch *启动成功： [root@sdk-25 elasticsearch01]# systemctl status elasticsearch.service● elasticsearch.service - Elasticsearch Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled) Active: active (running) since 六 2020-07-25 12:33:11 CST; 43min ago Docs: http://www.elastic.co Main PID: 1266 (java) CGroup: /system.slice/elasticsearch.service ├─1266 /usr/local/jdk/bin/java -Xms4g -Xmx4g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -Xss1m -Djava.a... └─1431 /usr/local/elasticsearch01/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/controller7月 25 12:33:11 sdk-25 systemd[1]: Started Elasticsearch.
es默认生成的密码如下:
/usr/local/elasticsearch01/bin/x-pack/setup-passwords auto
[root@sdk-25 elasticsearch01]# /usr/local/elasticsearch01/bin/x-pack/setup-passwords auto Initiating the setup of passwords for reserved users elastic,kibana,logstash_system.
The passwords will be randomly generated and printed to the console.
Please confirm that you would like to continue [y/N]y
Changed password for user kibana
PASSWORD kibana = FvdqDOUkXvEijZKjfB8p
Changed password for user logstash_system
PASSWORD logstash_system = VUFZn9iL4AEJrH3Owkdq
Changed password for user elastic
PASSWORD elastic = 5BheRCDLKSvT1ZP1zhHf
[root@sdk-25 elasticsearch01]# curl http://127.0.0.1:9200 {"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication token for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication token for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}[root@sdk-25 elasticsearch01]#通过账户和密码访问:
[root@sdk-25 elasticsearch01]# curl --user elastic:5BheRCDLKSvT1ZP1zhHf http://127.0.0.1:9200{ "name" : "node25", "cluster_name" : "elasticsearch", "cluster_uuid" : "YSRMrxOBTZW7hicZqZ-Dhg", "version" : { "number" : "6.2.4", "build_hash" : "ccec39f", "build_date" : "2018-04-12T20:37:28.497551Z", "build_snapshot" : false, "lucene_version" : "7.2.1", "minimum_wire_compatibility_version" : "5.6.0", "minimum_index_compatibility_version" : "5.0.0"
}, "tagline" : "You Know, for Search"}忘记ES密码找回方法:
https://www.cnblogs.com/mere/p/12165637.html
使用命令ES_HOME/bin/x-pack/users创建一个基于本地问价认证的超级管理员:
[root@sdk-25 config]# /usr/local/elasticsearch01/bin/x-pack/users useradd my_admin -p 5BheRCDLK12389Sv -r superuser[root@sdk-25 config]#
通过api重置elastic超级管理员的密码:
curl -u my_admin -XPUT 'http://localhost:9200/_xpack/security/user/elastic/_password?pretty' -H 'Content-Type: application/json' -d'{"password" : "5BheRCDLK12389Sv"}'
[root@sdk-25 config]# curl -u my_admin -XPUT 'http://localhost:9200/_xpack/security/user/elastic/_password?pretty' -H 'Content-Type: application/json' -d'{"password" : "5BheRCDLK12389Sv"}'
Enter host password for user 'my_admin': 5BheRCDLK12389Sv
{ }
[2020-07-25T14:19:50,117][INFO ][o.e.x.s.a.f.FileUserPasswdStore] [node25] users file [/usr/local/elasticsearch01/config/x-pack/users] changed. updating users... )
[2020-07-25T14:19:50,124][INFO ][o.e.x.s.a.f.FileUserRolesStore] [node25] users roles file [/usr/local/elasticsearch01/config/x-pack/users_roles] changed. updating users roles...使用原来的密码登录失败:
[root@sdk-25 ~]# curl --user elastic:5BheRCDLKSvT1ZP1zhHf http://127.0.0.1:9200{"error":{"root_cause":[{"type":"security_exception","reason":"failed to authenticate user [elastic]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"failed to authenticate user [elastic]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}[root@sdk-25 ~]# [2020-07-25T14:23:51,638][INFO ][o.e.x.s.a.AuthenticationService] [node25] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]使用新的密码是成功的:
[root@sdk-25 ~]# curl --user elastic:5BheRCDLK12389Sv http://127.0.0.1:9200{ "name" : "node25", "cluster_name" : "elasticsearch", "cluster_uuid" : "YSRMrxOBTZW7hicZqZ-Dhg", "version" : { "number" : "6.2.4", "build_hash" : "ccec39f", "build_date" : "2018-04-12T20:37:28.497551Z", "build_snapshot" : false, "lucene_version" : "7.2.1", "minimum_wire_compatibility_version" : "5.6.0", "minimum_index_compatibility_version" : "5.0.0"
}, "tagline" : "You Know, for Search"}校验下密码是否重置成功:
curl -u elastic 'http://127.0.0.1:9200/_xpack/security/_authenticate?pretty'[root@sdk-25 ~]# curl -u elastic 'http://localhost:9200/_xpack/security/_authenticate?pretty'Enter host password for user 'elastic': 5BheRCDLK12389Sv
{ "username" : "elastic", "roles" : [ "superuser"
], "full_name" : null, "email" : null, "metadata" : { "_reserved" : true
}, "enabled" : true}ElasticSearch之CURL操作:
https://blog.csdn.net/diyiday/article/details/83927744
[root@local-216 ~]# /usr/share/elasticsearch/bin/x-pack/users useradd my_admin -p admin123987 -r superuser
ES学习参考资料:
https://www.cnblogs.com/leeSmall/p/9189078.html
四、单台服务器安装3个ES实例，配置基于X-pack密码认证的ES集群
配置文件内容如下:
[root@sdk-25 logs]# cat /usr/local/elasticsearch01/config/elasticsearch.ymlcluster.name: esclusternode.name: es1node.master: truenode.data: truepath.data: /data/elasticsearch01/datapath.logs: /data/elasticsearch01/logsbootstrap.memory_lock: truebootstrap.system_call_filter: falsehttp.port: 9200transport.tcp.port: 9300network.host: 127.0.0.1discovery.zen.minimum_master_nodes: 2discovery.zen.ping_timeout: 3sdiscovery.zen.ping.unicast.hosts: ["127.0.0.1:9301","127.0.0.1:9302"]
[root@sdk-25 logs]# cat /usr/local/elasticsearch02/config/elasticsearch.ymlcluster.name: esclusternode.name: es2node.master: truenode.data: truepath.data: /data/elasticsearch02/datapath.logs: /data/elasticsearch02/logsbootstrap.memory_lock: truebootstrap.system_call_filter: falsehttp.port: 9201transport.tcp.port: 9301network.host: 127.0.0.1discovery.zen.minimum_master_nodes: 2discovery.zen.ping_timeout: 3sdiscovery.zen.ping.unicast.hosts: ["127.0.0.1:9300","127.0.0.1:9302"]
[root@sdk-25 logs]# cat /usr/local/elasticsearch03/config/elasticsearch.ymlcluster.name: esclusternode.name: es3node.master: truenode.data: truepath.data: /data/elasticsearch03/datapath.logs: /data/elasticsearch03/logsbootstrap.memory_lock: truebootstrap.system_call_filter: falsehttp.port: 9202transport.tcp.port: 9302network.host: 127.0.0.1discovery.zen.minimum_master_nodes: 2discovery.zen.ping_timeout: 3sdiscovery.zen.ping.unicast.hosts: ["127.0.0.1:9300","127.0.0.1:9301"]
三台ES实例都启动，并且三个实例都不要提前安装x-pack密码认证插件.三台实例会自动识别加入到集群中。
正确给ES集群设置密码的方法是：
一开始3个实例都不要安装x-pack插件设置密码。而是先配置好三个实例，都启动，从是master节点的实例上来安装x-pack插件来来设置密码。
然后其他的节点也都安装x-pack插件，然后重启其他的2个ES实例，但是其他的2个ES实例不需要设置密码
下面是三个ES实例都未安装x-pack密码插件时，查看集群的状态:
[root@sdk-25 plugins]# curl http://127.0.0.1:9200/_cat/nodes127.0.0.1 28 32 0 0.06 0.09 0.12 mdi - es3127.0.0.1 42 32 0 0.06 0.09 0.12 mdi * es1127.0.0.1 44 32 0 0.06 0.09 0.12 mdi - es2 [root@sdk-25 plugins]# curl http://127.0.0.1:9201/_cat/nodes127.0.0.1 42 32 0 0.00 0.04 0.10 mdi * es1127.0.0.1 44 32 0 0.00 0.04 0.10 mdi - es2127.0.0.1 28 32 0 0.00 0.04 0.10 mdi - es3 [root@sdk-25 plugins]# curl http://127.0.0.1:9202/_cat/nodes127.0.0.1 28 32 0 0.00 0.04 0.10 mdi - es3127.0.0.1 42 32 0 0.00 0.04 0.10 mdi * es1127.0.0.1 44 32 0 0.00 0.04 0.10 mdi - es2
master节点是ES1:
[root@sdk-25 plugins]# curl 'http://127.0.0.1:9200/_cat/master?v'id host ip node VojUpPevTV2tH56AwJN03g 127.0.0.1 127.0.0.1 es1
可以看到ES3节点是集群中的master，从是ES-master节点ES1的实例上来安装x-pack插件来给次实例设置密码。然后再给其他的2个ES实例也安装x-pack插件，并且重启其他的2个ES服务。
但是都不需要给这2个ES实例设置密码
[root@sdk-25 bin]# /usr/local/elasticsearch01/bin/elasticsearch-plugin install file:///data/soft/x-pack-6.2.4.zip-> Downloading file:///data/soft/x-pack-6.2.4.zip
[=================================================] 100%
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: plugin requires additional permissions @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.io.FilePermission \\.\pipe\* read,write
* java.lang.RuntimePermission accessClassInPackage.com.sun.activation.registries
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.RuntimePermission setFactory
* java.net.SocketPermission * connect,accept,resolve
* java.security.SecurityPermission createPolicy.JavaPolicy
* java.security.SecurityPermission getPolicy
* java.security.SecurityPermission putProviderProperty.BC
* java.security.SecurityPermission setPolicy
* java.util.PropertyPermission * read,write
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.htmlfor descriptions of what these permissions allow and the associated risks.
Continue with installation? [y/N]y
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: plugin forks a native controller @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
This plugin launches a native controller that is not subject to the Java
security manager nor to system call filters.
Continue with installation? [y/N]y
Elasticsearch keystore is required by plugin [x-pack-security], creating...
-> Installed x-pack with: x-pack-core,x-pack-deprecation,x-pack-graph,x-pack-logstash,x-pack-ml,x-pack-monitoring,x-pack-security,x-pack-upgrade,x-pack-watcher
[root@sdk-25 bin]#
[root@sdk-25 bin]# cd /usr/local/elasticsearch01/plugins/
[root@sdk-25 plugins]# ll总用量 0drwxrwxrwx 3 root root 213 5月 6 2018 ik
drwxrwxrwx 2 root root 113 5月 6 2018 pinyin
drwxr-xr-x 11 root root 244 7月 26 16:15 x-pack
[root@sdk-25 bin]#cd /usr/local/elasticsearch03
[root@sdk-25 bin]# chown -R elasticsearch.elasticsearch *[root@sdk-25 plugins]# ll总用量 0drwxrwxrwx 3 elasticsearch elasticsearch 213 5月 6 2018 ik
drwxrwxrwx 2 elasticsearch elasticsearch 113 5月 6 2018 pinyin
drwxr-xr-x 11 elasticsearch elasticsearch 244 7月 26 16:15 x-pack[root@sdk-25 config]# systemctl restart elasticsearch01[root@sdk-25 config]# /usr/local/elasticsearch01/bin/x-pack/users useradd my_admin -p 5BheRCDLK12389Sv -r superuser[root@sdk-25 config]# curl -u my_admin -XPUT 'http://localhost:9200/_xpack/security/user/elastic/_password?pretty' -H 'Content-Type: application/json' -d'{"password" : "5BheRCDLK12389Sv"}'Enter host password for user 'my_admin':
{ }给剩余的ES2 ES3实例安装x-pack插件，然后授权elasticsearch.elasticsearch，最后重启这个2个es实例：
/usr/local/elasticsearch03/bin/elasticsearch-plugin install file:///data/soft/x-pack-6.2.4.zip /usr/local/elasticsearch02/bin/elasticsearch-plugin install file:///data/soft/x-pack-6.2.4.zip [root@sdk-25 config]# cd /usr/local/ [root@sdk-25 local]# chown -R elasticsearch.elasticsearch elasticsearch0* [root@sdk-25 ~]# systemctl restart elasticsearch02; systemctl restart elasticsearch03
此时只能通过用户和密码来查看集群的状态:
[root@sdk-25 local]# curl --user elastic:5BheRCDLK12389Sv http://127.0.0.1:9200/_cat/nodes127.0.0.1 22 32 5 3.07 1.00 0.47 mdi - es2127.0.0.1 38 32 5 3.07 1.00 0.47 mdi * es1127.0.0.1 31 32 5 3.07 1.00 0.47 mdi - es3 [root@sdk-25 local]# curl --user elastic:5BheRCDLK12389Sv http://127.0.0.1:9201/_cat/nodes127.0.0.1 38 32 0 2.83 0.98 0.47 mdi * es1127.0.0.1 22 32 0 2.83 0.98 0.47 mdi - es2127.0.0.1 31 32 0 2.83 0.98 0.47 mdi - es3 [root@sdk-25 local]# curl --user elastic:5BheRCDLK12389Sv http://127.0.0.1:9202/_cat/nodes127.0.0.1 24 32 1 2.76 1.00 0.48 mdi - es2127.0.0.1 41 32 1 2.76 1.00 0.48 mdi * es1127.0.0.1 31 32 1 2.76 1.00 0.48 mdi - es3
不输入账户和密码访问节点报错：
[root@sdk-25 local]# curl http://127.0.0.1:9200/_cat/nodes{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication token for REST request [/_cat/nodes]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication token for REST request [/_cat/nodes]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}[root@sdk-25 local]#五、ES集群部署过程中遇到的问题
案例一：
[root@sdk-25 plugins]# /usr/local/elasticsearch01/bin/x-pack/setup-passwords autoUnexpected response code [404] from calling GET http://127.0.0.1:9202/_xpack/security/_authenticate?pretty Possible causes include: * The relative path of the URL is incorrect. Is there a proxy in-between? * The protocol (http/https) does not match the port. * Is this really an Elasticsearch server? ERROR: Uknown error
原因是安装完x-pack插件没有重新授权elasticsearch.elasticsearch.并且重启es服务，才导致的报错
案例二：
elasticsearch01 实例安装了x-pack 插件的，并且设置了ES的登录密码 其他的都没有安装，导致在配置集群时，实例elasticsearch01加入到集群中失败
实例es1 输出错误日志：
[zen-disco-node-failed({es1}{jbc_qu6ZQteoD1uH_o6eEg}{vmt_wvYPQwaHCPnBQOonzw}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=67336515584,
ml.max_open_jobs=20, ml.enabled=true}), reason(failed to ping, tried [3] times, each with maximum [30s] timeout)[{es1}{jbc_qu6ZQteoD1uH_o6eEg}
{vmt_wvYPQwaHCPnBQOonzw}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=67336515584, ml.max_open_jobs=20, ml.enabled=true}
failed to ping, tried [3] times, each with maximum [30s] timeout, {es1}{jbc_qu6ZQteoD1uH_o6eEg}{vmt_wvYPQwaHCPnBQOonzw}
{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=67336515584, ml.max_open_jobs=20, ml.enabled=true} failed to ping, tried [3] times, each with maximum [30s] timeout]]])查看集群中各个节点只能查看到实例2和实例3的，看不到实例1的：
[root@sdk-25 logs]# curl --user elastic:5BheRCDLK12389Sv http://127.0.0.1:9202/_cat/nodes 127.0.0.1 29 32 0 0.23 0.26 0.19 mdi * es3127.0.0.1 29 32 1 0.23 0.26 0.19 mdi - es2 [root@sdk-25 logs]# curl --user elastic:5BheRCDLK12389Sv http://127.0.0.1:9201/_cat/nodes 127.0.0.1 29 32 0 0.23 0.26 0.19 mdi * es3127.0.0.1 29 32 0 0.23 0.26 0.19 mdi - es2 [root@sdk-25 logs]# curl --user elastic:5BheRCDLK12389Sv http://127.0.0.1:9200/_cat/nodes 127.0.0.1 29 32 0 0.21 0.25 0.19 mdi * es3127.0.0.1 29 32 0 0.21 0.25 0.19 mdi - es2
正确给ES集群设置密码的方法是：
一开始3个实例都不要安装x-pack插件设置密码。而是先配置好三个实例，都启动，从是master节点的实例上来安装x-pack插件来来设置密码。
然后其他的节点也都安装x-pack插件，然后重启其他的2个ES实例，但是其他的2个ES实例不需要设置密码
下面是未安装x-pack密码插件时，查看集群的状态:
[root@sdk-25 plugins]# curl http://127.0.0.1:9200/_cat/nodes127.0.0.1 28 32 0 0.06 0.09 0.12 mdi - es1127.0.0.1 42 32 0 0.06 0.09 0.12 mdi * es3127.0.0.1 44 32 0 0.06 0.09 0.12 mdi - es2 [root@sdk-25 plugins]# curl http://127.0.0.1:9201/_cat/nodes127.0.0.1 42 32 0 0.00 0.04 0.10 mdi * es3127.0.0.1 44 32 0 0.00 0.04 0.10 mdi - es2127.0.0.1 28 32 0 0.00 0.04 0.10 mdi - es1 [root@sdk-25 plugins]# curl http://127.0.0.1:9202/_cat/nodes127.0.0.1 28 32 0 0.00 0.04 0.10 mdi - es1127.0.0.1 42 32 0 0.00 0.04 0.10 mdi * es3127.0.0.1 44 32 0 0.00 0.04 0.10 mdi - es2 [root@sdk-25 plugins]# curl 'http://127.0.0.1:9200/_cat/master?v'id host ip node VojUpPevTV2tH56AwJN03g 127.0.0.1 127.0.0.1 es3
可以看到ES3节点是集群中的master，从是ES-master节点的实例上来安装x-pack插件来给次实例设置密码。然后再给其他的2个ES实例也安装x-pack插件，并且重启其他的2个ES服务。
但是都不需要给这2个ES实例设置密码
此处不太理解：一旦重启ES实例，原先是maser节点的ES会切换到其他的节点上，那在原先的master节点上重启后设置的密码数据如何才能被复制到其他的节点上呢？？？？
©著作权归作者所有：来自51CTO博客作者wjw555的原创作品，如需转载，请注明出处，否则将追究法律责任