Ambari2.7.4+HDP3.1.4 离线安装（2）
4. 实现离线安装，更换yum源
4.1. 文件目录展示
4.1.1. http服务方式
[root@master ~]# yum -y install httpd [root@master ~]# service httpd restart Redirecting to /bin/systemctl restart httpd.service [root@master ~]# chkconfig httpd on复制代码
安装完成后，会生成/var/www/html目录（相当于Tomcat的webapps目录）
将之前下的Ambari、HDP、HDP-UTILS三个包放到 /var/www/html 下
[root@master ~]# cd /var/www/html/ [root@master html]# mkdir ambari复制代码
拷贝文件到ambari下面
[root@master html]# cd ambari/ [root@master ambari]# ls ambari-2.7.4.0-centos7.tar.gz HDP-3.1.4.0-centos7-rpm.tar.gz HDP-UTILS-1.1.0.22-centos7.tar.gz [root@master ambari]# tar -zxvf ambari-2.7.4.0-centos7.tar.gz [root@master ambari]# tar -zxvf HDP-3.1.4.0-centos7-rpm.tar.gz [root@master ambari]# mkdir HDP-UTILS [root@master ambari]# tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz -C HDP-UTILS [root@master ambari]# rm -rf ambari--2.7.4.0-centos7.tar.gz HDP-2.6.3.0-centos7-rpm.tar.gz HDP-UTILS-1.1.0.22-centos7.tar.gz [root@master ambari]# ls ambari HDP HDP-UTILS复制代码
4.1.2. nginx服务方式
找到nginx的配置， 在nginx server中的location中增加：
server {
listen 8001;
location / {
root /www//html/ambari/;
autoindex on;
autoindex_localtime on;
autoindex_exact_size off;
}
}复制代码参数说明：
root /data/image/
你需要开启浏览的目录，放访问
http://IP 时候显示的就是/data/image目录下的内容
autoindex_localtime on;
默认为off，显示的文件时间为GMT时间。
改为on后，显示的文件时间为文件的服务器时间
autoindex_exact_size off;
默认为on，显示出文件的确切大小，单位是bytes。
改为off后，显示出文件的大概大小，单位是kB或者MB或者GB
listen 8001;
访问端口号
[root@master ~]# cd /var/www/ [root@master html]# mkdir ambari复制代码
拷贝文件到ambari下面
[root@master html]# cd ambari/ [root@master ambari]# ls ambari-2.7.4.0-centos7.tar.gz HDP-3.1.4.0-centos7-rpm.tar.gz HDP-UTILS-1.1.0.22-centos7.tar.gz [root@master ambari]# tar -zxvf ambari-2.7.4.0-centos7.tar.gz [root@master ambari]# tar -zxvf HDP-3.1.4.0-centos7-rpm.tar.gz [root@master ambari]# mkdir HDP-UTILS [root@master ambari]# tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz -C HDP-UTILS [root@master ambari]# rm -rf ambari--2.7.4.0-centos7.tar.gz HDP-2.6.3.0-centos7-rpm.tar.gz HDP-UTILS-1.1.0.22-centos7.tar.gz [root@master ambari]# ls ambari HDP HDP-UTILS复制代码
访问http://172.29.30.61/ambari/ 看是否能访问
4.2. 制作本地源
4.2.1. 安装本地源制作相关工具（主节点）
# yum install yum-utils createrepo yum-plugin-priorities -y # createrepo ./复制代码
4.2.2. 修改文件里面的源地址（主节点）
注意文件路径，以自己为准
[root@master ambari]# vi ambari/centos7/2.7.4.0-118/ambari.repo #VERSION_NUMBER=2.7.4.0-118 [ambari-2.7.4.0] #json.url = http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json name=ambari Version - ambari-2.7.4.0 baseurl=http://172.29.30.61:8001/ambari/centos7/2.7.4.0-118 gpgcheck=1 gpgkey=http://172.29.30.61:8001/ambari/centos7/2.7.4.0-118/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 [root@master ambari]# cp ambari/centos7/2.7.4.0-118/ambari.repo /etc/yum.repos.d/ [root@master ambari]# vi HDP/centos7/3.1.4.0-315/hdp.repo #VERSION_NUMBER=3.1.4.0-315 [HDP-3.1.4.0] name=HDP Version - HDP-3.1.4.0 baseurl=http://172.29.30.61:8001/HDP/centos7/3.1.4.0-315 gpgcheck=1 gpgkey=http://172.29.30.61:8001/HDP/centos7/3.1.4.0-315/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 [HDP-UTILS-1.1.0.22] name=HDP-UTILS Version - HDP-UTILS-1.1.0.22 baseurl=http://172.29.30.61:8001/HDP-UTILS/HDP-UTILS/centos7/1.1.0.22 gpgcheck=1 gpgkey=http://172.29.30.61:8001/HDP-UTILS/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 [root@master ambari]# cp HDP/centos7/3.1.4.0-315/hdp.repo /etc/yum.repos.d/复制代码
以上就已经创建好了
使用yum的命令清一下缓存
# yum clean all # yum makecache # yum repolist复制代码
4.2.3. 将创建好的源文件拷贝到子节点（主节点）
#cd /etc/yum.repos.d # scp ambari.repo xxx2.hadoop.com:/etc/yum.repos.d/ # scp ambari.repo xxx3.hadoop.com:/etc/yum.repos.d/ # scp ambari.repo xxx4.hadoop.com:/etc/yum.repos.d/ # scp ambari.repo xxx5.hadoop.com:/etc/yum.repos.d/ # scp hdp.repo xxx2.hadoop.com:/etc/yum.repos.d/ # scp hdp.repo xxx3.hadoop.com:/etc/yum.repos.d/ # scp hdp.repo xxx4.hadoop.com:/etc/yum.repos.d/ # scp hdp.repo xxx5.hadoop.com:/etc/yum.repos.d/复制代码
5. 安装ambari-server
这里介绍两种模式，一种是默认postgresql数据库的安装方式，还有就是mysql方式，这里我们用到的是mysql，大家根据自身选一种即可
无论是用哪种，首先都要安装ambari-server
# yum -y install ambari-server复制代码
5.1. 默认数据库PostgreSQL安装方式（主节点）
[root@master yum.repos.d]# ambari-server setup Using python /usr/bin/python Setup ambari-server Checking SELinux... SELinux status is 'disabled' Customize user account for ambari-server daemon [y/n] (n)? n Adjusting ambari-server permissions and ownership... Checking firewall status... Checking JDK... [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8 [2] Custom JDK Enter choice (1): 2 WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts. WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts. Path to JAVA_HOME: /app/tools/java/jdk1.8.0_201 Validating JDK on Ambari Server...done. Completing setup... Configuring database... Enter advanced database configuration [y/n] (n)? n Configuring database... Default properties detected. Using built-in database. Configuring ambari database... Checking PostgreSQL... Running initdb: This may take up to a minute. Initializing database ... OK About to start PostgreSQL Configuring local database... Configuring PostgreSQL... Restarting PostgreSQL Creating schema and user... done. Creating tables... done. Extracting system views... ambari-admin-2.7.4.0-118.jar ........... Adjusting ambari-server permissions and ownership... Ambari Server 'setup' completed successfully. 启动ambari [root@master yum.repos.d]# ambari-server start Using python /usr/bin/python Starting ambari-server Ambari Server running with administrator privileges. Organizing resource files at /var/lib/ambari-server/resources... Ambari database consistency check started... Server PID at: /var/run/ambari-server/ambari-server.pid Server out at: /var/log/ambari-server/ambari-server.out Server log at: /var/log/ambari-server/ambari-server.log Waiting for server start........................................... Server started listening on 8080 DB configs consistency check: no errors and warnings were found. Ambari Server 'start' completed successfully.复制代码
成功启动后在浏览器输入Ambari地址：
http://172.29.30.61:8080 即可看到页面
5.2. MySql安装方式（主节点）
5.2.1. 安装MySql
[root@master ~]# wget http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm [root@master ~]# rpm -ivh mysql-community-release-el7-5.noarch.rpm [root@master ~]# yum install mysql-community-serve复制代码
5.2.2. 启动mysql，设置开机启动
[root@master ~]# service mysqld start [root@master ~]# vi /etc/rc.local #添加service mysqld start复制代码
5.2.3. 登录进mysql，初始化设置root 密码
[root@master ~]# mysql -uroot
设置登录密码
mysql> set password for 'root'@'localhost' = password('yourPassword');
添加远程登录用户
mysql> grant all privileges on *.* to 'root'@'%' identified by 'yourPassword';
远程登录
#mysql -uroot -h ip(远程的ip地址) -p复制代码5.2.4. 登录mysql，执行以下语句
CREATE DATABASE ambari; use ambari; CREATE USER 'ambari'@'%' IDENTIFIED BY 'ambari'; GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%'; CREATE USER 'ambari'@'localhost' IDENTIFIED BY 'ambar'; GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'localhost'; CREATE USER 'ambari'@'master' IDENTIFIED BY 'ambari'; GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'master'; FLUSH PRIVILEGES; source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql show tables; use mysql; select Host User Password from user where user='ambari'; CREATE DATABASE hive; use hive; CREATE USER 'hive'@'%' IDENTIFIED BY 'hive'; GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%'; CREATE USER 'hive'@'localhost' IDENTIFIED BY 'hive'; GRANT ALL PRIVILEGES ON *.* TO 'hive'@'localhost'; CREATE USER 'hive'@'master' IDENTIFIED BY 'hive'; GRANT ALL PRIVILEGES ON *.* TO 'hive'@'master'; FLUSH PRIVILEGES; CREATE DATABASE oozie; use oozie; CREATE USER 'oozie'@'%' IDENTIFIED BY 'oozie'; GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'%'; CREATE USER 'oozie'@'localhost' IDENTIFIED BY 'oozie'; GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'localhost'; CREATE USER 'oozie'@'master' IDENTIFIED BY 'oozie'; GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'master'; FLUSH PRIVILEGES;复制代码
5.2.5. 建立mysql与ambari-server的jdbc连接
[root@master yum.repos.d]# yum install mysql-connector-java [root@master yum.repos.d]# ls /usr/share/java [root@master yum.repos.d]# cp /usr/share/java/mysql-connector-java.jar /var/lib/ambari-server/resources/mysql-jdbc-driver.jar [root@master yum.repos.d]# vi /etc/ambari-server/conf/ambari.properties 添加server.jdbc.driver.path=/usr/share/java/mysql-connector-java.jar复制代码
5.2.6. 设置ambari-server
[root@master yum.repos.d]# ambari-server setup Using python /usr/bin/python Setup ambari-server Checking SELinux... SELinux status is 'enabled' SELinux mode is 'permissive' WARNING: SELinux is set to 'permissive' mode and temporarily disabled. OK to continue [y/n] (y)? y Customize user account for ambari-server daemon [y/n] (n)? y Enter user account for ambari-server daemon (root):ambari #ambari-server 账号。如果直接回车就是默认选择root用户 如果输入已经创建的用户就会显示： Adjusting ambari-server permissions and ownership... Checking firewall status... WARNING: iptables is running. Confirm the necessary Ambari ports are accessible. Refer to the Ambari documentation for more details on ports. OK to continue [y/n] (y)? y Checking JDK... #设置JDK。输入：2 [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8 [2] Custom JDK Enter choice (1): 2 #输入你自己的jdk位置/usr/java/jdk1.8.0_201-amd64 WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts. WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts. Path to JAVA_HOME: /usr/java/jdk1.8.0_201-amd64 Validating JDK on Ambari Server...done. Check JDK version for Ambari Server... JDK version found: 8 Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server. Checking GPL software agreement... GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? n Completing setup... Configuring database... Enter advanced database configuration [y/n] (n)? y #数据库配置。选择：y Configuring database... Choose one of the following options: [1] - PostgreSQL (Embedded) [2] - Oracle [3] - MySQL / MariaDB [4] - PostgreSQL [5] - Microsoft SQL Server (Tech Preview) [6] - SQL Anywhere [7] - BDB Enter choice (1): 3 #选择mysql数据库类型。输入：3 # 设置数据库的具体配置信息，根据实际情况输入，如果和括号内相同，则可以直接回车。 Hostname (localhost): Port (3306): Database name (ambari): Username (ambari): ambari Enter Database Password (bigdata): ambari Re-enter password: ambari Configuring ambari database... Configuring remote database connection properties... WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql Proceed with configuring remote database connection properties [y/n] (y)? y Extracting system views... ambari-admin-2.7.4.0.118.jar .... Ambari repo file contains latest json url http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json, updating stacks repoinfos with it... Adjusting ambari-server permissions and ownership... Ambari Server 'setup' completed successfully. [root@master yum.repos.d]# ambari-server start Using python /usr/bin/python Starting ambari-server Ambari Server running with administrator privileges. Organizing resource files at /var/lib/ambari-server/resources... Ambari database consistency check started... Server PID at: /var/run/ambari-server/ambari-server.pid Server out at: /var/log/ambari-server/ambari-server.out Server log at: /var/log/ambari-server/ambari-server.log Waiting for server start........................................... Server started listening on 8080 DB configs consistency check: no errors and warnings were found. Ambari Server 'start' completed successfully.复制代码
如果启动失败建议查看
日志在/var/log/ambari-server/ambari-server.log里面
重置ambari-server
[root@master ~]# ambari-server stop [root@master ~]# ambari-server reset [root@master ~]# ambari-server setup复制代码
如果选择的是mysql方式，就需要先执行上面的语句，然后手动将mysql里面创建的数据库进行删除
[root@master ~]# mysql -uroot -p mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | ambari | | hive | | oozie | | performance_schema | +--------------------+ rows in set (0.00 sec) mysql> drop database ambari; mysql> drop database hive; mysql> drop database oozie;复制代码
如果在安装的过程中出现了错误，又想重新安装，可以在ambari-server开启的情况下，执行下面的语句来移除已安装的包，然后再通过不同的情况选择上面两种方式的一种对ambari-server进行重置
python /usr/lib/python2.6/site-packages/ambari_agent/HostCleanup.py --silent复制代码
5.3. Mysql驱动检查
为了后续安装HDP时hive可以正常安装，此处需要检查mysql驱动是否符合要求
第一：数据库、用户名、密码是否正确，安装驱动
yum install mysql-connector-java复制代码
第二：mysql驱动是否存在
/usr/share/java/mysql-connector-java.jar 是否有mysql驱动
vim /etc/ambari-server/conf/ambari.properties
配置文件添加server.jdbc.driver.path=/usr/share/java/mysql-connector-java.jar
第三：设置ambari的mysql驱动
在master主机命令行界面执行即可（前提是上面第二条件都满足）
ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar复制代码
6. 安装配置部署HDP集群
6.1. 登录过程
如果你以上安装成功
输入主机ip：8080则会看到如下界面
账户：admin 密码：admin
6.2. 安装向导
6.2.1. 配置集群名字
6.2.2. 选择版本并修改为本地源地址
Centos的选择rathat7，HDP和DHP-UTILS的url地址填上之前改好的
使用当前安装包需要修改HDP，HDP-3.1.4.0
ambari确认过了就next继续了
6.2.3. 安装配置
上传之前存好的秘钥文件id_rsa
6.2.4. 确认安装ambari的agent
确认安装ambari的agent，并检查出现的问题，这个地方经常出现错误
如图就出现了错误，点击Failed的查看错误日志
我这里的错误是ambari的8040等端口无法访问的问题，我放开了8000-9000的端口就可以了，我之前也遇到很多其他的问题，具体问题具体分析，多查谷歌，百度，国外的网站更容易解决问题
检查主机可能会发现之前漏下的问题，比如说我这里防火墙没关他就会出现提示
检查无误，NEXT→通过即可
如果这个步骤失败了错误，记得多看日志，多找问题，如果还不行的话，回档咯
[root@master ~]# # ambari-server stop #停止命令 [root@master ~]# # ambari-server reset #重置命令 [root@master ~]# # ambari-server setup #重新设置 [root@master ~]# # ambari-server start #启动命令复制代码
6.2.5. 大数据服务组件安装
勾选你所需要的
6.2.6. 节点分配
6.2.7. 分配主从
6.2.8. 安装配置
hive和oozie的数据库用户密码填上之前创建好的
如果安装了hive，ooize等，需要修改成我们本地建好的库，jdbc-mysql也要配置好
6.2.9. 概况部署
警告这里我这就忽略掉了，后期我们再修复
7. 集群配置
7.1. hive配置更新
修改hive配置，关闭acid
hive.strict.managed.tables=false hive.create.as.insert.only=false metastore.create.as.acid=false复制代码
新增hive-site配置，关闭客户端校验
hive.metastore.client.capability.check = false复制代码
设置数据库路径访问权限（KUDU使用）
hdfs dfs -chmod -R 777 /warehouse/tablespace/managed/hive复制代码
7.2. hdfs配置更新
增加配置项，方便查看文件，新增默认用户
hadoop.http.staticuser.user=hdfs复制代码
7.3. Spark配置更新
关闭hiveacid后，访问metastore值需要配置，该配置项为spark度metastore文件默认配置，原值为spark，读自己的
metastore.catalog.default=hive
作者：怀瑾握瑜的嘉与嘉
链接：https://juejin.cn/post/7031337758010540039