第一步:设置机器免密登录,并设置主机名和hosts文件
1、配置hostname主机名
vim /etc/hostname
三台机器分别是 hadoop1,、hadoop2、hadoop3
2、配置hosts文件
vim /etc/hosts
192.168.31.146 hadoop1
192.168.31.147 hadoop2
192.168.31.148 hadoop3
3、配置三台服务器免密登录
scp /etc/hosts root@hadoop2
scp /etc/hosts root@hadoop3
第二步:配置环境变量
2.1编辑配置文件
vim /etc/profile
export JAVA_HOME=/opt/env/jdk1.8.0_202
export HADOOP_HOME=/opt/env/hadoop-3.2.3
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export PATH=.:$JAVA_HOME/bin:$PATH:$HADOOP_HOME/bin
同步环境变量
scp /etc/profile root@hadoop2
scp /etc/profile root@hadoop3
第三步:下载安装包并解压
3.1、下载
cd /opt/env
wget https://downloads.apache.org/hadoop/core/hadoop-3.2.3/hadoop-3.2.3.tar.gz
tar -xf hadoop-3.2.3.tar.gz
3.2、创建对应目录
mkdir -p $HADOOP_HOME/tmp $HADOOP_HOME/hdfs $HADOOP_HOME/hdfs/data $HADOOP_HOME/hdfs/name $HADOOP_HOME/logs
第四步配置
4.1、配置hadoop-env.sh
vim $HADOOP_HOME/etc/hadoop/hadoop-env.sh
export JAVA_HOME=/opt/env/jdk1.8.0_202
export HADOOP_HOME=/opt/env/hadoop-3.2.3
export HDFS_NAMENODE_USER=root
export HDFS_DATANODE_USER=root
export HDFS_SECONDARYNAMENODE_USER=root
export YARN_RESOURCEMANAGER_USER=root
export YARN_NODEMANAGER_USER=root
4.2
vim $HADOOP_HOME/etc/hadoop/core-site.xml
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://hadoop1:9000</value>
<description>HDFS的URI,文件系统://namenode标识:端口号</description>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/opt/hadoop-3.2.1/tmp</value>
<description>namenode上本地的hadoop临时文件夹</description>
</property>
</configuration>
4.3
vim $HADOOP_HOME/etc/hadoop/hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>3</value>
<description></description>
</property>
</configuration>
4.4
vim $HADOOP_HOME/etc/hadoop/mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
4.5
vim $HADOOP_HOME/etc/hadoop/yarn-site.xml
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
4.6、 编辑workers文件并添加内容
vim $HADOOP_HOME/etc/hadoop/workers
hadoop1
hadoop2
hadoop3
第五步:打包配置好的hadoop然后拷贝到对应机器上
tar cf hadoop-3.2.3.tar hadoop-3.2.3
scp hadoop-3.2.3.tar root@hadoop2:/opt/env
scp hadoop-3.2.3.tar root@hadoop3:/opt/env
ssh root@hadoop2 "tar -xf /opt/env/hadoop-3.2.3.tar -C /opt/env/"
ssh root@hadoop3 "tar -xf /opt/env/hadoop-3.2.3.tar -C /opt/env/"
第六步:格式化hdfs
$HADOOP_HOME/bin/hdfs namenode -format hadoop_cluster
第七步:启动
$HADOOP_HOME/sbin/start-all.sh
正常启动会打印如下
Starting namenodes on [hadoop1]
Starting datanodes
Starting secondary namenodes [hadoop1]
Starting resourcemanager
Starting nodemanagers
jps查看hadoop1节点上进程
SecondaryNameNode
ResourceManager
DataNode
NodeManager
NameNode
其他节点上进程
DataNode
NodeManager
停止
$HADOOP_HOME/sbin/stop-all.sh
第八步:如果格式化失败,需要重新格式化,需要删除对应的额数据
分别在三台机器上执行
rm -rf $HADOOP_HOME/tmp/* $HADOOP_HOME/hdfs/* $HADOOP_HOME/hdfs/data/* $HADOOP_HOME/hdfs/name/* $HADOOP_HOME/logs/*
第八步:访问
6.4、启动访问
http://192.168.137.22:9870 主节点(nameNode information 存在于主节点上 )
http://192.168.137.22:8088/cluster 主节点(All Applications 集群运行状态,以及任务执行服务)
http://192.168.137.(23|24|25):8042/node 数据节点(NodeManager 数据节点管理服务,存在于DataNode数据节点上)
http://192.168.137.(23|24|25):9864/datanode.html 数据节点(DataNode,数据存储, 存在于DataNode数据节点上)
http://hadoop1:9870
http://hadoop1:8088
http://hadoop2:8042
http://hadoop2:9864
http://hadoop3:8042
http://hadoop3:9864

