jdk, zookeeper 설치

jdk 8버전, zookeeper 3.5.9버전 설치

1. 설치파일 다운로드

jdk 다운로드 : https://github.com/AdoptOpenJDK/openjdk8-upstream-binaries/releases

zookeeper 다운로드 : https://www.apache.org/dyn/closer.lua/zookeeper/zookeeper-3.5.9/apache-zookeeper-3.5.9-bin.tar.gz

dwhdfad01, 02, 03 서버 똑같이 배포

2) ~/usr/local 경로에 압축 해제 (서버 3대에 똑같이 배포)

$ tar -xvzf OpenJDK8U-jdk_x64_linux_8u312b07.tar.gz
$ cd openjdk-8u312-b07

3) java 환경변수 설정, 적용 (zookeeper, hadoop, hive, spark 미리 다 추가)

$ vi ~/.bash_profile

# java
export JAVA_HOME=$HOME/usr/local/openjdk-8u312-b07
export CLASSPATH=$CLASSPATH:$JAVA_HOME/jre/lib/ext:$JAVA_HOME/lib/tools.jar

# python
export PYTHON_HOME=$HOME/usr/local/Python3.9.9

# zookeeper
export ZOOKEEPER_HOME=$HOME/usr/local/apache-zookeeper-3.5.9-bin

# postgresql
export POSTGRES_HOME=$HOME/usr/local/postgres

# hadoop
export HADOOP_HOME=$HOME/usr/local/hadoop-3.3.1
export HADOOP_INSTALL=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export HADOOP_YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/narive"

# hive
export HIVE_HOME=$HOME/usr/local/apache-hive-3.1.2-bin
export HIVE_CONF=$HIVE_HOME/conf
export HIVE_AUX_JARS_PATH=$HIVE_HOME/lib

# spark
export SPARK_HOME=$HOME/usr/local/spark-3.2.0-bin-hadoop3.2
export SPARK_MASTER_HOST=wasadm
export YARN_CONF_DIR=$HADOOP_HOME/etc/hadoop
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop

export LD_LIBRARY_PATH=$PYTHON_HOME/lib:$POSTGRES_HOME/lib:$LD_LIBRARY_PATH

PATH=$PATH:$HOME/.local/bin:$HOME/bin:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HIVE_HOME:$SPARK_HOME/bin:$ZOOKEEPER_HOME/bin:$PYTHON_HOME/bin:$POSTGRES_HOME/bin:$PATH
export PATH

$ source ~/.bash_profile

3) ~/bin 에 java 링크 생성 및 설치 확인

$ cd ~/bin
$ ln -s /home/wasadm/usr/local/openjdk-8u312-b07/bin/java java
$ java -version
openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)

4) zookeeper 압축 해제 (서버 3대에 똑같이 배포)

$ cd ~/usr/local
$ tar -xvzf apache-zookeeper-3.5.9-bin.tar.gz
$ cd apache-zookeeper-3.5.9-bin

5) zookeeper conf 파일 수정

$ cd ~/usr/local/apache-zookeeper-3.5.9-bin/conf
$ cp zoo_sample.cfg zoo.cfg

# 데이터 디렉토리로 사용할 디렉토리 생성
$ mkdir /DATA/zookeeper

$ vi zoo.cfg
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/hntsw/DATA/zookeeper
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
maxClientCnxns=60
maxSessionTimeout=180000
server.1=ip입력:2888:3888
server.2=ip입력:2888:3888
server.3=ip입력:2888:3888
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

6) zookeeper myid 파일 생성 (각 서버별로 아이디가 다르다)

# 01서버 
$ vi /hntsw/DATA/zookeeper/myid
1
# 02서버 
$ vi /hntsw/DATA/zookeeper/myid
2
# 03서버 
$ vi /hntsw/DATA/zookeeper/myid
3

7) zookeeper 링크 파일 실행

# 01,02,03 서버 실행
$ ~/usr/local/apache-zookeeper-3.5.9-bin/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/wasadm/usr/local/apache-zookeeper-3.5.9-bin/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED 

# status 명령어로 Mode를 확인하여 누가 leader고 follower인지 확인 가능
$ ~/usr/local/apache-zookeeper-3.5.9-bin/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/wasadm/usr/local/apache-zookeeper-3.5.9-bin/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: leader