KAFKA单机版安装步骤

发表:2018-02-14 15:22:29

一、关闭SELINUX、开启防火墙9092端口

1、关闭SELINUX

vi /etc/selinux/config
#SELINUX=enforcing #注释掉
#SELINUXTYPE=targeted #注释掉
SELINUX=disabled #增加

setenforce 0 #使配置立即生效


2、配置防火墙,开启9092端口
vi /etc/sysconfig/iptables #编辑防火墙配置文件

# Firewall configuration written by system-config-firewall
# Manual customization of this file is not recommended.
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 9092 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
COMMIT


service iptables restart #最后重启防火墙使配置生效


二、安装JDK

kafka运行需要JDK支持

1、下载JDK

http://soft.beauty-soft.net/lib/jdk-7u9-linux-i586.tar.gz

注意:直接复制到下载工具进行下载,版本请使用JDK7,JDK8有可能不兼容kafka_2.11-0.8.2.1

下载完成之后,上传到/usr/local/src目录下

2、安装JDK1

tar zxvf jdk-7u9-linux-i586.tar.gz
cd jdk1.7.0_09/
cp -r jdk1.7.0_09 /usr/java1.7


3、添加JDK到系统环境变量

vi /etc/profile #编辑,在最后添加以下代码1

SCALA_HOME=/usr/local/scala
JAVA_HOME=/usr/java1.7
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
PATH=$JAVA_HOME/bin:$PATH:$SCALA_HOME/bin
export JAVA_HOME CLASSPATH PATH


source /etc/profile #使配置文件立即生效

java -version #查看JDK版本信息


三、安装zookeeper

kafka内置有zookeeper,这里我们将独立安装。步骤如下。
1、下载zookeeper
wget -c http://soft.beauty-soft.net/lib/zookeeper-3.4.3.tar.gz
2、解压并安装
tar zxvf zookeeper-3.4.3.tar.gz
cp -r zookeeper-3.4.3 /usr/local/zookeeper/
cp /usr/local/zookeeper/conf/zoo_sample.cfg /usr/local/zookeeper/conf/zoo.cfg

zoo.cfg配置文件如下。

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPort=2181
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
dataDir=/usr/local/zookeeper/zookeeper-data
dataLogDir=/usr/local/zookeeper/logs


创建相应目录
mkdir -p /usr/local/zookeeper/zookeeper-data
mkdir -p /usr/local/zookeeper/logs


四、安装kafka

1、下载kafka

wget -c http://soft.beauty-soft.net/lib/kafka_2.10-0.10.2.0.tgz

解压
tar -xzvf kafka_2.10-0.10.2.0.tgz

复制到安装目录
cp -r  kafka_2.10-0.10.2.0 /usr/loal/kafka

2、配置kafka

[root@cloud ~]# cat /usr/local/kafka/config/server.properties
broker.id=0
port=9092
host.name=localhost
log.dirs=/usr/local/kafka/log/kafka
zookeeper.connect=localhost:2181


四、创建启动脚本

[root@cloud ~]# cat /home/sh/start_kafka.sh
#!/bin/sh
#启动zookeeper
/usr/local/zookeeper/bin/zkServer.sh start
sleep 3 #等3秒后执行
#启动kafka
/usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/server.properties &



五、测试KAFKA


创建:

kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic accesslog


查看:

kafka-topics.sh --describe --zookeeper localhost:2181
kafka-topics.sh --describe --zookeeper spark.master:2181 --topic accesslog


生产:

kafka-console-producer.sh --broker-list localhost:9092 --topic accesslog --from-beginning



消费:

kafka-console-consumer.sh --zookeeper localhost:2181 --topic accesslog







相关文章