Both Apache Kafka and Storm complement each other for accurate real-time streaming analysis of the big data in very little time. Kafka-Storm integration allows developers to easily publish streams of data using Storm topologies and run continuously.
Three main classes that oversee the integration of Kafka-Storm API are -
- BrokerHosts
The BrokerHosts interface consists of two implementations - ZkHosts and StaticHosts. ZkHosts offers a fast way for tracking and maintaining the details of Kafka brokers 9n ZooKeeper while StaticHosts manually select the Kafka brokers.
The signature of ZkHosts is -
public ZkHosts(String brokerZkStr, String brokerZkPath)
public ZkHosts(String brokerZkStr)
where brokerZkStr refers to the ZooKeeper host and brokerZkPath is the ZooKeeper path for maintaining the Kafka broker and its details.
- KafkaConfig API
This API defines the configuration settings of the Kafka cluster and its signature is -
public KafkaConfig(BrokerHosts hosts, string topic)
where hosts are the BrokerHosts (ZkHosts / StaticHosts) and the topic corresponds to the name of the topic.
- SpoutConfig API
This API is an extension of KafkaConfigAPI and is responsible for storing additional ZooKeeper information. Its signature is -
public SpoutConfig(BrokerHosts hosts, string topic, string zkRoot, string id)
where zkRoot is the ZooKeeper root path and id corresponds to the state of the offsets that identifies the spout.
