[ACCEPTED]-Kafka single consumer failure in a group-apache-kafka
The Rebalance will happen automatically 16 which can be set in the consumer config 15 ( zookeeper.session.timeout.ms ). As per 14 the documentaion
zookeeper.session.timeout.ms : ZooKeeper session timeout. If 13 the consumer fails to heartbeat to zookeeper 12 for this period of time it is considered 11 dead and a rebalance will occur. default 10 value is 6000 ms
The other live consumer 9 in the same group will start to recieve 8 the message after the timeout interval.
Configure 7 this timeout value as per your requirements.
Also 6 some more info from the kafka documentation:
Consumer 5 rebalancing fails (you will see ConsumerRebalanceFailedException): This 4 is due to conflicts when two consumers are 3 trying to own the same topic partition. The 2 log will show you what caused the conflict 1 (search for "conflict in ").
- If your consumer subscribes to many topics and your ZK server is busy, this could be caused by consumers not having enough time to see a consistent view of all consumers in the same group. If this is the case, try Increasing rebalance.max.retries and rebalance.backoff.ms.
- Another reason could be that one of the consumers is hard killed. Other consumers during rebalancing won't realize that consumer is gone after zookeeper.session.timeout.ms time. In the case, make sure that rebalance.max.retries * rebalance.backoff.ms > zookeeper.session.timeout.ms.
More Related questions
We use cookies to improve the performance of the site. By staying on our site, you agree to the terms of use of cookies.