Agfdhyk

I have been doing some performance tests with kafka cluster for my project. I have a question regarding the send call and the 'acks' property of producer. I observed below numbers with below invocation of send call. This is a simple fire and forget call.

producer.send(record); // fire and forget call

The topic has 5 partitions and I see below results with different acks value and replication factor. The kafka cluster has 5 nodes running with default values and using local disk

acks             Replication factor=1              Replication factor=3

0                  1330k msgs/sec                    1260k msgs/sec

1                  1220k msgs/sec                    1200k msgs/sec

-1(all)            1220k msgs/sec                    325k msgs/sec  

As you can see as the acks value changes from 0 to all, the producer throughput decreases. What I am not able to understand is that if the producer send call is fire and forget in nature (see above) and producer is not waiting for any acknowledgements then why does the producer throughput drops as we move to stronger acks guarantees?

Any insights into how acks and producer send call works internally in Kakfa would be greatly appreciated.

P.S. I had asked this on kafka users mailing list but didn't get a reply so asking this on SO.

The fact that you haven't a callback in the send method doesn't mean that it's fire and forget at the underlying level.
You have configured the producer with 3 different levels of ack which determine the "fire and forget" status or not.
With acks = 0, it means that the producer send the message but doesn't wait for any acks from the broker; it's the real "fire and forget". So as you can see it provides the higher throughput.
With acks = 1, the producer waits for the ack. This ack is sent by the broker (to which the producer is connected and that hosts the leader replica). It's not "fire and forget" of course.
With acks = -1, the producer waits for the ack. This ack is sent by the broker as above but only after having the messages replicated to all the replica followers on the other brokers. Of course in this case the throughput decrease if you increment the replication factor, because the message needs to be copied by more brokers (min.insync.replicas) before the "leader" broker returns back the ack to the producer.
Notice that with replication factor = 1, the ack = 1 and ack = -1 has same throughput because there is just one replica (the leader) so no need to copy to followers.

This is something about how kafka handling the produce request.
First, KafkaProducer.send is async by default. KafkaProducer has taken the heavily working for batch your produce request and send to broker. The broker will acking with produce response which in turn need wait for the min.insync.replicas from the remote followers. That's the reason.