Module rdkafka::producer::base_producer

source ·
Expand description

Low-level Kafka producers.

For more information about the producers provided in rdkafka, refer to the producer module documentation.

§BaseProducer

The BaseProducer is a low level Kafka producer designed to be as similar as possible to the underlying C librdkafka producer, while maintaining a safe Rust interface.

Production of messages is fully asynchronous. The librdkafka producer will take care of buffering requests together according to configuration, and to send them efficiently. Once a message has been produced, or the retry count reached, a callback function called delivery callback will be called.

The BaseProducer requires a ProducerContext which will be used to specify the delivery callback and the DeliveryOpaque. The DeliveryOpaque is a user-defined type that the user can pass to the send method of the producer, and that the producer will then forward to the delivery callback of the corresponding message. The DeliveryOpaque is useful in case the delivery callback requires additional information about the message (such as message id for example).

§Calling poll

To execute delivery callbacks the poll method of the producer should be called regularly. If poll is not called, or not often enough, a RDKafkaErrorCode::QueueFull error will be returned.

§ThreadedProducer

The ThreadedProducer is a wrapper around the BaseProducer which spawns a thread dedicated to calling poll on the producer at regular intervals, so that the user doesn’t have to. The thread is started when the producer is created, and it will be terminated once the producer goes out of scope.

A RDKafkaErrorCode::QueueFull error can still be returned in case the polling thread is not fast enough or Kafka is not able to receive data and acknowledge messages quickly enough. If this error is returned, the caller should wait and try again.

Re-exports§

Structs§