Module mz_kafka_util::client

Helpers for working with Kafka’s client API.

Structs

• BrokerRewrite
  Rewrites a broker address.
• MzClientContext
  A ClientContext implementation that uses tracing instead of log macros.
• TimeoutConfig
  Configurable timeouts for Kafka connections.
• TunnelingClientContext
  A client context that supports rewriting broker addresses.

Enums

• BrokerRewriteHandle 🔒
• GetPartitionsError
  The error returned by get_partitions.
• MzKafkaError
  A structured error type for errors reported by librdkafka through its logs.
• TunnelConfig
  Tunneling clients used for re-writing ports / hosts

Constants

• DEFAULT_FETCH_METADATA_TIMEOUT
  A reasonable default timeout when fetching metadata or partitions.
• DEFAULT_KEEPALIVE
  Default to true as they have no downsides https://github.com/confluentinc/librdkafka/issues/283.
• DEFAULT_METADATA_FETCH_INTERVAL
  The interval we will fetch metadata from, unless overridden by the source.
• DEFAULT_PROGRESS_RECORD_FETCH_TIMEOUT
  The timeout for reading records from the progress topic. Set to something slightly longer than the idle transaction timeout (60s) to wait out any stuck producers.
• DEFAULT_SOCKET_CONNECTION_SETUP_TIMEOUT
  The rdkafka default.
• DEFAULT_SOCKET_TIMEOUT
  The rdkafka default.
• DEFAULT_TOPIC_METADATA_REFRESH_INTERVAL
  A reasonable default timeout when refreshing topic metadata. This is configured at a source level.
• DEFAULT_TRANSACTION_TIMEOUT
  The rdkafka default.

Functions

• create_new_client_config
  Build a new rdkafka ClientConfig with its log_level set correctly based on the passed through tracing::Level. This level should be determined for target: "librdkafka".
• create_new_client_config_simple
  A simpler version of create_new_client_config that defaults the log_level to INFO and should only be used in tests.
• get_partitions
  Retrieve number of partitions for a given topic using the given client

Type Aliases

• PartitionId
  Id of a partition in a topic.