I've been looking into switching over from Kafka to Red Panda and was wondering if anyone had experience with using Kafka producers to connect to Red Pandas in Rust. I can send strings to Red Panda, however any data that I serialize using serde_json
is not received. I can however successfully send it to Kafka. My understanding is that it should be exactly the same, however if I swap out the Kafka brokers to my Red Panda brokers in the producer I do not receive the JSON data in Red Panda. No errors are thrown. I am using the rdkafka
crate and the BaseProducer
. The code is similar to the following:
let producer: BaseProducer = ClientConfig::new()
.set("bootstrap.servers", "LISTOFBROKERS")
.set("message.timeout.ms", "5000")
.create()
.expect("Producer creation error");
// works
producer
.send(
BaseRecord::to("topic")
.key(&"some_key")
.payload(&"some_value"),
)
.expect("failed to send message");
// Works with Kafka, not Red Panda, requires poll
let json_data = serde_json::to_string(&custom_strct).unwrap();
producer
.send(
BaseRecord::to("topic")
.key("some_key")
.payload(&json_data),
).expect("Failed to Send Message");
producer.poll(Duration::from_millis(100));