I'm just simply reading IRC messages in a loop, and then running it through a function ( start() ) of mine in another file to parse the data. I've noticed in channels with large amounts of messages running through that sometimes the buffer or loop will lag, and then there will be more than one message stuck in the buffer.. Is there something wrong? Or is that what happens when too many people all message at once? Because it was screwing up my code in start(), as I have yet to account for the buffer being overfilled (I just assumed it would only send one message per loop iteration).
Might this be an issue from converting the bytes to UTF8?
use tokio::net::TcpStream;
use tokio::io::{AsyncWriteExt, AsyncReadExt};
use std::str::from_utf8;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut stream = TcpStream::connect("irc.chat.twitch.tv:6667").await?;
stream.write(b"PASS password\r\n NICK nickname\r\n").await?;
stream.write(b"CAP REQ :twitch.tv/commands twitch.tv/tags\r\n").await?;
stream.write(b"JOIN #channel\r\n").await?;
stream.flush().await?;
let mut buf = [0; 2500];
loop {
let n = stream.read(&mut buf).await?;
if from_utf8(&buf).unwrap() == "" {
break;
}
let data = message::start(from_utf8(&buf[0..n]).unwrap_or(&"").trim());
println!("{}: {}", data.get("display-name").unwrap_or(&""), data.get("message").unwrap_or(&""));
//println!("{:?}", from_utf8(&buf[0..n]).unwrap().trim());
buf = [0;2500];
}
Ok(())
}
The exact details of how much data read will place in the buffer depend on a whole bunch of varied factors. It will vary across operating systems, network conditions, and load on the machine just to name a few.
If message::start is assuming it's getting a full single message this code is simply incorrect. You should continue reading from the stream until you have a full message. You will also need to check if multiple messages are in the buffer from a single read and pass each message to start separately, which sounds like the problem you're running into.
To put it another way: TcpStream has no idea how long your messages are going to be, it can only give you the data it has available at the moment. If that data is more or less than a single complete message you have to handle that logic yourself
Right thanks, I was assuming it would just send one message in but the buffer has no idea what that means when it's receiving so much more in a high-traffic situation, so I was wrong.
This is what gets sent in a perfect situation, but in a busy channel, multiple versions of these end up getting mashed together, I was turning it into a toml-format string and deserializing it but it's getting hard to do when they're combined.. Are there any libraries you'd recommend for parsing as such?
I guess I could just make sure only one message gets sent in like you said.