I implemented some tests for TCP client that uses tokio. When I run cargo test tests sometimes OK, sometimes some of them failed (the issue is related only to async test with client.connect() call).
This is my tests:
#[cfg(test)]
mod tests {
use std::io::{Read, Write};
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpListener;
use crate::Client;
use crate::client::types::ClientFlags;
use crate::network::session::types::{ActionFlags, StateFlags};
const HOST: &str = "127.0.0.1";
const PORT: u16 = 1234;
const WRONG_HOST: &str = "1.2.3.4";
const PACKET: [u8; 8] = [1, 2, 3, 4, 5, 6, 7, 8];
#[tokio::test]
async fn test_client_create() {
let client = Client::new();
let reader = &mut *client.reader.lock().await;
assert!(reader.is_none());
let writer = &mut *client.writer.lock().await;
assert!(writer.is_none());
let warden_crypt = &mut *client.warden_crypt.lock().await;
assert!(warden_crypt.is_none());
let client_flags = &mut *client.flags.lock().await;
assert_eq!(&mut ClientFlags::NONE, client_flags);
let input_queue = &mut *client.input_queue.lock().await;
assert!(input_queue.is_empty());
let output_queue = &mut *client.output_queue.lock().await;
assert!(output_queue.is_empty());
let data_storage = &mut *client.data_storage.lock().await;
assert!(data_storage.players_map.is_empty());
let session = &mut *client.session.lock().await;
assert!(session.session_key.is_none());
assert!(session.me.is_none());
assert!(session.warden_module_info.is_none());
assert!(session.config.is_none());
assert!(session.follow_target.is_none());
assert!(session.server_id.is_none());
assert!(session.party.is_empty());
assert_eq!(ActionFlags::NONE, session.action_flags);
assert_eq!(StateFlags::NONE, session.state_flags);
}
#[tokio::test]
async fn test_client_connect() {
let mut client = Client::new();
if let Some(listener) = TcpListener::bind(format!("{}:{}", HOST, PORT)).await.ok() {
client.connect(HOST, PORT).await.ok();
let reader = &mut *client.reader.lock().await;
assert!(reader.is_some());
let writer = &mut *client.writer.lock().await;
assert!(writer.is_some());
}
}
#[tokio::test]
#[should_panic]
async fn test_client_connect_with_wrong_data() {
let mut client = Client::new();
if let Some(listener) = TcpListener::bind(format!("{}:{}", HOST, PORT)).await.ok() {
client.connect(WRONG_HOST, PORT).await.unwrap();
}
}
#[tokio::test]
async fn test_client_read_incoming_data() {
let mut client = Client::new();
if let Some(listener) = TcpListener::bind(format!("{}:{}", HOST, PORT)).await.ok() {
client.connect(HOST, PORT).await.ok();
if let Some((mut stream, _)) = listener.accept().await.ok() {
stream.write(&PACKET).await.unwrap();
client.handle_read().await;
let packet = client.input_queue.lock().await.pop_front().unwrap();
assert_eq!(PACKET.to_vec(), packet[0]);
}
}
}
#[tokio::test]
async fn test_client_write_outcoming_data() {
let mut client = Client::new();
if let Some(listener) = TcpListener::bind(format!("{}:{}", HOST, PORT)).await.ok() {
client.connect(HOST, PORT).await.unwrap();
if let Some((mut stream, _)) = listener.accept().await.ok() {
let mut buffer = Vec::new();
client.output_queue.lock().await.push_back(PACKET.to_vec());
client.handle_write().await;
stream.read(&mut buffer).await.unwrap();
assert_eq!(PACKET.to_vec(), buffer);
}
}
}
}
could somebody help to find what is wrong ? Next tests sometimes fail: test_client_connect_with_wrong_data, test_client_write_outcoming_data and test_client_read_incoming_data
With the tests running in parallel (in separate threads), each thread might try to bind to HOST, PORT concurrently, resulting in a race condition. In that case, if let Some(listener) will silently discard the error and the body of some tests will not get a chance to run.
most common error with test_client_connect_with_wrong_data is next:
running 5 tests
test client::tests::test_client_create ... ok
test client::tests::test_client_connect_with_wrong_data - should panic ... FAILED
test client::tests::test_client_write_outcoming_data ... ok
test client::tests::test_client_read_incoming_data ... ok
test client::tests::test_client_connect ... ok
failures:
---- client::tests::test_client_connect_with_wrong_data stdout ----
note: test did not panic as expected
when I run then cargo test --bin idewave-cli several times, I got error from another test test_client_write_outcoming_data too:
left: `[1, 2, 3, 4, 5, 6, 7, 8]`,
right: `[]`', src\client\mod.rs:435:17
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
failures:
client::tests::test_client_connect_with_wrong_data
client::tests::test_client_write_outcoming_data
test result: FAILED. 3 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
error: test failed, to rerun pass '--bin idewave-cli'
also sometimes I got this error:
running 5 tests
test client::tests::test_client_create ... ok
test client::tests::test_client_connect ... ok
test client::tests::test_client_write_outcoming_data ... ok
test client::tests::test_client_connect_with_wrong_data - should panic ... FAILED
test client::tests::test_client_read_incoming_data ... FAILED
failures:
---- client::tests::test_client_connect_with_wrong_data stdout ----
note: test did not panic as expected
---- client::tests::test_client_read_incoming_data stdout ----
Connected to 127.0.0.1:1234
thread 'client::tests::test_client_read_incoming_data' panicked at 'called `Option::unwrap()` on a `None` value', src\client\mod.rs:416:74
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
failures:
client::tests::test_client_connect_with_wrong_data
client::tests::test_client_read_incoming_data
test result: FAILED. 3 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
just found, tests are runned always in different order. In case test_client_connect_with_wrong_data is last in the order, all tests are OK. Tried multiple times and it was reproduced multiple times. Not sure if this can be a reason.
running 5 tests
test client::tests::test_client_create ... ok
test client::tests::test_client_connect_with_wrong_data - should panic ... ok
test client::tests::test_client_read_incoming_data ... FAILED
test client::tests::test_client_write_outcoming_data ... FAILED
test client::tests::test_client_connect ... FAILED
failures:
---- client::tests::test_client_read_incoming_data stdout ----
thread 'client::tests::test_client_read_incoming_data' panicked at 'Cannot connect: Os { code: 10049, kind: AddrNotAvailable, message: "The requested address is not valid in its context." }', src\client\mod.rs:112:17
---- client::tests::test_client_write_outcoming_data stdout ----
thread 'client::tests::test_client_write_outcoming_data' panicked at 'Cannot connect: Os { code: 10049, kind: AddrNotAvailable, message: "The requested address is not valid in its context." }', src\client\mod.rs:112:17
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
---- client::tests::test_client_connect stdout ----
thread 'client::tests::test_client_connect' panicked at 'Cannot connect: Os { code: 10049, kind: AddrNotAvailable, message: "The requested address is not valid in its context." }', src\client\mod.rs:112:17
failures:
client::tests::test_client_connect
client::tests::test_client_read_incoming_data
client::tests::test_client_write_outcoming_data
test result: FAILED. 2 passed; 3 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
also I tried to set 0 port for listener and 1234 port for client.connect
if let Some(listener) = TcpListener::bind(format!("{}:{}", HOST, 0)).await.ok() {
client.connect(HOST, 1234).await.unwrap();
}
and got error:
---- client::tests::test_client_write_outcoming_data stdout ----
thread 'client::tests::test_client_write_outcoming_data' panicked at 'Cannot connect: Os { code: 10061, kind: ConnectionRefused, message: "No connection could be made because the target machine actively refused it." }', src\client\m
od.rs:112:17
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
---- client::tests::test_client_connect stdout ----
thread 'client::tests::test_client_connect' panicked at 'Cannot connect: Os { code: 10061, kind: ConnectionRefused, message: "No connection could be made because the target machine actively refused it." }', src\client\mod.rs:112:17
Well, if you have the OS pick a random port, then you can't just keep using the port 1234. The test would only succeed if the random choice happened to be 1234. You can access the port it chose using TcpListener::local_addr.
#[tokio::test]
async fn test_client_write_outcoming_data() {
let mut client = Client::new();
// PORT == 0 here
if let Some(listener) = TcpListener::bind(format!("{}:{}", HOST, PORT)).await.ok() {
// connect to random port
let local_addr = listener.local_addr().unwrap();
client.connect(HOST, local_addr.port()).await.ok();
client.output_queue.lock().await.push_back(PACKET.to_vec());
if let Some((mut stream, _)) = listener.accept().await.ok() {
let mut buffer = Vec::new();
// here writer.write(&packet).await; send not-empty packet
client.handle_write().await;
// but stream return empty buffer here
stream.read(&mut buffer).await.unwrap();
assert_eq!(PACKET.to_vec(), buffer);
}
}
}
and seems like it helped with random test fail, but now stream not work. It not read or write. So I debugged and sure that inside client.handle_write().await; I have packet, but next line stream.read(&mut buffer).await.unwrap(); not work. It just return empty buffer. Could you help on this ?