Readdir/glob only works once

help
1

Hi all,

I'm facing a strange problem trying to read entries in a directory.

I'm very new to rust so this problem is probably related to me not fully understanding the language :frowning:

Anyway,
I've got the following code (I haven't included the crates and such that are required to compile this code and it's partially snippets from the real code so it may not compile):

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    tracing_subscriber::registry()
        .with(
            tracing_subscriber::EnvFilter::try_from_default_env()
                .unwrap_or_else(|_| "example_websockets=debug,tower_http=error".into()),
        )
        .with(tracing_subscriber::fmt::layer())
        .init();
	
    let (tx, rx) = channel::<String>(1);

    let state: AppState = AppState {
        recv: Arc::new(Mutex::new(rx)),
        send: tx,
    };

    // Adding the API functions
    let router: Router = Router::new()
        .route("/func", post(program))
	.layer(
            TraceLayer::new_for_http()
                .make_span_with(DefaultMakeSpan::default().include_headers(true)),
        )
        .layer(ServiceBuilder::new().layer(CorsLayer::very_permissive()))
        .with_state(state);
	
	 let app: Router = Router::new().nest("/api", router);

    println!("Starting server on port: {}", port);

    let server_string: String = format!("127.0.0.1:{:?}", port);

    let listener: tokio::net::TcpListener = tokio::net::TcpListener::bind(server_string)
        .await
        .unwrap();

    axum::serve(
        listener,
        app.into_make_service_with_connect_info::<SocketAddr>(),
    )
    .await
    .unwrap();
    
    Ok(())
}


async fn func(State(state): State<AppState>) -> Response {
    find_file("some_path");
    StatusCode::OK.into_response()
}

async fn find_file(path: &String, match_string: &String) -> Result<String, Box<dyn Error>> {

    let p = Path::new(path);

    if p.is_dir() {
        let mut entries = fs::read_dir(p).await?;

        while let Ok(Some(entry)) = entries.next_entry().await {
            println!("Entry: {:?}", entry.path());
	}
	 } else {
        println!("{:?} is not a directory", p);
    }
    
    Ok(())
}

This code starts a web server on port localhost::8081 and waits for input.

To execute the code, I use the following:
curl -v http://localhost:8081/api/program --data "func" --header "Content-type: application/json"

The idea is that when you call the program correctly you should get a listing of all files in a directory.

The problem that I have is with the find_file function. It works perfectly the first time I call the it, but the second time it stubbornly refuses to read anything from the directory.
I've also tried this with glob but I get exactly the same behavior. It also doesn't matter if the function is async or not.

I've been struggling with this for some time and I really don't understand what's going on here.

I generally assume that if a function works once, it works other times as well, but I must be missing something here.

I'm using the 2021 edition of Rust (according to my cargo.toml file) pasted below:

[package]
name = "program"
version = "0.1.0"
edition = "2021"
publish = false
description = "My program"

[dependencies]
libc = "0.2"
ihex = "3.0"


tokio = { version = "1.28", features = ["full"] }
tokio-tungstenite = "0.21"
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
headers = "0.4"
axum = { version = "0.7.5", features = ["ws"] }
axum-extra = { version = "0.9.3", features = ["typed-header"] }
serde = { version = "1.0.147", features = ["derive"] }
serde_json = "1.0.91"
tower-http = { version = "0.5.0", features = ["fs", "trace", "cors"] }
tower = { version = "0.4", features = ["util"] }
chrono = { version = "0.4", features = ["serde"] }
lazy_static = "1.4.0"
# uuid = "1.3.1"
libffi = "3.2.0"
thiserror = "1.0.61"
clap = { version = "4.5.10", features = ["derive"] }
config = "0.14.0"
flate2 = "1.0.30"
tar = "0.4.41"
rand = "0.8.5"
openssl = "0.10.66"
aes-gcm = "0.10.3"
crc32fast = "1.4.2"
error-chain = "0.12.4"
glob = "0.3.1"
byteorder = "1.5.0"

[dependencies.uuid]
version = "1.10.0"
features = [
    "v4",                # Lets you generate random UUIDs
    "fast-rng",          # Use a faster (but still sufficiently random) RNG
    "macro-diagnostics", # Enable better diagnostics for compile-time UUIDs
    "serde",             # adds the ability to serialize and deserialize a UUID using `serde`
    "bytemuck",          # adds a `Pod` trait implementation to `Uuid` for byte manipulation
    "borsh",
]
2

what's the output of the second run? to help diagnostic, you can unwrap() every result so you get a panic with a stack trace and an error message when any system call fails.

3

Interesting.

I get ${PATH_NAME} is not a directory, which is quite correct.

The path gets stacked in my code so, the first time I get ${PATH]} and the second time I get ${PATH}${PATH}. Completely failed to see that in the debug prints :frowning:

Found the bug.

Before I read the directory, I have a prepare function that extracts a tar.gz into the directory and in that function I do: PATH.lock().unwrap().push_str(&directory.as_str());

I have this definition for PATH: static PATH: Mutex<String> = Mutex::new(String::new()); so every time I call the extract function, I push the original ${PATH} into that string.

Guess that I need to figure out how to just set the string and not append to it. :face_with_diagonal_mouth:

It seems that PATH.lock().unwrap().clear(); solves that problem, and yes, I should check it the files are already extracted before I try to extract them :wink:

Thanks for asking me basic questions and breaking my current loop of thought. :pray:

4

is there a reason you use String instead of PathBuf? paths are tricky to handle correctly, that's why there's a path module in the standard library. PathBuf is esentially OsString under the hood, but it understands path related concepts like "components", parents, root, absolute vs relative, etc.

2 Likes
5

That's gross. Why are using global state for something that's clearly local?

But until you fix that, …

There's nothing easier than a plain old assignment:

*PATH.lock().unwrap() = the_new_path;
2 Likes
6

Yes, I build the final path up in stages based on additional data from different functions, so a String is easy for me to handle (and what I'm used to from C :smile:)

Also the function where the path is used later on (external C library) requires a const char * as input so I find it easier to utilize a String in this case.

/Jocke!

7

Thank's for recommending the assignment.

BRs,
/Jocke!

8

if you mean the final path is built incrementally in pieces, the PathBuf is even more suitable. example:

// absolute path
let root_dir = Path::new("/srv/www");
// relative path
let main_js = Path::new("js/main.min.js");
// `join()` create a new `PathBuf`
let static_dir = root_dir.join("static");
// `push()` modifies existing variable
let mut target = root_dir.clone();
target.push("static");
target.push(main_js);

strings may seem easy (presumably due to familiarity), but it's also easy to make mistakes. that's the why we need type safety. for example, File, TcpListener, TcpStream, are different types, but in C, they are just int.

you should really learn the rust standard library, which helps to write idiomatic rust code.

on the one hand, rust strings are not NUL-terminated, you need an extra conversion (e.g. CString::new()) at the FFI boundary nevertheless, even if you use String.

on the other hand, PathBuf is just wrapper of OsString, which is the same as String if your path is valid utf8; and even if it's not utf8, you can get the underlying raw bytes just as easy. see OsStr::as_encoded_bytes().

so, using PathBuf isn't more work than using String, but it's less error-prone and more convenient to use.

2 Likes
9

Thank you very much for your comments.

I'm trying my best to learn, but I'll make a sh*tload of mistakes along the way. :sob:

Getting these tips and suggestions in such a positive manner is highly appreciated and I'll change the String that I'm using according to your suggestion.

Very best regards,
/Jocke!

PS. Not used to this forum, but how do you manage to quote from my comments? DS.

1 Like
10

select the text you wish to quote with mouse cursor, you'll see buttons popping up, press the "Quote" button and a [quote] tag will be automatically inserted into your reply text box.

the following animation is from the official discourse documentation site:

2 Likes
11

Thanks. :slight_smile:

BRs,
/Jocke!