Polling a remote endpoint (from a CLI)

I’m writing a CLI. It posts a file to a remote server (working fine) and then polls the server until a particular resource is available. I’m not sure what the best approach is. It looks like I might want to use tokio’s poll_fn, but I’m not sure if it’s a good fit and work bringing the entire tokio runtime into my app.

Please recommend approaches. Code examples or links to tutorials are definitely welcome.

I don’t know if you can poll a HTTP server until somethings is ready, in the traditional sense, that would require the server waking up the client, but you could just GET the resource every five seconds until it returns a 2XX code.

I guess you could also do a backoff, but I don’t know how well that would work.

I do remember something about web notifications being a thing, but I think that requires some sort of external service, and the protocol probably does not have an implementation in Rust.

Of course I didn’t mean polling in the sense of epoll or something like that. I meant, I’m expecting a resource to eventually exist. The endpoint will 404 if it isn’t ready. In that case, I want to sleep for a bit and try again. I may also want to update a console spinner or something using indicatif at some point in the future, but that’s not essential right this minute.

IMO this is a algorithmic question.
I think you should use the simplest approach in the first instance.

Can you ask the server (poll) once? Do that. Use a http request or whatever. If it 404s then sleep for a period and try again.

Polling once a second or so from a single client is a very light weight for a server. Of course it does not scale well. But is that important?

If you do not need tokio run-time then do not get it just for this. Use the smallest hammer that will hammer in the nail, not the biggest. And if it is not a nail do not use a hammer.

HTH

I’m not worried about the approach. I’m fine with polling the server. I’ve solved this problem in other languages and it works fine. As an exercise for my own benefit, I’m porting the solution to Rust. So, for example, what would the Rust solution look like that just loops and sleeps? I’ve tried that before and struggled with all the moving and borrowing, which made me think that I might be taking the wrong approach…

If you have any current code you probably should post it.

If you don’t, what HTTP library are you using?

Here’s what I just did, which works fine. Not sure why I was getting all the move and borrow issues last time I tried this…

let result = loop {
  let result = fetch_conversion(host, key)?;
  if result.status().is_success() {
    break result;
  } else {
    thread::sleep(Duration::from_secs(15));
  }
};
1 Like

Looks good, though fifteen seconds seems a bit much.

I know what the server is doing on the other end. It really isn’t :laughing:

3 Likes