How to add a threadpool to an API Client?


#1

I’m trying to write an API client for the Reddit HTTP API. The client uses hyper and I am using the evental crate to allow the client to be used in an async manner. Here is a sketch of the client:

use std::sync::Arc;
use hyper::Client;
use eventual::Future;

pub struct RedditAPIClient {
    client: Arc<Client>
}

impl RedditAPIClient {
    pub fn new() -> RedditAPIClient {
        RedditAPIClient {client: Arc::new(Client::new())}
    }

    pub fn fetch_homepage(&self) -> Future<RedditListing, ()> {
        let client = self.client.clone();
        return Future::spawn(move || {
           // Using the hyper client, fetch json and return a rust struct.
        });
    }
}

My problem now is that the Future::spawn method creates a new thread for executing the code and completing the future. I would instead like to add a threadpool to the RedditAPIClient struct that API requests can be dispatched to.

I tried to use the threadpool struct from [syncbox] (https://carllerche.github.io/syncbox/syncbox/struct.ThreadPool.html) (dependency of eventual), but it seems it can only run tasks that have a 'static lifetime, which means it isn’t usable for my client because I don’t think an API client should have a 'static lifetime.

What other options are available to me?


#2

eventual is probably requiring the 'static lifetime because thread::spawn is… I tried to do something similar for sending an HTTP client request with asynchronous Hyper in my Web framework (yes I know somewhat unintuitive but sometimes it’s useful to be able to send a query to a third-party server), but I took the easy way and wrote in a blocking way, using park/unpark for the first time :slight_smile:

I think there’s a way if you have a boxed closure that you can call in thread 1, and execute the request in thread 2, and send updated results with a channel from thread 2 to thread 1, and have each time thread 1 call the closure with the data. This way, the closure can have a short, finite lifetime instead of 'static.