Storing a Struct with an Injected Async Callback in another Struct

code review
#1

Hey Guys,

Rust noob trying to learn by building my first project: Tor on mobile.
I'm trying to create a mobile library for developers to easily integrate Tor into their projects.

I'm standing on the shoulder of giants by combining to great crates:
Libtor: Tor Daemon
https://crates.io/crates/libtor
Torut: Control the Tor Daemon
https://crates.io/crates/torut

What i'm trying to do is use libtor to spawn the Tor Daemon and then use Torut to control and shut it down when an app goes in the background and then spin it back up when foregrounded via a C and JNI FFI.

I've manged to get my POC to compile on iOS/Android etc, but now I'm trying to make it into a Pacakge, I'm trying to do the following:

pub struct TorService {
    socks_port: u16,
    control_port: String,
    _handle: Option<JoinHandle<Result<u8, libtor::Error>>>, // Stores the thread handle returned by libtor for the daemon thread
    _ctl: RefCell<Option<G>>, // Stores the authenticated  daemon control connection returned by Torut
}

I setup G to have the following type:

type G = AuthenticatedConn<
    TcpStream,
    Fn(AsyncEvent<'static>)  -> Future<Output = Result<(), ConnError>>,
>;

based on Torut's authenticated connection's definition, which also requires an Async callback to be injected :
Github link:
teawithsand/torut/blob/master/src/control/conn/authenticated_conn.rs#L54-L61
(sorry for partial url, only allowed 2 links per post as a noob)

impl<S, H, F> AuthenticatedConn<S, H>
    where
        S: AsyncRead + Unpin,
    // there fns make use of event handler so it's needed
        H: Fn(AsyncEvent<'static>) -> F,
        F: Future<Output=Result<(), ConnError>>,
{

I define my Async handler as

fn handler(
    event: AsyncEvent<'static>,
) -> Pin<Box<dyn Future<Output = Result<(), ConnError>> + '_>> {
    Box::pin(async move { Ok(()) })
}

My issues all lie in trying to store the AuthenticatedConn in TorService._handle:

let mut ac = self.get_control_auth_conn(Some(Box::new(handler))).await;
 *self._ctl.borrow_mut() = Some(ac);

no matter what typing I try, the compiler just hates me, things i've tried so far:

type G = AuthenticatedConn<
    TcpStream,
    fn(AsyncEvent<'static>) -> Pin<Box<dyn Future<Output = Result<(), ConnError>> + '_>>>,
>;

i get

error[E0308]: mismatched types
   --> tor/src/lib.rs:123:44
    |
123 |             *self._ctl.borrow_mut() = Some(ac);
    |                                            ^^ expected fn pointer, found fn item
    |
    = note: expected struct `torut::control::AuthenticatedConn<_, fn(torut::control::AsyncEvent<'_>) -> std::pin::Pin<std::boxed::Box<dyn futures::Future<Output = std::result::Result<(), torut::control::ConnError>>>>>`
               found struct `torut::control::AuthenticatedConn<_, fn(torut::control::AsyncEvent<'static>) -> std::pin::Pin<std::boxed::Box<(dyn futures::Future<Output = std::result::Result<(), torut::control::ConnError>> + 'static)>> {handler}>`

If i try:

type G = AuthenticatedConn<
    TcpStream,
    Box<dyn Fn(AsyncEvent<'static>) -> Pin<Box<dyn Future<Output = Result<(), ConnError>> + '_>>>,
>;

i get

error[E0308]: mismatched types
   --> tor/src/lib.rs:124:44
    |
124 |             *self._ctl.borrow_mut() = Some(ac);
    |                                            ^^ expected trait object `dyn std::ops::Fn`, found fn item
    |
    = note: expected struct `torut::control::AuthenticatedConn<_, std::boxed::Box<dyn std::ops::Fn(torut::control::AsyncEvent<'static>) -> std::pin::Pin<std::boxed::Box<(dyn futures::Future<Output = std::result::Result<(), torut::control::ConnError>> + 'static)>>>>`
               found struct `torut::control::AuthenticatedConn<_, std::boxed::Box<fn(torut::control::AsyncEvent<'static>) -> std::pin::Pin<std::boxed::Box<(dyn futures::Future<Output = std::result::Result<(), torut::control::ConnError>> + 'static)>> {handler}>>`

finally if i try:

type G = AuthenticatedConn<
    TcpStream,
    Fn(AsyncEvent<'static>) -> Future<Output = Result<(), ConnError>>,
>;

i get

error[E0277]: the size for values of type `(dyn std::ops::Fn(torut::control::AsyncEvent<'static>) -> (dyn futures::Future<Output = std::result::Result<(), torut::control::ConnError>> + 'static) + 'static)` cannot be known at compilation time
  --> tor/src/lib.rs:41:11
   |
41 |     _ctl: RefCell<Option<G>>,
   |           ^^^^^^^^^^^^^^^^^^ doesn't have a size known at compile-time
   |
  ::: <snip>/.cargo/registry/src/github.com-1ecc6299db9ec823/torut-0.1.6/src/control/conn/authenticated_conn.rs:33:33
   |
33 | pub struct AuthenticatedConn<S, H> {
   |                                 - required by this bound in `torut::control::AuthenticatedConn`
   |
   = help: the trait `std::marker::Sized` is not implemented for `(dyn std::ops::Fn(torut::control::AsyncEvent<'static>) -> (dyn futures::Future<Output = std::result::Result<(), torut::control::ConnError>> + 'static) + 'static)`

I've been at this for 5 days now and i Just can't seem to get it working.

:sob:

The reason i'm trying to store the control connection in the TorService is to be able to stop/start the Daemon on App foregrounding/back-grounding while keeping the FFI very basic.

I would really appreciate any guidance on what i'm doing wrong, or what i'm doing right but Rust thinks is wrong or maybe i'm just doing it all wrong. At this point i'm just looking to get an understanding on how one would approach/fix this issue in rust!

Apologies for the long post and if i have missed any important information, kindly do let me know, thank you so much for reading so far!

#2

Can you please edit the post to include the full error messages instead of the IDE popups.

#3

Done. Thank you so much for reading the post!

#4

The first thing I see where some trouble can be had is here:

type G = AuthenticatedConn<
    TcpStream,
    Fn(AsyncEvent<'static>)  -> Future<Output = Result<(), ConnError>>,
>;

It seems like you are mixing up types and traits here, because Fn and Future are traits, not types. When you want to use a type as a trait, you must either use generics or trait objects.

Generics. A generic is when you say <T: SomeTrait>. This only applies for structs and functions. It's not the approach we want here.

Trait objects. Given a trait MyTrait, there exists a special type dyn MyTrait called a trait object. This is a special type that any type that implements the trait can be converted into, although its important to note that a conversion is necessary to obtain a value of type trait object because it is its own distinct type.

Additionally, a trait object cannot ever be "bare". I'll leave out the explanation for why for now, but the consequence is that trait objects must always be behind some kind of pointer. There are many types of pointers that can be used, e.g. &, Box or Arc, but the point is that some kind of pointer is necessary.

Note: If you try to use a trait where a type is expected, the compiler understands that as the trait's trait object type. However this is deprecated syntax — the new syntax requires that you add dyn in front of the trait object, and since you have two instances of this in your type alias, you likely have two compiler warnings about the lack of dyn being deprecated.

So in conclusion, you need to wrap both in some kind of pointer. When dealing with Fn trait objects, the pointer type you want is usually Box, although you sometimes see people using Arc to share it. As for Future, the trait objects from the async world usually want something special: Pin<Box<...>>. So the final type looks like this:

type G = AuthenticatedConn<
    TcpStream,
    Box<dyn Fn(AsyncEvent<'static>)  -> Pin<Box<dyn Future<Output = Result<(), ConnError>>>>>,
>;

Now, you might need some + Send and + Sync annotations somewhere in there too, but maybe not considering you're already using the non-thread safe type RefCell.

Recall also that a conversion is necessary. To turn a closure into a boxed closure, use Box::new(the_closure). As for the future, use Box::pin(the_future).

I haven't looked at the rest, but let me know how far this gets you.

#5

Thank you so much for the reply and taking the time to explain traits and types, it really does help.
In regards to using Boxs and Pins, I had already tried the example you had brought forward.
In my post:

and

If i change my handler to be a closure

error[E0308]: mismatched types
   --> tor/src/lib.rs:124:44
    |
119 |             let mut ac = self.get_control_auth_conn(Some(Box::new(|event:AsyncEvent<'static>| Box::pin(async move{ })))).await;
    |                                                                   ---------------------------------------------------
    |                                                                   |                                              |
    |                                                                   |                                              the found generator
    |                                                                   the found closure
...
124 |             *self._ctl.borrow_mut() = Some(ac);
    |                                            ^^ expected trait object `dyn std::ops::Fn`, found closure
    |
   ::: /<snip>/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:60:43
    |
60  | pub const fn from_generator<T>(gen: T) -> impl Future<Output = T::Return>
    |                                           ------------------------------- the found opaque type

Again thank you so much for taking the time to explain and teach me!

#6

What is the return type of get_control_auth_conn?

#7

Strangest thing !!

I just changed G's signature from:

to

type F = Box<dyn Fn(AsyncEvent<'static>)  -> Pin<Box<dyn Future<Output = Result<(), ConnError>>>>>;
type G = AuthenticatedConn< TcpStream, F >;

and manually coerced handle into F

let mut ac = self.get_control_auth_conn(Some(Box::new(handler) as F)).await;
ac.wait_bootstrap().await.unwrap();

and now it compiles!!!!!!! :partying_face:

I don't get why the compiler need to be told that, but it works now ??

#8

:tada:

Sometimes when there's a layer of indirection until the type hint, it can't figure it out.

#9

I'm so freakin happy !!! Thank you so much for taking the time to reply alice. You really helped me to think outside the Box (excuse the phun)!

1 Like
#10

You're welcome :slight_smile: