How to allow the user of my library to implement some part of the library?

Hello.

I am developing a library that currently uses the only one component from std library - Mutex.
I want to make std an optional dependency in my library and to allow the user to implement the code for Mutex themselves if they compile the library without std.

I don't want to use dynamic dispatch of the implementation, for example by having a global static &dyn Mutex (Mutex is a trait here defined in my library). It is technically unnecessary since the implementation will be determined at compile time and may not be changed. Also, it reduces compiler's potential for optimizations.

Also, I'd prefer not to use generics on my types. It will significantly increase complexity of the code, while not providing much benefit from generics. Allowing multiple implementations of Mutex is redundant.

It would be great to have something like associative types, but on the crate level. I couldn't find anything like that supported.

Are there any other options to allow the user to implement some part of the library's code?
Maybe there are some great examples of such polymorphism in popular rust libraries?

That doesn't exist as a language feature today. There's been discussion about how something like it would be nice -- inject the async executor, the allocator, etc -- but it hasn't crystallized to anything.

If you really aren't willing to use generics, maybe try the static, but with LTO. If it's a readonly static, LTO might well be able to notice that it's always calling the same thing and deal with that.

2 Likes

I don't quite understand your proposed approach with static.
Could you please explain it in more details?

I don't really get this. Allowing multiple implementations is the very thing you are trying to allow, simply by letting the user customize any part, since you can't restrict who and how many times should be allowed to write such an implementation. You are never, ever going to be able to reliably forbid 3rd-parties from providing two different implementations, and quite frankly, I wouldn't ever want to use a library that makes such an attempt.

Just write your code generically against traits, and let your users implement that trait for their own types. That's the most sensible approach.

5 Likes

I agree with you. It would indeed cause problems if two crates in the same application depend on the library, while each trying to provide their own implementation.

I think I will stick to generics unless I find a better approach.

Thank you for your notice.

I've run into a similar problem before where downstream crates need to provide an implementation of something to an upstream crate, where it's not feasible to pass the dependency directly (i.e. by creating a generic function that accepts some object implementing the Mutex trait).

This was my answer on Extern "C" and Wasm with C dependency:

And the actual solution:

You might find our story interesting.

For some background, we've currently got 20+ implementations for a particular set of WIT files, and not being able to put wit_bindgen_export!() in a shared crate was becoming a real pain (can't share code for trait implementations, can't use ?, no convenience methods, code is super verbose, etc.). Wasmer also haven't updated their fork of wit-bindgen in quite a while so we couldn't use #193 without switching our WebAssembly VM from Wasmer to Wasmtime.

To deal with this, we took inspiration from how Rust's #[global_allocator] works...

First, we call wit_bindgen_rust::export!() in some shared crate.

// support/src/guest/bindings.rs

wit_bindgen_rust::import!("../wit-files/rune/runtime-v2.wit");
wit_bindgen_rust::export!("../wit-files/rune/proc-block-v2.wit");

Then we created a trait which represents our resource and would be implemented downstream.

// support/src/guest/proc_block.rs

pub trait ProcBlock {
    fn tensor_constraints(&self) -> TensorConstraints;
    fn run(&self, inputs: Vec<Tensor>) -> Result<Vec<Tensor>, RunError>;
}

Next comes extern "Rust" definitions for our ProcBlock's constructor and a free function the host uses to find out more about its functionality (name, description, input tensors, etc.).

// support/src/guest/proc_block.rs

extern "Rust" {
    fn __proc_block_metadata() -> Metadata;
    fn __proc_block_new(
        args: Vec<Argument>,
    ) -> Result<Box<dyn ProcBlock>, CreateError>;
}

From there, we can write a shim implementation in our support crate for the ProcBlockV2 trait that wit-bindgen generates.

// support/src/guest/proc_block.rs

struct ProcBlockV2;

impl proc_block_v2::ProcBlockV2 for ProcBlockV2 {
    fn metadata() -> Metadata {
        logging::initialize_logger();
        unsafe { __proc_block_metadata() }
    }

    fn create_node(
        args: Vec<Argument>,
    ) -> Result<wit_bindgen_rust::Handle<self::Node>, CreateError> {
        logging::initialize_logger();
        let proc_block = unsafe { __proc_block_new(args)? };
        Ok(Handle::new(Node(Box::new(proc_block))))
    }
}

pub struct Node(Box<dyn ProcBlock>);

impl proc_block_v2::Node for Node {
    fn tensor_constraints(&self) -> TensorConstraints {
        self.0.tensor_constraints()
    }

    fn run(&self, inputs: Vec<Tensor>) -> Result<Vec<Tensor>, RunError> {
        self.0.run(inputs)
    }
}

And to wrap it all up, we have a macro which end users can use to generate __proc_block_metadata() and friends.

// support/src/guest/proc_block.rs

/// Tell the runtime that a WebAssembly module contains a proc-block.
#[macro_export]
macro_rules! export_proc_block {
    (metadata: $metadata_func:expr, proc_block: $proc_block:ty $(,)?) => {
        #[doc(hidden)]
        #[no_mangle]
        pub fn __proc_block_metadata() -> $crate::guest::Metadata { $metadata_func() }

        #[doc(hidden)]
        #[no_mangle]
        pub fn __proc_block_new(
            args: Vec<$crate::guest::Argument>,
        ) -> Result<Box<dyn $crate::guest::ProcBlock>, $crate::guest::CreateError> {
            fn assert_impl_proc_block(_: &impl $crate::guest::ProcBlock) {}

            let proc_block = <$proc_block>::try_from(args)?;
            assert_impl_proc_block(&proc_block);

            Ok(Box::new(proc_block) as Box<dyn $crate::guest::ProcBlock>)
        }
    };
}

I really like this approach because, besides the export_proc_block!() macro, it all looks like normal Rust code and you don't need to know about wit-bindgen types (see here for an example). It also solves the awkward super issue where downstream crates need to somehow inject their types and free function implementations into a crate higher up the dependency tree.

2 Likes

That's an interesting approach, make the linker resolve the dependency injection.
Though I don't think it's suitable for my current development, partly because of H2CO3's notice, maybe I will use this approach in my future projects.
Thank you for sharing!