Function which takes an Iterator<Item = Box<dyn Any>>

I would like to write a function that takes an Iterator<Item = Box<dyn Any>> and returns a Vec<Box<dyn Any>> as result, doing something with each of the items the iterator yields.

Consider the following code:

#![allow(dead_code, unused_variables)]

use std::any::Any;

#[derive(Debug)]
struct S<T> {
    field: T,
}

fn wrap<T>(
    elements: impl IntoIterator<IntoIter = impl Iterator<Item = T>>,
) -> Vec<S<T>> {
    let mut vec = Vec::new();
    for element in elements {
        vec.push(S { field: element });
    }
    vec
}

fn dyn_wrap(
    elements: impl IntoIterator<
        IntoIter = impl Iterator<Item = Box<dyn Any>>,
    >,
) -> Vec<Box<dyn Any>> {
    todo!("How to implement this?")
}

fn main() {
    let a = wrap([1, 2]);
    println!("{a:?}");
    let b = wrap(["Hello", "World"]);
    println!("{b:?}");
    let c = dyn_wrap(vec![
        Box::new(7) as Box<dyn Any>,
        Box::new("seven") as Box<dyn Any>,
    ]);
    println!("{c:?}");
}

(Playground)

Output:

[S { field: 1 }, S { field: 2 }]
[S { field: "Hello" }, S { field: "World" }]

Errors:

   Compiling playground v0.0.1 (/playground)
    Finished dev [unoptimized + debuginfo] target(s) in 1.24s
     Running `target/debug/playground`
thread 'main' panicked at 'not yet implemented: How to implement this?', src/main.rs:25:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Is it possible to implement dyn_wrap akin to wrap?

Either I am tripping or aren't you simply trying to call elements.into_iter().collect() with extra steps?

I'm mapping each element too (wrapping it in this example). But in my later code, the wrap function will perform other tasks too. This is my real-life code, but I wanted to keep the example simple:

unsafe fn open_or_opt_create_dbs<'a, K, V, C>(
    self: &Arc<Self>,
    options: impl IntoIterator<
        IntoIter = impl Iterator<Item = &'a DbOptions<K, V, C, Option<CString>>> + ExactSizeIterator,
    >,
    create: bool,
) -> Result<Vec<Db<K, V, C>>, io::Error>
where
    K: ?Sized + Storable,
    V: ?Sized + Storable,
    C: Constraint,
{
    let options = options.into_iter();
    let mut dbs = Vec::with_capacity(options.len());
    let db_guard = self.db_mutex.lock().unwrap();
    let mut txn_inner = MaybeUninit::<*mut lmdb::MDB_txn>::uninit();
    check_err_code(lmdb::mdb_txn_begin(
        self.inner,
        null_mut(),
        if create {
            0 as LmdbFlags
        } else {
            lmdb::MDB_RDONLY as LmdbFlags
        },
        txn_inner.as_mut_ptr(),
    ))?;
    let txn_inner = txn_inner.assume_init();
    let txn = TxnBackend {
        env_backend: self,
        inner: txn_inner,
        cursors: Default::default(),
    };
    for option in options {
        let mut db_inner = MaybeUninit::<lmdb::MDB_dbi>::uninit();
        let mut flags = option.cleansed_lmdb_flags();
        if create {
            flags |= lmdb::MDB_CREATE as LmdbFlags;
        }
        check_err_code(lmdb::mdb_dbi_open(
            txn.inner,
            option.name.as_ref().map_or(null(), |x| x.as_ptr()),
            flags,
            db_inner.as_mut_ptr(),
        ))?;
        let db_inner = db_inner.assume_init();
        let db_backend = DbBackend {
            env_backend: Arc::downgrade(self),
            inner: db_inner,
        };
        if !K::TRIVIAL_CMP && !K::OPTIMIZE_INT {
            check_err_code(lmdb::mdb_set_compare(
                txn.inner,
                db_backend.inner,
                Some(compare_function::<K>),
            ))?;
        }
        if !V::TRIVIAL_CMP && !V::OPTIMIZE_INT && C::DUPLICATE_KEYS {
            check_err_code(lmdb::mdb_set_dupsort(
                txn.inner,
                db_backend.inner,
                Some(compare_function::<V>),
            ))?;
        }
        dbs.push(Db {
            key: PhantomData,
            value: PhantomData,
            constraint: PhantomData,
            backend: ArcByAddr::new(db_backend),
        });
    }
    txn.commit()?;
    drop(db_guard);
    Ok(dbs)
}

P.S.: I wanted to design the interface such that the caller can't hold the internal db_mutex lock for a long time.

What is the signature of your actual dyn_wrap function?

It would be something like:

unsafe fn open_or_opt_create_dbs_dyn<'a>(
    self: &Arc<Self>,
    options: impl IntoIterator<
        IntoIter = impl Iterator<Item = &'a Box<dyn Any>> + ExactSizeIterator,
    >,
    create: bool,
) -> Result<Vec<Box<dyn Any>>, io::Error>
where
    K: ?Sized + Storable,
    V: ?Sized + Storable,
    C: Constraint;

I think you can do it with downcast_ref. The module docs for Any has some examples.

But which type do I downcast to? I won't know (I think). Hence my question if anyone knows a way how to fix the todo!("How to implement this?") in my OP.

If you don't know the type at all, you won't be able to call things on the iterator items (like you have called let mut flags = option.cleansed_lmdb_flags();). But, you can still iterate all right and simply accumulate.

The caller of dyn_wrap would know the type, but the dyn_wrap function itself will only be able to use trait methods, of course.

But you have "objects" of trait Any - which means you cannot access any methods without a downcast.

Yes, I'll later have Any + SomeTrait.

I tried to modify my original example to expect an additional trait (which has an associated type that reflects the wrapped type), but couldn't get the original example working.

You should probably use just dyn SomeTrait then, without Any?

Maybe. But… to even understand if it's possible what I try to achieve, I would like to get the above example working (or understand why it's not possible to get that example working).

I don't need particular methods in that example case. I need to be able to wrap the value in S. But not sure how to express that with a trait (whether Any or SomeOtherTrait).

Maybe you can describe your situation agin, please?

Here is my understanding
1.You have a collection of trait objects
2.You have a transformation, which acts element-wise
3.You want to call: IntoIter,Map with transformer, collect

So, as far as I can tell, the only problem is step 2.
Is this close to your problem? (Maybe not)
If it is close, can you describe your transformation?

Regarding your wrap vs dyn_wrap: It is often much easier to write a transformer forba specific type thrn for a trait object, so I can "feel" your struggles.

For me, it is often helpful to start with an enum, with a handful of real-world variants. Then, once I manage write the transformer for this one, I tackle the trait object case (which is often much harder)

Note also that it's impossible to consume dyn Trait via the method on Trait. You can either consume the Box<dyn Trait>, treating it as a "black box" (pun unintended), i.e. in your initial example S would be not generic, but just a wrapper around Box<dyn Trait>; or you can treat it as a reference and build the output from scratch (with possibly partially cloned data, if the trait provides such capabilities).

What's missing here?

fn dyn_wrap(
    elements: impl IntoIterator<
        IntoIter = impl Iterator<Item = Box<dyn Any>>,
    >,
) -> Vec<Box<dyn Any>> {
    elements
        .into_iter()
        .map(|field| -> Box<dyn Any> { Box::new(S { field }) })
        .collect()
}

The problem is that the contained values are of type S::<dyn Any> instead of S<i32> and S<&'static str>. Thus:

/* … */
fn main() {
    /* … */
    let c = dyn_wrap(vec![
        Box::new(7) as Box<dyn Any>,
        Box::new("seven") as Box<dyn Any>,
    ]);
    let c1: Option<&S<i32>> = c[0].downcast_ref();
    let c2: Option<&S<&'static str>> = c[1].downcast_ref();
    println!("{c1:?}");
    println!("{c2:?}");
}

(Playground)

Output:

…
None
None

Instead, I would like c to be something like this:

#![allow(dead_code, unused_variables)]

use std::any::Any;

#[derive(Debug)]
struct S<T> {
    field: T,
}

fn main() {
    let c: Vec<Box<dyn Any>> = vec![
        Box::new(S { field: 7 }),
        Box::new(S { field: "seven" }),
    ];
    let c1: Option<&S<i32>> = c[0].downcast_ref();
    let c2: Option<&S<&'static str>> = c[1].downcast_ref();
    println!("{c1:?}");
    println!("{c2:?}");
}

(Playground)

Output:

Some(S { field: 7 })
Some(S { field: "seven" })

I hope that clarifies my problem.

P.S.: In my real-life code, I want to be able to open or create several databases at once using open_or_opt_create_dbs even if those databases have different type arguments K, V, and C. That's why I guess I need some dyn type. But I also need the result to be downcast-able.

I think my problem is (and I didn't clarify it yet, sorry about that):

  • I want to perform the transformation of a list of values of different type in a single method call.
  • I want to downcast the results such that I recover concrete types afterwards (because I know which types I gave as input into the function and which types are to be expected as output of the function).

The enum won't help me (I think), because I need to be generic over T (or K, V, C in my real-world example).

I hope to be able to publish my full source soon, so maybe I can also get back to this issue with the full real-life example.

One more question:
Can you solve your Problem for a single item?

I expect something like

  • apply transformer to trait object
  • receiving a new trait object
  • downcast it to your given type
    This should work also for collections

I'm interested to learn more about your case

Ah, I see. So you're going to need to move the boxed object, even though you don't know what it is [1]. To me that sounds like you need a Box<Self>-taking method on a trait; the trait implementation will know the underlying type and can move it into an S for you.

trait Burrito: Debug {
    fn wrap_in_s<'s>(self: Box<Self>) -> Box<dyn Any + 's> where Self: 's;
}

impl<T: Debug> Burrito for T {
    fn wrap_in_s<'s>(self: Box<Self>) -> Box<dyn Any + 's> where Self: 's {
        let field = *self;
        Box::new(S { field })
    }
}

fn dyn_wrap(
    elements: impl IntoIterator<
        IntoIter = impl Iterator<Item = Box<dyn Burrito>>,
    >,
) -> Vec<Box<dyn Any>> {
    elements
        .into_iter()
        .map(<dyn Burrito as Burrito>::wrap_in_s)
        .collect()
}

Playground.


  1. unless S is transparent and can be transmuted perhaps ↩︎

2 Likes