Function which takes an Iterator<Item = Box<dyn Any>>

help
#1

I would like to write a function that takes an Iterator<Item = Box<dyn Any>> and returns a Vec<Box<dyn Any>> as result, doing something with each of the items the iterator yields.

Consider the following code:

#![allow(dead_code, unused_variables)]

use std::any::Any;

#[derive(Debug)]
struct S<T> {
    field: T,
}

fn wrap<T>(
    elements: impl IntoIterator<IntoIter = impl Iterator<Item = T>>,
) -> Vec<S<T>> {
    let mut vec = Vec::new();
    for element in elements {
        vec.push(S { field: element });
    }
    vec
}

fn dyn_wrap(
    elements: impl IntoIterator<
        IntoIter = impl Iterator<Item = Box<dyn Any>>,
    >,
) -> Vec<Box<dyn Any>> {
    todo!("How to implement this?")
}

fn main() {
    let a = wrap([1, 2]);
    println!("{a:?}");
    let b = wrap(["Hello", "World"]);
    println!("{b:?}");
    let c = dyn_wrap(vec![
        Box::new(7) as Box<dyn Any>,
        Box::new("seven") as Box<dyn Any>,
    ]);
    println!("{c:?}");
}

(Playground)

Output:

[S { field: 1 }, S { field: 2 }]
[S { field: "Hello" }, S { field: "World" }]

Errors:

   Compiling playground v0.0.1 (/playground)
    Finished dev [unoptimized + debuginfo] target(s) in 1.24s
     Running `target/debug/playground`
thread 'main' panicked at 'not yet implemented: How to implement this?', src/main.rs:25:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Is it possible to implement dyn_wrap akin to wrap?

#2

Either I am tripping or aren't you simply trying to call elements.into_iter().collect() with extra steps?

#3

I'm mapping each element too (wrapping it in this example). But in my later code, the wrap function will perform other tasks too. This is my real-life code, but I wanted to keep the example simple:

unsafe fn open_or_opt_create_dbs<'a, K, V, C>(
    self: &Arc<Self>,
    options: impl IntoIterator<
        IntoIter = impl Iterator<Item = &'a DbOptions<K, V, C, Option<CString>>> + ExactSizeIterator,
    >,
    create: bool,
) -> Result<Vec<Db<K, V, C>>, io::Error>
where
    K: ?Sized + Storable,
    V: ?Sized + Storable,
    C: Constraint,
{
    let options = options.into_iter();
    let mut dbs = Vec::with_capacity(options.len());
    let db_guard = self.db_mutex.lock().unwrap();
    let mut txn_inner = MaybeUninit::<*mut lmdb::MDB_txn>::uninit();
    check_err_code(lmdb::mdb_txn_begin(
        self.inner,
        null_mut(),
        if create {
            0 as LmdbFlags
        } else {
            lmdb::MDB_RDONLY as LmdbFlags
        },
        txn_inner.as_mut_ptr(),
    ))?;
    let txn_inner = txn_inner.assume_init();
    let txn = TxnBackend {
        env_backend: self,
        inner: txn_inner,
        cursors: Default::default(),
    };
    for option in options {
        let mut db_inner = MaybeUninit::<lmdb::MDB_dbi>::uninit();
        let mut flags = option.cleansed_lmdb_flags();
        if create {
            flags |= lmdb::MDB_CREATE as LmdbFlags;
        }
        check_err_code(lmdb::mdb_dbi_open(
            txn.inner,
            option.name.as_ref().map_or(null(), |x| x.as_ptr()),
            flags,
            db_inner.as_mut_ptr(),
        ))?;
        let db_inner = db_inner.assume_init();
        let db_backend = DbBackend {
            env_backend: Arc::downgrade(self),
            inner: db_inner,
        };
        if !K::TRIVIAL_CMP && !K::OPTIMIZE_INT {
            check_err_code(lmdb::mdb_set_compare(
                txn.inner,
                db_backend.inner,
                Some(compare_function::<K>),
            ))?;
        }
        if !V::TRIVIAL_CMP && !V::OPTIMIZE_INT && C::DUPLICATE_KEYS {
            check_err_code(lmdb::mdb_set_dupsort(
                txn.inner,
                db_backend.inner,
                Some(compare_function::<V>),
            ))?;
        }
        dbs.push(Db {
            key: PhantomData,
            value: PhantomData,
            constraint: PhantomData,
            backend: ArcByAddr::new(db_backend),
        });
    }
    txn.commit()?;
    drop(db_guard);
    Ok(dbs)
}

P.S.: I wanted to design the interface such that the caller can't hold the internal db_mutex lock for a long time.

#4

What is the signature of your actual dyn_wrap function?

#5

It would be something like:

unsafe fn open_or_opt_create_dbs_dyn<'a>(
    self: &Arc<Self>,
    options: impl IntoIterator<
        IntoIter = impl Iterator<Item = &'a Box<dyn Any>> + ExactSizeIterator,
    >,
    create: bool,
) -> Result<Vec<Box<dyn Any>>, io::Error>
where
    K: ?Sized + Storable,
    V: ?Sized + Storable,
    C: Constraint;
#6

I think you can do it with downcast_ref. The module docs for Any has some examples.

#7

But which type do I downcast to? I won't know (I think). Hence my question if anyone knows a way how to fix the todo!("How to implement this?") in my OP.

#8

If you don't know the type at all, you won't be able to call things on the iterator items (like you have called let mut flags = option.cleansed_lmdb_flags();). But, you can still iterate all right and simply accumulate.

#9

The caller of dyn_wrap would know the type, but the dyn_wrap function itself will only be able to use trait methods, of course.

#10

But you have "objects" of trait Any - which means you cannot access any methods without a downcast.

#11

Yes, I'll later have Any + SomeTrait.

I tried to modify my original example to expect an additional trait (which has an associated type that reflects the wrapped type), but couldn't get the original example working.

#12

You should probably use just dyn SomeTrait then, without Any?

#13

Maybe. But… to even understand if it's possible what I try to achieve, I would like to get the above example working (or understand why it's not possible to get that example working).

I don't need particular methods in that example case. I need to be able to wrap the value in S. But not sure how to express that with a trait (whether Any or SomeOtherTrait).

#14

Maybe you can describe your situation agin, please?

Here is my understanding
1.You have a collection of trait objects
2.You have a transformation, which acts element-wise
3.You want to call: IntoIter,Map with transformer, collect

So, as far as I can tell, the only problem is step 2.
Is this close to your problem? (Maybe not)
If it is close, can you describe your transformation?

Regarding your wrap vs dyn_wrap: It is often much easier to write a transformer forba specific type thrn for a trait object, so I can "feel" your struggles.

For me, it is often helpful to start with an enum, with a handful of real-world variants. Then, once I manage write the transformer for this one, I tackle the trait object case (which is often much harder)

#15

Note also that it's impossible to consume dyn Trait via the method on Trait. You can either consume the Box<dyn Trait>, treating it as a "black box" (pun unintended), i.e. in your initial example S would be not generic, but just a wrapper around Box<dyn Trait>; or you can treat it as a reference and build the output from scratch (with possibly partially cloned data, if the trait provides such capabilities).

#16

What's missing here?

fn dyn_wrap(
    elements: impl IntoIterator<
        IntoIter = impl Iterator<Item = Box<dyn Any>>,
    >,
) -> Vec<Box<dyn Any>> {
    elements
        .into_iter()
        .map(|field| -> Box<dyn Any> { Box::new(S { field }) })
        .collect()
}
#17

The problem is that the contained values are of type S::<dyn Any> instead of S<i32> and S<&'static str>. Thus:

/* … */
fn main() {
    /* … */
    let c = dyn_wrap(vec![
        Box::new(7) as Box<dyn Any>,
        Box::new("seven") as Box<dyn Any>,
    ]);
    let c1: Option<&S<i32>> = c[0].downcast_ref();
    let c2: Option<&S<&'static str>> = c[1].downcast_ref();
    println!("{c1:?}");
    println!("{c2:?}");
}

(Playground)

Output:

…
None
None

Instead, I would like c to be something like this:

#![allow(dead_code, unused_variables)]

use std::any::Any;

#[derive(Debug)]
struct S<T> {
    field: T,
}

fn main() {
    let c: Vec<Box<dyn Any>> = vec![
        Box::new(S { field: 7 }),
        Box::new(S { field: "seven" }),
    ];
    let c1: Option<&S<i32>> = c[0].downcast_ref();
    let c2: Option<&S<&'static str>> = c[1].downcast_ref();
    println!("{c1:?}");
    println!("{c2:?}");
}

(Playground)

Output:

Some(S { field: 7 })
Some(S { field: "seven" })

I hope that clarifies my problem.

P.S.: In my real-life code, I want to be able to open or create several databases at once using open_or_opt_create_dbs even if those databases have different type arguments K, V, and C. That's why I guess I need some dyn type. But I also need the result to be downcast-able.

#18

I think my problem is (and I didn't clarify it yet, sorry about that):

  • I want to perform the transformation of a list of values of different type in a single method call.
  • I want to downcast the results such that I recover concrete types afterwards (because I know which types I gave as input into the function and which types are to be expected as output of the function).

The enum won't help me (I think), because I need to be generic over T (or K, V, C in my real-world example).

I hope to be able to publish my full source soon, so maybe I can also get back to this issue with the full real-life example.

#19

One more question:
Can you solve your Problem for a single item?

I expect something like

  • apply transformer to trait object
  • receiving a new trait object
  • downcast it to your given type
    This should work also for collections

I'm interested to learn more about your case

#20

Ah, I see. So you're going to need to move the boxed object, even though you don't know what it is [1]. To me that sounds like you need a Box<Self>-taking method on a trait; the trait implementation will know the underlying type and can move it into an S for you.

trait Burrito: Debug {
    fn wrap_in_s<'s>(self: Box<Self>) -> Box<dyn Any + 's> where Self: 's;
}

impl<T: Debug> Burrito for T {
    fn wrap_in_s<'s>(self: Box<Self>) -> Box<dyn Any + 's> where Self: 's {
        let field = *self;
        Box::new(S { field })
    }
}

fn dyn_wrap(
    elements: impl IntoIterator<
        IntoIter = impl Iterator<Item = Box<dyn Burrito>>,
    >,
) -> Vec<Box<dyn Any>> {
    elements
        .into_iter()
        .map(<dyn Burrito as Burrito>::wrap_in_s)
        .collect()
}

Playground.

  1. unless S is transparent and can be transmuted perhaps ↩︎

2 Likes