How to make a non-clonable field clone?

help
1

Hi,

I've an interesting situation in which I have a struct with a fields that is neither copy nor clone (in fact is has zero derived traits) but the holding struct gets moved into an async context that means when I want to access said field I need a value, which isn't working because the field is not clone...

Specifically, the struct looks like this:

pub struct IggyStream {
    iggy_producer: IggyProducer, // IggyProducer does not derive any traits. 
}

And is used like that:

 let res = IggyStream::new(&iggy_client, &stream_config).await;
    assert!(res.is_ok());
    let iggy_stream = res.unwrap();
   
   // This does not work 
   let message_producer = iggy_stream.producer().to_owned();

    let token = CancellationToken::new();
    let token_consumer = token.clone();
    tokio::spawn(async move {
        match iggy_stream // iggy_stream moves here
            .consume_messages(&PrintEventConsumer {}, token)
            .await
        {
            Ok(_) => {}
            Err(err) => {
                eprintln!("Failed to consume messages: {err}");
            }
        }
    });

    // Can't do this b/c of reference to moved value :-( 
    let res = message_producer.send_one(message).await;
    assert!(res.is_ok());

I've tried putting the iggy_producer field into an arc, but somehow this does not work presumably b/c Arc only counts references.

The only solution I found so far is to create a wrapper struct for the iggy_producer, put the IggyProducer into an Arc, and derice clone for the wrapper struct. I find this quite cumbersome relative to the actual goal of only getting the field by value.

Is there any other way solve this problem of getting a non-clone field by value in this specific instance?

2

It's not possible to answer this without seeing definitions of the functions and types.

If the producer is borrowing from iggy_stream (it's a reference or the Producer type has a lifetime) then you won't be able to keep that object and move the stream at the same time.

If consume_messages() takes self, then the implementation requires the call to be the last use of this object, and reserves the right to destroy it immediately, so there's nothing you can do to keep it.

It's uncertain whether this library supports your use-case at all.

Note that generic .to_owned() is implemented for shared references, and doesn't do anything for them! You get the same reference back (that's because & is Copy, and Copy implies Clone, and ToOwned is implemented on everything with Clone).

1 Like
3

Thank you,

the crux of the matter really is/was that the underlying type IggyProducer cannot be clone due to a tricky mutex in it. I had a conversation with the maintainers and there is no way that is going to change anytime soon.

it does exactly that. So, yes, last use.

In the end I solved this with stuffing the IggyProducer into an Arc, create a getter that returns theArc, called to_owned on it, and then used that one and that did the trick.

I don't know if that is the best way, but at least it does not need any wrapper structs.

Full code and a usage example below; and sorry for not linking it right away:

4

Bidirectional channels are usually split into sender and receiver halves to avoid the need for this kind of workaround. IggyStream::consume_messages() takes ownership, but it only uses the iggy_consumer field; the iggy_producer field that you are cloning via Arc is ignored.

Bundling IggyConsumer and IggyProducer into a single struct is counterproductive when you want the caller to be able to bifurcate.

You can potentially make the fields public so that IggyStream can be destructured with a pattern (requires either an extension trait to provide the consume_message() method etc. or a new-type with inherent method). The other idea is return a tuple instead of struct with public fields, and that can be destructured the same way that other APIs are used.

There are lots of examples of sender/receiver pairs in the wild:

1 Like
5

Thank you @parasyte

Based on your suggestions, I refactored the code to return a pair of (producer, consumer) basically making the builder stateless and, indeed, it makes things so much more coherent and in line with established best practices.

Also, I moved the consume_messages into a dedicated extension trait so that it is available to any IggyConsumer regardless of how it is build.

As background, I started out using Iggy for an internal project, but soon ran into some, say, verbosity, although the existing dev tooling is quite good for what it is, but I wanted something a lot more concise and expressive so I started experimenting around although I haven't written this kind of stuff before.

While talking to the iggy core team on the project discord, they expressed the option to contribute this to their SDK so if things go well, this exploration might move into a PR when the code is a bit more polished and tested.

Again, thank you for your invaluable insight to make this stream builder better and more in line with established best practices.

And here is the link to the updated repo:

1 Like