Emulating private function in traits

I would like to know if my code is idiomatic, and if it has pitfall that I wasn't expected. This is a re-wrote of the NVI (Non-virtual interface) from C++. It allows to explicitly specify the customization point of an algorithm.

First, the C++ implementation:

class Base {
public:
    // the entry point of the algorithm
    // this function isn't virtual, it's not a customization point
    void foo {
        common_part();
        custom_behavior();
    }
private:
    // this isn't a virtual function, so this isn't a customization point either
    void common_part() { /* ... */ }

    // pure virtual function -> it must be overrided in a child class
    // this is the customization point
    virtual void custom_behavior() = 0;
};

class Implementation: public Base {
private:
    // implements the custom behavior for this specific class
    void custom_behavior() { /* ... */ }
}

The function foo() is the only function in the public API of the class. common_part() is private, and should not be overridable by the child class Implementation. custom_behavior is also private, and must be overriden in the child class.


How do I do the same in Rust? Base describes an interface, so I think it should be the trait, while Implementation would be a struct implementing this trait.

  1. How do I prevent foo() and common_part() from being overwritten in Implementation?
  2. How do I prevent a user of Base to call common_part() or custom_behavior()?

I have no idea how to do (1).

For (2), we need to emulate the priv keyword since all functions in a trait (unlike everything else in Rust) are public by default, and not private. So far the best idea I had was to create a private module, and add the private function in it:

// public trait
pub trait Base: private::BaseDetails {
    fn foo(&self) {
        self.common_part();
        self.custom_behavior();
    }
}

// private module
mod private {
    // public trait in a private module
    pub trait BaseDetails {
        fn common_part(&self) { /* ... */ }
        fn custom_behavior(&self)
    }
}

struct Implementation;
impl Base for Implementation{} // nothing to do
impl private::BaseDetails for Implementation {
    fn custom_behavior(&self) { /* ... */ }
}

  • Is this idea valid?
  • Does it correctly prevent user of the trait Base to use common_part() and custom_behavior()?
  • Can all structs implements private::BaseDetails? Or do the module private must be a pub mod? If so, does it still prevent user of the trait Base to use common_part() and custom_behavior()?
  • How can we do (1) and have non-customization points in a Trait?

The things you want to do with traits are not available in Rust. Traits are not abstract base classes; they are interfaces. If you want private methods, they should be implemented on your types directly. Use a trait when you want code to be generic over types, not for code reuse. playground

Look at it this way: When a trait is implemented on a type, any code that has the trait in scope may call any and all trait methods on the type. What you probably want to do in this case is move the common and custom code to the individual types, and define an empty trait that they all share. This will allow code to be generic over these types, while keeping private methods private.

3 Likes

I think I should have added an example of use. The whole raison d'être of the NVI pattern is to have a function that manipulate the base class, and call the public function, without knowing what particular version of the algorithm is used. This is one of the many possible implementation of the strategy pattern.

In C++:

void usage(const& Base base) {
    base.foo();
}
int main(void) {
    Implementation im;
    usage(im);
}

In Rust:

pub fn usage<T: Base>(base: &T) {
    base.foo():
}
pub fn main() {
    let im = Implementation::new();
    usage(&im);
}

I think that the implementation you did on your playground link cannot easily be modified to be able to:

  • write a usage() function that can take any type that implement a trait with a foo() function
  • not having to duplicate the code of foo() for any implementation of that trait

I may (and I wish) that I'm wrong and that I'm missing something.

1 Like

Something like this? playground

I know, it's not exactly what you are asking. But that's because traits do not provide the capability you want. Traits provide interfaces for generic functions. That's all.

The only constraint missing from this example is that the trait methods are both unique to each type. If sharing some code between all implementors is necessary, it can be done with a default trait method (at the expense that this method is visible to callers) or perhaps with a secondary private trait (at the expense that callers may be able to bring that trait into scope, especially if they need to implement it on their own types).

Click here if you want to see other approaches and their pros and cons
mod private {
    pub
    trait BaseMethods {
        fn foo (self: &'_ mut Self)
        ;
        fn common_part (self: &'_ mut Self)
        ;
    }

    pub
    trait Base : BaseMethods {
        fn custom_behavior (self: &'_ mut Self)
        ;
    }

    impl<T : ?Sized + Base> BaseMethods for T {
        fn foo (self: &'_ mut Self)
        {
            self.common_part();
            self.custom_behavior();
        }

        fn common_part (self: &'_ mut Self)
        {
            // ...
        }
    }
}
pub
use private::Base;

struct Implementation { /* ... */ }
impl Base for Implementation {
    // One has to provide `custom_behavior`
    fn custom_behavior (self: &'_ mut Self)
    {
        // ...
    }
}

fn main ()
{
    let ref mut it = Implementation { /* ... */ };
    // Only someone with private access can do this:
    {
        use private::BaseMethods;
        it.foo();
    }
    // But anybody can do this:
    (it as &mut dyn Base).foo();
}

In the "normal" case, one needs to be able to refer to BaseMethods to call its methods (bringing the trait in scope, or using the fully qualified syntax), so by virtue of only reexporting Base and not BaseMethods out of a non-pub module, one cannot call common_part or foo. That being said, I think this "property" is a byproduct of how modules, traits and privacy work in Rust, and was not something truly intended; since BaseMethods is a pub trait, in theory it is usable.

For instance, when using dyn / trait objects, all the (now virtual) methods become accessible, even those of BaseMethods. I couldn't come up with another way to "cheat" this hacky privacy, though.

This means that, if in your case, you do not need to support trait objects / dynamic dispatch, you can forbid foo and common_part from becoming virtual by adding a where Self : Sized clause to them:

pub
trait BaseMethods {
    fn foo (self: &'_ mut Self)
    where
        Self : Sized,
    ;
    // ...
}

If you still want to support dynamic dispatch but want the methods to be "private", there is a(n even) hacky(-er) solution, which would be to have these methods require a dummy but private parameter:

mod private {
    pub
    struct FooPrivacyToken;
    pub
    trait BaseMethods {
        fn foo (self: &'_ mut Self, _: FooPrivacyToken)
        ;
        // ...
    }
    // ...
}

and this way, since FooPrivacyToken, much like BaseMethods, is hidden by the private module, it will be literally impossible for anybody to call these methods without access to private innards, i.e., somebody located within the module that contains the definition of private (which you can, if you want, make pub(in crate)).


Something less hacky?

The current solutions are hacky since they rely on traits, which mimic the OP's API usage as much as possible, but are not, in Rust, the right tool for this.

Inheritance-based patterns ought to be replaced by composition, and this is no exception:

if, for instance, need private fields on your Base C++ class,

pub
trait Base {
    fn custom_behavior (self: &'_ mut Self)
    ;
}

struct WithBase<B : Base> {
    inner: B,
    // + private fields of "C++'s `Base`"
}
impl<B : Base> WithBase<B> {
    fn foo (self: &'_ mut Self)
    {
        self.common_behavior();
        self.inner.custom_behavior();
    }

    fn common_behavior (self: &'_ mut Self)
    {
        // ...
    }
}

and, otherwise, and quite simpler:

  • In hindsight the first solution I have offered is actually interesting when you want a non-overridable default impl, but whose usage can be public (paramount if you want to have a non-unsafe trait offering an unsafe method, for instance)
pub
trait Base {
    fn custom_behavior (self: &'_ mut Self)
    ;
}

pub(in crate)
trait PrivateBaseMethods : Base {
    fn foo (self: &'_ mut Self)
    {
        self.common_behavior();
        self.custom_behavior();
    }

    fn common_behavior (self: &'_ mut Self)
    {
        // ...
    }
}
impl<T : ?Sized + Base> PrivateBaseMethods for T {}
  • This, in fact, is just an extension trait (i.e., method call sugar for what could simply have been a bunch of classic (generic) function definitions), which happens to be private (like the functions would have been).
4 Likes

"Traditional classes" as provided in C++, Java, etc are a grabbag of several theoretically unrelated features which in Rust are mostly separate and orthogonal, such as modules, privacy, runtime polymorphism, inheritance, etc. IMO this question is a classic example of how that grabbag causes us to frame a very simple use case in terms of several unrelated features that make it much harder than it really needs to be. Rust simply doesn't have traditional inheritance, so we're effectively forced to tease apart which of the many pieces of inheritance we're actually interested in here.

By far the simplest option is to make Algorithm (let's stop calling it Base) a struct that contains a closure which you pass to it on construction.

struct Algorithm<F: Fn()->()> {
    f: F
}

impl<F: Fn()->()> Algorithm<F> {
    fn new(f: F) -> Algorithm<F> {
        Algorithm{ f }
    }
    fn usage(&self) {
        println!("begin usage");
        (self.f)();
        println!("end usage");
    }
}

fn main() {
    let a = Algorithm::new(|| println!("foo"));
    a.usage();
}

IIUC this will fully monomorphize the "customized function" and thus should have the same performance characteristics as typical C++ NVIs.

If your use case needs more than just this (which is entirely possible), then we need to hear more details to recommend the best way of implementing it.

5 Likes

I will need to re-read more than once all of your responses. For some reason it's hard for me to connect the dots!


I will rephrase my need using Rust-like syntax. Let's just suppose for a moment that any public function in a trait need to marked with pub, and by default they are private. Let's also suppose that final applied to a function in a trait means that this function cannot be overwritten by implementation of that trait. All implementation of that trait would share the same implementation. This effectively means that even if this function is called though a dyn MyTrait, a final function would be statically dispatched. Side effect: this also means that you can't access any field directly in a final function since the implementation in the trait itself (as thus doesn't know what the fields of the concrete type are), but you can use private accessors if need be.

trait Algorithm {
    // public final function, cannot be overwritten
    pub final fn do_it(&mut self) {
        self.common_part();
        self.customization_point();
    }

    // private final function, cannot be overwritten
    final fn common_part(&mut self) { /* ... */ }

    // private non-final (virtual) function, can and must be overwritten
    // EDIT: non-final functions are what currently any function in trait are in Rust
    fn customization_point(&mut self);
}

struct VariantA { /* ... */ };
impl VariantA { /* ... */ };
impl Algorithm from VariantA {
    // Note: this function is both the only one that can be specialized, and must be specialized
    fn customization_point(&mut self) { /* ... */ }
}

struct VariantB { /* ... */ };
impl VariantB { /* ... */ };
impl Algorithm from VariantB {
    fn customization_point(&mut self) { /* ... */ }
}

pub fn usage<T: Algorithm>(algo: &mut T) {
    algo.do_it();
}

pub fn main() {
    let variant_a: impl Algorithm = VariantA::new();
    usage(&mut variant_a); // static dispatch

    let vec_variants: Vec<Box<dyn Algorithm>> = vec![Box::new(variant_a), Box::new(VariantB::new())];
    for v in &mut vec_variants {
        // I'm not 100% confident with the syntax of how to acces references to element of a vec in a box
        // but you should get the idea
        usage(v); // dynamic dispatch
    }
}

To give a concrete example, let's say that I want to display a list of numbers, and based on their value, they should be bold, strike, or without formatting. That formatting is an abstract property, because I will have two implementations: one for markdown, and one for html.

trait TextFormatter {
    // let's assume that we can use impl in trait. We can get around it, but this adds some boilerplate.
    fn bold(&self) -> impl Display;
    fn strike(&self) -> impl Display;
}

pub void print_numbers<T: TextFormatter>(format: &T, numbers: &[usize]) {
    for n in &numbers {
        if n == 0 {
            println!("{}", format.strike(n));
        } else if n == 10 {
            println!("{}", format.bold(n));
        } else {
            println!("{}", n);
        }
    }
}

struct MarkdownFormatter;
impl TextFormatter for MarkdownFormatter { /* ... */ }

struct HtmlFormatter;
impl TextFormatter for HtmlFormatter { /* ... */ }

pub fn main() {
    let numbers = [1, 2, 10, 11];
    if (use_html) {
        print_numbers(&HtmlFormatter, &numbers);
    } else if (use_markdown) {
        print_numbers(&MarkdownFormatter, &numbers);
    }
}

Then we have current discussion of how to have the best implementation to have some part of the formatting common to all implementation of TextFormatter without allowing print_numbers() to have access to any of those implementation details.

This makes sense, but I don't think it's much of a complication. My knee-jerk reaction is to simply take the code from my previous post and swap Fn()->() for TextFormatter, e.g. struct Algorithm<TF: TextFormatter> { tf: TF }.

(I wanted to write out actual code, but the TextFormatter pseudocode above is incomplete/uncompilable in so many ways I'd likely butcher your intent trying to make that fully concrete)

Given that example, which works would work fine (if it weren't for the impl Display), I guess you already have your answer: a normal (generic) function seems to fulfill your needs, so I guess what you are after is some good old .method() sugar.

In Rust, "converting" a classic function to method sugar is done through what is called extension traits (c.f. PrivateBaseMethods in my previous post): its methods can be seen as final and statically dispatched1, and depending on whether the (extension) trait is public or not, these final methods will be pub or not. Here goes the pattern applied to your example:

use ::core::fmt;

pub
trait TextFormatter {
    fn bold (self: &'_ Self, it: &'_ impl fmt::Display)
      -> String
    ;

    fn strike (self: &'_ Self, it: &'_ impl fmt::Display)
      -> String
    ;
}

struct MarkdownFormatter;
impl TextFormatter
    for MarkdownFormatter
{
    fn bold (self: &'_ Self, it: &'_ impl fmt::Display)
      -> String
    {
        format!("**{}**", it)
    }
    
    fn strike (self: &'_ Self, it: &'_ impl fmt::Display)
      -> String
    {
        format!("~~{}~~", it)
    }
}

struct HtmlFormatter;
impl TextFormatter
    for HtmlFormatter
{
    fn bold (self: &'_ Self, it: &'_ impl fmt::Display)
      -> String
    {
        format!("<b>{}</b>", it)
    }
    
    fn strike (self: &'_ Self, it: &'_ impl fmt::Display)
      -> String
    {
        format!("<del>{}</del>", it)
    }
}

pub
trait TextFormatterExt : TextFormatter {
    fn print_numbers (self: &'_ Self, numbers: &'_ [usize])
    {
        for n in numbers {
            match *n {
                | 0 => println!("{}", self.strike(n)),
                | 10 => println!("{}", self.bold(n)),
                | _ => println!("{}", n),
            }
        }
    }
}
impl<T : ?Sized + TextFormatter> TextFormatterExt
    for T
{}

fn main ()
{
    let ref numbers = [1, 0, 2, 10, 11];
    if ::rand::random() {
        HtmlFormatter.print_numbers(numbers);
    } else {
        MarkdownFormatter.print_numbers(numbers);
    }
}

1 If trait objects are involved, there will alway be a dynamic dispatch at some point, but for the print_numbers case, for instance, you will never have both the print_numbers call be virtual and the .bold(), .strike(), ... call it makes be virtual too. If you use dyn TextFormatter, then the print_numbers() call will be statically dispatched, but each .bold(), ... call in its body is a virtual call, and if you use dyn TextFormatterAux, then .print_numbers() will be dynamically dispatched but the inner .bold(), ... will be statically dispatched: Playground

3 Likes

Not exactly, and that's the issue of writing an example that's too simple, while being realistic. In the example I gave, there was only the public interface of the TextFormatter concept. I will use again the Rust-like syntax with explicit pub and the final keyword.

use std::fmt;
use std::fmt::{Display, Formatter};

trait TextFormatter {
    // public function, cannot be overwritten
    pub final fn bold<T: Display>(&self, value: T) -> impl Display {
        struct Bold<TF: TextFormatter>(T: Display);
        impl Display for Bold {
            fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
                TF::bold_impl(self.0)
            }
        }
        Bold<TF>(value)
    }

    // public function, cannot be overwritten
    pub final fn strike(&self) -> impl Display { /* same logic than bold */ }

    // private function, must be overwritten
    fn bold_impl<T: Display>(value: T);

    // private function, must be overwritten
    fn strike_impl<T: Display>(value: T);
}

enum MarkdownFormatter{}
impl TextFormatter for MarkdownFormatter {
    fn bold_impl<T: Display>(f: &mut Formatter<'_>, value: &T) -> fmt::Result {
        write!(f, "**{}**", value)
    }
    fn strike_impl<T: Display>(f: &mut Formatter<'_>, value: &T) -> fmt::Result {
        write!(f, "~~{}~~", value)
    }
}

enum HtmlFormatter{}
impl TextFormatter for HtmlFormatter { /* implement bold_impl() and strike_impl() */ }

There are a few things that are important:

  • bold() and strike() must be part of the public API, and should not be overwritten
  • bold_impl() and strike_impl() should not be part of the public API, and must be defined in any implementation of TextFormatter.
  • It should also be impossible by construction that an implementation of TextFormatter forgot to call the common part. The following C++ code is bad, because it's possible to forgot to call the function important_validation().
class Algorithm {
    public: virtual void do_something();
    protected: void important_validation() final;
}
class Implementation: public Algorithm {
    protected: void do_something() override {
         // oups! I forgot to call import_validation() here
         do_something_dangerous();
    }
}

Note: if you want to see how I emulate the impl Display, you can look this playground link and the details of how I came to this in a blog post I wrote, but this isn't directly related to this conversation so I didn't wanted to add the syntastxic noice.

You can iterate with the ideas I've suggested to achieve what you want:


emulate the impl Display

Yeah, I wanted to avoid the boilerplate for the example, but since it turns out that the boilerplate is kind of the reason for your requirement I've decided not to avoid it this time :wink:

1 Like

EDIT: @Yandros we found something more or less at the same time. My implementation doesn't requires macros. Could you please take a look and tell me what you think of it? I found this discussion really interesting, and I'm learning a lot.

I found a way to do it.

I will give you gist, without the shenanigan to make impl Trait in Trait work because it's much clearer:

use std::fmt;
use std::fmt::{Display, Formatter};
use std::marker::PhantomData;

// First, the public trait
pub trait TextFormatter 
{
    fn bold<T: Display>(value: T) -> impl Display;
    fn strike<T: Display>(value: T) -> impl Display;
}

// Then the implementation of the common bits
// It can use the internal parts internally
impl <TF: private::TextFormatterImpl> TextFormatter for TF {
    fn bold<T: Display>(value: T) -> FormattedBold<Self, T> {
        struct FormattedBold<F: TextFormatter + ?Sized, T: Display>(T, PhantomData<F>);
        impl<F: private::TextFormatterImpl, T: Display> Display for FormattedBold<F, T>
        {
            fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
                F::bold_impl(f, &self.0)
            }
        }
        FormattedBold(value, PhantomData)
    }

    fn strike<T: Display>(value: T) -> FormattedStrike<Self, T> {
        pub struct FormattedStrike<F: TextFormatter + ?Sized, T: Display>(T, PhantomData<F>);
        impl<F: private::TextFormatterImpl, T: Display> Display for FormattedStrike<F, T> {
            fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
                F::strike_impl(f, &self.0)
            }
        }
        FormattedStrike(value, PhantomData)
    }
}

// Then the specification of the private parts
mod private {
    use std::fmt;
    use std::fmt::{Display, Formatter};
    
    pub trait TextFormatterImpl {
        fn bold_impl<T: Display>(f: &mut Formatter<'_>, value: &T) -> fmt::Result;
        fn strike_impl<T: Display>(f: &mut Formatter<'_>, value: &T) -> fmt::Result;
    }
}

// Implementing the private part automatically make it implement the public API
pub enum MarkdownFormatter{}
impl private::TextFormatterImpl for MarkdownFormatter {
    fn bold_impl<T: Display>(f: &mut Formatter<'_>, value: &T) -> fmt::Result {
        write!(f, "**{}**", value)
    }
    fn strike_impl<T: Display>(f: &mut Formatter<'_>, value: &T) -> fmt::Result {
        write!(f, "~~{}~~", value)
    }
}

// Use the public API
pub fn usage<T: TextFormatter>(some_value: i32) {
    println!("A number in bold: {}", T::bold(some_value));
    println!("A number striked: {}", T::strike(some_value));
    // by construction, it's not possible to call the private part of TextFormatterImpl
}

pub fn main() {
    usage::<MarkdownFormatter>(42);
}
1 Like

Yes, our soulutions are pretty much the same indeed :smile:. That being said:

  • Documentation-wise, I like to see in the rendered docs that the return type of a trait's method (that, in practice, is just an impl Display) be named something like implDisplay. Hence my usage of helper modules to avoid name clashing, and factoring all that with a macro (of course not necessary);

  • You have chosen a design where a TextFormatter instance is deprived of runtime information (no &self parm; only type-level / compile-time information). I was tempted to go in that direction too (it is, imho, more elegant to stay in the type-level realm), but at the end of the day, I chose to use unit structs with &self state available instead, even if I'm not using it anywhere. This is because, if later on, someone realises they need an instance with runtime state, it is easy to add a struct ... {/* state */} impl TextFormatterImpl for ... to achieve it, where in the pure type-level case a rework would be necessary.

  • You have used an associated type (Self::Self_ : Impls) where I haven't; I hadn't thought of doing that, to be honest. I even think I prefer your associated type approach a bit more, since it is more "compositional" rather than inheritance-based :slightly_smiling_face: ; but it relies on the state-less property I mentioned (otherwise you'd need to add a get_Self_(&self) -> &'_ Self::Self_ or something along those lines).

Overall, I think the fact we have arrived to the same solution has been indeed quite interesting, so as to compare the little differences between the two approaches.

This is indeed a good idea. And here you go!

pub trait TextFormatter
{
    type Self_: private::TextFormatterImpl;

    fn bold<T: Display>(value: T) -> FormattedBold<Self::Self_, T>
    where FormattedBold<Self::Self_, T>: Display {
        FormattedBold(value, PhantomData)
    }

    fn strike<T: Display>(value: T) -> FormattedStrike<Self::Self_, T>
    where FormattedStrike<Self::Self_, T>: Display {
        FormattedStrike(value, PhantomData)
    }
}

It's very visible in the documentation generated by cargo doc that the output implements Display.


First version that doesn't pass self all around.
Second version that pass &self all around. There is quite a lot of differences. Most notably, we need to have explicit lifetime everywhere, I had to move the implementation of bold() and strike() in the private module (l36-39) since FormattedBold takes a &TextFormattedImpl from now on, and finally MarkdownFormatter needs to be a unique struct – instead of a stateless enum – since we need to be able to instantiate it.

1 Like

@Yandros What I like with out of our approach so far:

  • The public interface is clearly separated from the private interface
  • The private interface doesn't leak

What I dislike:

  • When you implement the private interface (TextFormatterImpls), your types implements the public interface (TextFormatter) automatically, but implicitly. This make it not really clear.

Instead of

pub struct MarkdownFormatter;
impl private::TextFormatterImpl for MarkdownFormatter { // <-- this line
    fn bold_impl<T: Display>(&self, f: &mut Formatter<'_>, value: &T) -> fmt::Result {
        write!(f, "**{}**", value)
    }   
    fn strike_impl<T: Display>(&self, f: &mut Formatter<'_>, value: &T) -> fmt::Result {
        write!(f, "~~{}~~", value)
    } 
}

I would have prefered a lot to be able to do

pub struct MarkdownFormatter;
impl TextFormatter for MarkdownFormatter { // <-- this line is different
    fn bold_impl<T: Display>(&self, f: &mut Formatter<'_>, value: &T) -> fmt::Result {
        write!(f, "**{}**", value)
    }   
    fn strike_impl<T: Display>(&self, f: &mut Formatter<'_>, value: &T) -> fmt::Result {
        write!(f, "~~{}~~", value)
    }
}

But I cannot find a way to do it without having the private interface leaking.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.