Design patterns for composability with traits (i.e. typeclasses)?

[keean]

How will you determine how many dictionary slots to have in your 'box' type? I still think this reduces to the same problem as my solution using type classes, and hence I would rather not introduce union types when they don't seem necessary. I think we will have to explore the implementation to see if these two are equivalent, as I think. I already have an implementation plan. The module interface files generated by the compiler already have to include the trait bounds for traits, so we have all the information when compiling to compose bound 'extensions' at link time, it would just mean deferring the memory layout of boxed traits until all modules have been processed, and the final set of bounds accumulated.

Regarding the tangent, I think coercions are a bad thing, and I favour only explicit conversions (this also helps type inference, and results in less type annotations).

[keean]

@shelby3

To my eye, it is redundant to dictate a distinction between access and scoping because the uppercase restriction in my grammar makes it obvious and also naming is more consistent. And distinguishing access to methods which don't input &self is also obvious because type names begin with an uppercase letter and instance names do not. For me, language design is an artform and I want it to be elegant like a painting. Einstein also wanted his theories to be elegant, which is one reason he resisted quantum theory (and I think he will be proven correct).

I am not bothered by ':' in type definitions. I tend to favour using lower-case for types and upper case for type variables like in a Prolog program, but I think this is down to personal taste. I don't like significant whitespace as it makes cutting and pasting code very difficult, and you cannot use code auto-layout tools to fix someone elses code to meet your own preferred styling.

However, I suspect any 'logical' arguments here are post-rationalisation. The great thing about creating your own language is you can do everything exactly as you want. The bad thing is that it splits the user base of existing languages. This is what lead to the creation of Haskell, originally there were many competing individual lazy-pure-functional languages, so all the different authors got together in a committee and created Haskell. I suspect that nobody is completely happy with the design of Haskell, but everyone can live with the compromises, and it results in a bigger community, and a common platform for further research.

I also thing imperative code is easier to read and more understandable than the functional style. This is due to the objects having stable types, and visible indexing. The functional style of chaining maps and folds results in invisible intermediate types that make reading someone else's code much harder. I call it "write only code" As debugging is harder than writing code, and as code will be in maintenance for longer than it is in development, I think we should optimise code for readability and understandability. If you use all of your intelligence to write the code, then because debugging is harder, you will be by definition not clever enough to debug it. Complexity is the problem, and the solution is that we should strive for simplicity. Here I mean simplicity of concept, not simplicity of grammar or less typing.

[shelby3]

@keean

I am not bothered by ':' in type definitions. I tend to favour using lower-case for types and upper case for type variables like in a Prolog program

If you use lowercase for types, then you can't distinguish then from instance (aka variable) names except by context. In C++/Java/Scala, type parameters (aka variables) are usually all uppercase and type names are Camel case with first letter uppercase. Variable (and function/method) names are usually lower case or first letter lower case with Camel case for function names.

C was a very good language because it had a minimum of syntax. Symbol soup looks noisy. The dot operator can be thought of narrowing/accessing the type, so it makes sense we should use the dot operator for both type member access (since those members are part of the structural type even if we aren't employing structural typing else where in the language) and Type scoping is narrowing/accessing the nominal type.

@keean

The bad thing is that it splits the user base of existing languages.

Not necessarily. Did Python split any existing user base? Ruby's? Rather I think Python identified a niche for readable consistency.

If someone identifies a niche and hits that niche on the target, they can succeed.

So the key is identifying Rust's niche and focus on it.

Haskell's niche is apparently people who want to code like math. It isn't just that it is a functional language, but the lazy evaluation which enables expressing infinite series and other math.

@keean

I also thing imperative code is easier to read and more understandable than the functional style. This is due to the objects having stable types, and visible indexing. The functional style of chaining maps and folds results in invisible intermediate types that make reading someone else's code much harder. I call it "write only code"

I agree about "write only" code and as I wrote in another post today, this is likely because of Haskell's global type inference (which btw prevents a first-class union and thus Haskell can never implement my proposal in this thread).

Btw, afaics functional programming can be readable by annotating the code with the types. Afaics, readability can be higher in a functional style, if done well.

@shelby3

Rust and Haskell both have typeclasses and afaics they both need HKTs. Arguably what makes Haskell so obtuse are:

1. Uses spaces instead of commas and parenthetic grouping for function application.
2. Has global type inference so the function, data type, and typeclass (aka trait) signatures are rarely displayed explicity.
3. Is 100% immutable, thus requires category theory monads to emulate mutability.
4. Has lazy evaluation and memoization which complicates memory space and timing design and analysis.
5. Abstract data types (ADTs) are more powerful (in some ways and less in others) and less verbose boilerplate to accomplish the same employing struct, trait, mod, and enum. This conciseness and lack of verbiage, along with being an unfamiliar declaration to C/C++/Java programmers, makes it obtuse to some.
6. Etc.

My point is that Rust has many of the same roughly equivalent abstractions that Haskell has, and will probably need HKTs as well. But it doesn't necessarily have to be as obtuse to C/C++/Java programmers as Haskell is.

P.S. It is Sunday and I have an obligation to be at a beach party, so further replies will be delayed.

[keean]

@shelby3

If you use lowercase for types, then you can't distinguish then from instance (aka variable) names except by context. In C++/Java/Scala, type parameters (aka variables) are usually all uppercase and type names are Camel case with first letter uppercase. Variable (and function/method) names are usually lower case or first letter lower case with Camel case for function names.

I don't disagree with this, but I think I want to distinguish between type variables and types too. Most of my coding does not involve concrete types, so having type variables distinguishable seems better to me.

@shelby3

Not necessarily. Did Python split any existing user base? Ruby's? Rather I think Python identified a niche for readable consistency.

I think they both took users away from Perl There are only so many programmers so new languages must take away from somewhere.

@shelby3

Btw, afaics functional programming can be readable by annotating the code with the types. Afaics, readability can be higher in a functional style, if done well.

Well more different types in an expression is worse, and having to add type annotations is worse. It also doesn't solve the indexing problem, and that is having variables for index lookup is much clearer than maps and folds especially when you have more than one index in play. See my permutations example in JavaScript I posted.

[marcianx]

@shelby3

Thanks. I was aware of that. I was also referring to needing to put return twice, having to use braces instead of =.

So to clarify, you now know you don't have to put return even once as in my second example? In case that wasn't as clear, omitting the final semicolon on any block returns the value of the last statement. E.g. equivalent to your first version:

fn foo<T, C>(x: C) where T:Add+Nullable, C:Enumerable<T> {
   fn f(prev: T, cur: T) {
      if prev == null { cur } else { prev + cur }
   }
   x.fold(null, f);
}

Omitting return when it's not needed is the preferred convention, I believe.

Rust's lambda syntax, I believe, was inspired by Ruby's block syntax, though it's slightly different from it (in Ruby, the |...| goes inside the braces, whereas in Rust it's outside.) Rust used to emulate Ruby's block-passing syntax but by using the do keyword. This syntax was removed in RFC: remove `do` · Issue #10815 · rust-lang/rust · GitHub

[shelby3]

@marcianx

In case that wasn't as clear, omitting the final semicolon on any block returns the value of the last statement.

Sorry to be pedantic, and yes I was aware of that special rule and frankly I think that is silly that only don't need to put a semicolon on the last block that is the result expression. I'd prefer never put semicolons and the result expression would also be the value of the last expression in the block, or () if the last statement in the block is not an expression, e.g. a while block.

@marcianx

So to clarify, you now know you don't have to put return even once as in my second example?

if prev == null { cur } else { prev + cur }

I knew that return was optional on the last line, but I didn't know if Rust was capable of recognizing if-else as an expression (and I actually thought of checking it since Scala can, but...). I don't make such assumptions (and hadn't bothered to try it because I like general, consistent rules so I don't need to try it in order to not be surprised), because of the special case rules instead of a general one that everything is a expression (and not statement which I think one of the reasons Scala didn't include while) or my generalized variant (to allow for while statements).

@marcianx

Rust's lambda syntax, I believe, was inspired by Ruby's block syntax, though it's slightly different from it (in Ruby, the |...| goes inside the braces, whereas in Rust it's outside.) Rust used to emulate Ruby's block-passing syntax but by using the do keyword. This syntax was removed in RFC: remove `do` · Issue #10815 · rust-lang/rust · GitHub

The 2 cents issue for me with lambdas is consistency. Since -> is used to indicate result type on functions, it would be more consistent IMO if it is also used to indicate the result expression for lambdas (anonymous functions). Here I must also criticize Scala for using : for function result types (consistent with function arguments yet...) and => for lambdas (which is non-standard from the -> of Haskel et al, and appears to be an attempt at consistency with = for function blocks that are expressions instead of brace-delimited blocks).

I am just arguing that lack of special cases, makes memorizing and learning a language less surprising.

[marcianx]

Oh, I didn't find it pedantic at all, and I'm glad you were aware. Your specific before and after examples left me guessing as to the precise points of confusion, so I tried to make sure I addressed the ones I guessed.

Regarding syntactical changes — you were comparing the lambda syntax to other languages; I only meant to provide you context for this choice (most likely Ruby), not argue for or against any specific syntax.

That being said, a few newcomers have previously argued for changes to syntax post rust stabilization and the invariable answer, understandably, from the development team has been that the time for that bikeshedding is long past. Rust has a lot of important things in its roadmap to develop toward and backward-incompatible syntactical changes on a stabilized language would be a tremendous burden to the community.

In any case, if you feel the need to argue for it further, perhaps a new thread would be more appropriate.

[shelby3]

Thanks also.

@marcianx

from the development team has been that the time for that bikeshedding is long past

Note I am just expressing (my "2 cents") what I would do differently and what the motivations might be for me to maybe create my own programming language, rather than adopt Rust for my current targeted need:

@shelby3

I am not trying to change Rust. I am merely trying to analyze what is and possibly offer "win-win" suggestions that can work from where Rust is now, not changing what is already water-under-the-bridge.

Note also, that I am interested in identifying Rust's target focus, so I might (and others do apparently) find a use case for Rust:

@shelby3

If someone identifies a niche and hits that niche on the target, they can succeed.

So the key is identifying Rust's niche and focus on it.

[shelby3]

@keean

Well more different types in an expression is worse, and having to add type annotations is worse.

I believe we are in agreement?

@shelby3

@keean

Returning to the topic, I'm fairly new to Rust, but have worked with type-systems for a while, I am coming to the conclusion that HKT are not going to be a problem... I think if you have type inference, adding complexity to the type system to make the value-level code simpler is okay, as you should not have to think about the type system to write simple programs.

I presume you mean local type inference of "expressions" (i.e. not having to explicitly specify for example the types of a lambda function) and not global inference, since I assumed we both agree that global inference elides type annotation necessary to see the polymorphic type class (aka trait) bounds on a function.

[shelby3]

@shelby3

Symbol soup looks noisy.

Example, the following I see the pattern :: more than I see the names (which can be alleviated somewhat with syntax coloring and setting the :: to display in low contrast to the background):

@kirillkh

• Vec::iter() gives a std::slice::Iter
• Vec::iter_mut() gives a std::slice::IterMut
• Vec::into_iter() gives a std::vec::IntoIter

I'd prefer to write that as follows where the . almost disappears:

• Vec.iter() gives a Std.Slice.Iter
• Vec.iter_mut() gives a Std.Slice.IterMut
• Vec.into_iter() gives a Std.Vec.IntoIter

@keean

I am not bothered by ':' in type definitions.

Two dots : is less (½ as?) distracting than four ::, but still IMO more (twice as?) distracting as one .

Perhaps different people visualize code differently. I seem to have a strong visual pattern matching which is difficult for me to turn off, thus the :: is akin to a lawn mower outside while I am trying to listen to music. I don't seem to be autistic, but perhaps that is a slightly autistic type of skill/handicap although I wonder if others have the same distraction. Note it is not incredibly distracting and I am being perfectionist/pedantic with this comment. What happens is I need to make a conscious effort to read it versus just having it sort of instantly absorbed (wherein I am reading it but it is probably like driving where we do it unconsciously because of muscle memory).

But note I don't prefer eliding punctuation which makes the code less readable, e.g. arguably Haskell's elision of parenthesis around function arguments and spaces instead of comma-delimited list of arguments. Haskell is requiring me to lookup or memorize the function declarations so I know how many arguments each function consumes.

Edit: perhaps it is more distracting because combined with doubling a colon, a single colon has a normal meaning in English, math, and computing such that it shouldn't be used as a delimiter, and especially given the delimited names can be as short as 2 or 3 characters in length. The valid use of a colon in English and computer languages (other than functional programming Cons) is to introduce a part of a sentence that is a logical continuation of the part before it. And for math, predominately to mean "such that". Thus it doesn't make sense to have colon-delimited list, since there should only be one part before and aft. Whereas, historically the single dot period has been a delimiter in computing (not just computer languages).

[shelby3]

@marcianx

As for the ternary operator, it was removed in its infancy Remove ternary operator · Issue #1698 · rust-lang/rust · GitHub with a recently closed attempt to resurrect it https://github.com/rust-lang/rfcs/issues/1362. I do favor its brevity, however.

I agree with much of the logic against including the ternary operator, but only if if-else isn't forcing that noisy brace-delimited syntax:

if !prev { cur } else { prev + cur }

I'd prefer:

if !prev: cur else prev + cur

Or even more clear:

if !prev then cur else prev + cur

Than:

!prev ? cur : prev + cur

But remove syntax coloring and assume we are using a plain text editor:

if !prev { cur } else { prev + cur } <--- this style isn't available in a Haskell/Python block indenting instead of brace-delimited

if !prev: cur else prev + cur

if !prev then cur else prev + cur

Then the English text variants render low contrast confusing the keywords and the adjacent juxtaposed fragments of code, thus we either need the brace-delimited or the operator delimited ternary:

!prev ? cur : prev + cur

Edit: I added a comment at the issue tracker linking to this post.

Edit#2: I'm thinking the best would be to force only ternary on single or double line if and if-else:

!prev ? cur
!prev ? cur : prev + cur

!prev ? cur
      : prev + cur

But I'm not sure if I prefer that compared to:

if !prev: cur

if !prev: cur else prev + cur

if !prev: cur
else prev + cur

Or:

if (!prev) cur

if (!prev) cur else prev + cur

if (!prev) cur
else prev + cur

Or:

if !prev then cur

if !prev then cur else prev + cur

if !prev then cur
else prev + cur

Then for multi-line blocks either brace-delimited or my preferred indented delimited with keywords:

if !prev {
   cur
}
else {
   prev + cur
}

if !prev
   cur
else
   prev + cur

[shelby3]

@keean

Can we talk about some code? How about this: Rust Playground

We have 3 modules, one defines circle, one square and one a shapes collection and processing function. We can put the circle in the shapes collection because it implements HasArea. We can easily extend with the new type Shape that is defined in a separate module by defining an implementation of HasArea, all without changing any of the code in modules a, b & c. Good so far.

It doesn't meet the requirement I stated in the OP of the thread, which is if the API we can't edit provides us a collection of trait Foo (not trait HasArea) and we want a collection of trait Shape then your proposal doesn't suffice:

@shelby3

In no case should the solution require rebuilding the collection to convert types.

Which is why I said we must not discard the data types and thus can't use a trait as the element type of the collection. So your idea is not even equivalent to my proposal in terms of solving the problem I laid out in the OP nor solves the use-case example I provided.

@keean

To extend in the other direction, lets say we want to get the colour of every shape in the collection, we can obviously define new implementations of HasColour for the shapes that need it (Circle) in the same way we did for HasArea, the problem is we are required to modify module c to change the collection trait.

If Rust had first class intersection (conjunctions), then we could just declare the element type of the collection to be Shape /\ Color, but that still wouldn't be a substitute for my proposal.

@keean

We could fix this by allowing 'main' to extend the trait bounds of Shape, as I suggested. This will require some form of whole program analysis to work out the size of the Box (but I think it will however you try and solve this).

@keean

I still think this reduces to the same problem as my solution using type classes, and hence I would rather not introduce union types when they don't seem necessary. I think we will have to explore the implementation to see if these two are equivalent, as I think. I already have an implementation plan. The module interface files generated by the compiler already have to include the trait bounds for traits, so we have all the information when compiling to compose bound 'extensions' at link time, it would just mean deferring the memory layout of boxed traits until all modules have been processed, and the final set of bounds accumulated.

I had already explained upthread that your idea couldn't work, but my conclusion was based on your prior confusing description of using scoping of the trait for the element type of the collection to model adding a union of new data types to collection and binding them to the unbounded varieties of trait bounds for any function that wants to input the collection.

Now you seem to be describing something different which doesn't even accomplish the requirements I laid out in the OP, thus isn't equivalent in expression and solving the Expression Problem compared to what I am proposing.

The ONLY way to solve the problem I am articulating is with my proposal.

[keean]

@shelby3

I had already explained upthread that your idea couldn't work.

I did not agree with your explanation. I think that this will work just as well as the union idea, and has precisely the same consequences on compilation.

@shelby3

The ONLY way to solve the problem I am articulating is with my proposal.

Can you explain concisely what the problem is you are trying to solve? The solution I proposed solves the expression problem whilst still using traits. What problem are you trying to solve?

I have explained how I would implement the trait solution, and as the definition of soundness is that the type system only admit correct programs with respect to the implementation, then if the implementation works for all the type semantics it is by definition sound.

In this case we are starting with a sound type system, and extending it with a single property (trait extension), all we need do is prove we have an implementation for trait extension to prove soundness.

[shelby3]

@keean

The solution I proposed solves the expression problem whilst still using traits. What problem are you trying to solve?

No it doesn't. I explained in the prior post what it doesn't solve. Why are you suggesting code and substitutes when you ostensibly don't even understand yet the problem I am trying to solve. Wouldn't it behove you to re-read the thread again first. Perhaps on second reading from start to finish, I won't be required to repeat myself. If it is still unclear after re-reading, then I can endeavor to clarify what is not clear based on questions.

@keean

In this case we are starting with a sound type system, and extending it with a single property (trait extension), all we need do is prove we have an implementation for trait extension to prove soundness.

But irrelevant, because it doesn't solve the Expression Problem as I have explained the problem that can't be solved with any form of monkeying around with trait objects.

@shelby3

Afaics there simply isn't any way to remove the invariant that the data types that were already in the collection have been erased from the compiler's AST by the time you need to add the new trait to the conjunction which is the element type.

[keean]

The suggestion I made allows new types to be added to a collection without touching code outside of the new module you are implementing, you simply 'impl' all the traits required to insert the type in the collection. It also allows you to add new traits to the collection by 'extending' the bounds of the collection trait, like "extend Shape with HasArea", providing you provide 'impl' for each type that might be inserted into the collection. This provides extension in both directions, without having to modify any of the existing code whether in the same module or a different module.

@shelby3

Afaics there simply isn't any way to remove the invariant that the data types that were already in the collection have been erased from the compiler's AST by the time you need to add the new trait to the conjunction which is the element type.

I don't see how this matters. We don't need to know the types, in fact it is important we don't (otherwise we don't really have runtime polymorphism).

[shelby3]

@keean

The suggestion I made allows new types to be added to a collection without touching code outside of the new module you are implementing, you simply 'impl' all the traits required to insert the type in the collection.

You are not understanding the Expression Problem. Your suggestion does nothing to solve the problem I stated upthread that the consuming function may need an entirely different trait than the one in the collection's element type trait object. I repeated this so many times already. I just repeat this twice again in the past 2 minutes.

[keean]

@shelby3

the consuming function may need an entirely different trait than the one in the collection's element type trait object

Right but then you can locally extend the containing trait with the one you want. We keep the set of trait bounds on the Boxed trait open.

[shelby3]

@keean

Right but then you can locally extent the containing trait with the one you want.

My gosh, in the prior 2 minutes, I already explained why you can't:

@shelby3

It doesn't meet the requirement I stated in the OP of the thread, which is if the API we can't edit provides us a collection of trait Foo (not trait HasArea) and we want a collection of trait Shape then your proposal doesn't suffice

[keean]

@shelby3

It doesn't meet the requirement I stated in the OP of the thread, which is if the API we can't edit provides us a collection of trait Foo (not trait HasArea) and we want a collection of trait Shape then your proposal doesn't suffice

I really don't understand what you are trying to say here. We can extend Foo with Shape at any time in any module using the new feature of trait extension (not implemented in Rust or Haskell), which I have proposed an implementation method for.

[shelby3]

@keean

I really don't understand what you are trying to say here. We can extend Foo with Shape

If the element type of a collection provided to you (which you can't edit) is for example trait Foo trait object and your function wants a trait Shape trait object bounds, the compiler has erased the data types thus you have no way to know which data types are in that collection. There is nothing you can do with trait extensions to make the data types come back after they've been discarded from the compiler's AST (or not even provided in the case of a DLL or separately combined linkable modules).