Collecting Requirements for Rust GUI Library

1

Hello Rust community,

Two years ago I tried to develop my own software (https://devev.com/) in Rust, when I ran into a wall while implementing the user interface: there were no reasonable options available. I decided to write a binding engine between Rust and TypeScript so I could build an electron UI but I eventually moved the entire code base to TypeScript to reduce complexity in the project.

After looking at the Rust survey for 2019 this still looks like a big problem.

At this point I'm a much more experienced UI developer. In my free-time I'm building the software I mentioned above using web technologies. Professionally, I implemented UI in the Forza video game series for the past 4 years. Forza uses an in-house markup language and UI renderer so I have experience building these technologies.

I think I can have impact on the UI problems Rust is facing and I'm interested in gathering a list of requirements so I can make sure I'm actually solving the communities problems and not just building another UI library for the sake of it. I was hoping this thread could be a kicking off point for a discussion about what users would like to see.

I would like to start with what I see as two major requirements for an MVP:

  1. Web support
  2. Support for a Markup Language

By targeting the web I could leverage electron for the other platforms at first. Eventually each platform could have a specific implementation for performance but there's a lot of existing open source work that makes the web a low hanging fruit and more people are there than any other platform.

With respect to a markup language, I think this allows for other things to be built on top of the technology. In particular I'd eventually like to support exporting from Figma and/or Sketch into the markup language so that UI can be developed flexibly and reloaded without a compile. This would also make things like editors and debuggers easier to implement (think of the devtools built into the browser).

I'm interested to know what requirements others have and whether they align with mine. I don't want to develop yet another GUI technology for Rust if the community believes that an existing technology will make it far enough to fulfill their goals.

10 Likes
2

I think, there should be an intermediate layer/language development. We may write like some Generic GUI requirements in that intermediate language and then the language may be able to translate our requirements to any UI like web, desktop or handheld devices.

I know the stated requirements is complex but if once achieved, it will be amazing!

3

Check out SwingUI and Flutter where layout constraints are part of the code and not a resource.

1 Like
4

Something like QML but via rust macro's that can link to rust code would be nice, though that does lose out on the nice abilities of being able to redefine the QML's post distribution. The whole QML language has fantastic defining and binding abilities.

1 Like
5

Yew by @DenisKolodin & @jstarry works fine with Electron or webview. We’ve prototyped both for the Amethyst editor.

OrbTk by @FloVanGH recently demonstrated web & native support.

Druid by @raphlinus has a Proof of Concept in the works for web support.

6

I'd say that in general, the biggest focus should be on the layout engine. It doesn't help when you have fancy animated buttons when they aren't where you want them to be.

This is especially important for responsive design, if your UI should work from a 3.5" mobile phone up to a 50" screen.

My personal favorite is constraints-based layouts, where you define the location of all elements of a view in terms of a system of linear inequalities. It has the downside of being a bit harder to debug (especially when your system is overdetermined or underspecified), but it allows you to define locations as a human thinks of them. I think that Cassowary is a solver for this concept. There's even a Rust implementation of it.

6 Likes
7

One rust layout engine I've ran across a bit ago was:

8

I always wish there's a nice XAML library written in rust, and supporting the WPF vocabulary.

4 Likes
9

This is similar to what I was thinking. I've been using React recently and its nice as a developer but I find it much more difficult to build tooling when it has to generate code to/from a programming language.

On the other hand the XAML language can be verbose.

I was thinking of a markup language sort of like a XAML but:

  • Only supporting 1-way bindings to reduce some confusion
  • Instead of using DataContext, passing parameters would be more similar to passing props in React
  • Unlike XAML you could specify entirely new components in the markup without any code behind. This would essentially get rid of the various sorts of templates provided in XAML which are confusing and don't need to exist.
  • Styles would work more similar to XAML styles than CSS. Aspects of an element that could be styled would be exposed as properties on an element. Styles then would simply provide a reusable way to specify the values of properties.

I was playing around a little bit and came up with the following XML blurb. I'd love to hear people's thoughts on it:

<Component>
    <!-- Components would describe the properties they need using JSON for an MVP. With a small type system. Rust code instantiating a component would be required to pass in these props. The inherit markup extension would allow a component to inherit properties from the first element inside of it. If the same property is specified twice the later version overrides the first.-->
    <Component.props>{
        {Inherit ChildProps}
        "clear": "(ClickArgs) => void",
        "divide": "(ClickArgs) => void",
        "multiply": "(ClickArgs) => void",
        "add": "(ClickArgs) => void",
        "subtract": "(ClickArgs) => void",
        "dot": "(ClickArgs) => void",
        "zero": "(ClickArgs) => void",
        "one": "(ClickArgs) => void",
        "two": "(ClickArgs) => void",
        "three": "(ClickArgs) => void",
        "four": "(ClickArgs) => void",
        "five": "(ClickArgs) => void",
        "six": "(ClickArgs) => void",
        "seven": "(ClickArgs) => void",
        "eight": "(ClickArgs) => void",
        "nine": "(ClickArgs) => void"
    }</Component.props>

    <!-- Components could describe a list of resources which are like local variables that can be used throughout the component. The would be referenced with the syntax {{Resource name}} where name is the name specified in the name property of the resource.-->
    <Component.resources>
        <!-- Styles would allow you to re-usably set a list of properties on a component. Unlike CSS these could be any sort of property on the component. Style properties like margins would just be regular properties. Styles could include other styles the same way that properties could include other properties. Just like props, styles could use an inherit markup extension. if the same setter were used twice the later version overrides the first. -->
        <Style name="BottomMargin">{
            "margin_bottom": "5px",
        }</Style>

        <Component name="MarginButton">
            <Component.props>{
                "children": "string",
                "click": "(ClickArgs) => void"
            }</Component.props>

            <!-- Inside of a string markup extensions would be prefixed with "$". to make using { easier. A markup extension with no identifier is a binding by default. -->
            <Button margin_right="5px" click="{click}">${children}</Button>
        </Component>
    </Component.resources>

    <Flex orientation="Vertical">
        <!-- When a style is used it could be type-checked against the properties on an element. This would allow the same style to be applied to elements of different types. The {Resource} extension would allow you to reference an object in scope. -->
        <TextInput
            style="{Resource BottomMargin}"
            width="Stretch"
        >${calculation}</TextInput>

        <Flex style="{Resource BottomMargin}">
            <!-- New components defined in resources could be used directly in the markup if they are in scope. -->
            <MarginButton click="{clear}">C</MarginButton>
            <Button click="{divide}">/</Button>
        </Flex>

        <Flex style="{Resource BottomMargin}">
            <MarginButton click="{seven}">7</MarginButton>
            <MarginButton click="{eight}">8</MarginButton>
            <MarginButton click="{nine}">9</MarginButton>
            <Button click="{plus}">*</Button>
        </Flex>

        <Flex style="{Resource BottomMargin}">
            <MarginButton click="{four}">4</MarginButton>
            <MarginButton click="{five}">5</MarginButton>
            <MarginButton click="{six}">6</MarginButton>
            <Button click="{minus}">-</Button>
        </Flex>

        <Flex style="{Resource BottomMargin}">
            <MarginButton click="{one}">1</MarginButton>
            <MarginButton click="{two}">2</MarginButton>
            <MarginButton click="{three}">3</MarginButton>
            <Button click="{plus}">+</Button>
        </Flex>

        <Flex>
            <MarginButton click="{zero}">0</MarginButton>
            <Button click="{dot}">.</Button>
        </Flex>
    </Flex>
</Component>
1 Like
10

I've used separate layout configuration files and (even worse) layout editors and it always comes back to the practicality to set the layout constraints directly in code. That's the direction of Flutter and SwiftUI and it is good.

1 Like
11

Perhaps what is best then is a system which can support either markup or UI written directly in code. Your experience differs from my own. We built in-house tooling for our designers. When they became productive in the markup the up-front cost in dev time quickly paid for itself.

I suspect developers would prefer to work directly in code, hence the prevalence of frameworks like flutter. On the other hand design tools are more familiar to the less technical crowd. React is nice because designers have some familiarity with HTML so if you stick to certain parts of JavaScript, less technical people can still be productive writing the markup. As mentioned before though, it can make writing tools challenging.

I feel strongly that markup is a must eventually though. Especially coupled with the learning curve of Rust, I think it would be very difficult for less technical front end people to become productive writing UI in code alone.

2 Likes
12

I've been in so many scenarios where management thinks that the designer should be able to do most of the design and layout in separate tools and then the code will work by itself. Worst case project was a fully working set-top box UI framework that was rewritten in Flash just so designers could supposedly tinker with the UI as much as they wanted (basically this redesign was so full of bugs compared with original stable C/C++ system so that the whole project was scrapped).

The best scenarios I'e been involved is when the designer and the SW engineer iterates the design via code itself. We could prototype and go up to full production code within days, and the code was efficient.

PS: There's a reason Apple ditched Interface builder in SwiftUI.

1 Like
13

Well... All Markup languages are painfully verbose and before compression I found this bad from a bandwidth perspective too.

Certainly you would do this. RDF (or eek! the semantic web) are like this too. However IDE's offer you a property sheet which helps a lot.

Static vs dynamic (too) is a huge issue but the code hooks must be there. It should be rational and theory based. It should not be confusing anymore than your sample is. That's a lot of additional code and design work of course.

Do you build into the language though? It sounds like a crate with UI types but I have a user's view and the compiler isn't my headache to address.

I would look at building in internationalization. As a STD feature at least given it might bloat the core. I mention this becase it applies equally to this issue. It is a primary requirement.

I am pondering how that would be done for the language itself, at the precompile stage. IE German Rust perhaps. Just like style formatting options. Ultimately you need standard formatted (US English) Rust for Git and the compiler both. This is difficult at a global scale.

That said, imagine the impact on adoption.

14

My favorite UI language so far has been QML. This works great for very graphical layouts, but I’ve never used it for a native layout. I suspect it works just as well though.

QML does have an intermediate layer - a scene graph is constructed from the code, which is then handed off to graphics card to render. This allows for it to optimize for things like minimizing shader switches. That being said, this knowledge is ~ 7 years out of date, even though I last used it a couple years ago.

I’ve also worked with UI in C# (early Unity stuff) and C++ (Qt Widgets, JUCE).

To add to the list of current UI efforts, there’s also moxie (moxie/README.md at main · anp/moxie · GitHub)

1 Like
15

In your opinion, shouldn't that have been more a prototyping strategy? I certainly see your point.

16

It really depends upon what you are doing but maybe most of the time. Dunno.

Developers I knew wanted minimum verbosity and readable code. It being less (dev) technical might matter perhaps when domain experts code but not generally.

Attrubutes and macros (or traits) helped this. Generics being the best. Others would say Lambda and closures.

The issue here seems to be the UI is a part of the code from the beggining and reflects design. I am a UI guy but that is something you don't want to be confused about. As demonstrated above the UI team was a big problem. It is an Agile methodology risk too.

The thing is I want not just better standard UI components but markup is:

  1. Declarative. And while I use tools with type inference I would prefer strongly typed. For example TypeScript was a good idea. Rust avoids or discourages ambiguous returns or exceptions and the dreaded null for example. It is explicit and readable.

  2. It is well suited to enforce code hooks via design patterns as well as for API and semantic definitions.

  3. If you don't need to build your output and controls dynamically you certainly avoid the run time cost. Performance counts.

The bad:
You 100% have to have an exact markup attribute to code (type or struct) property correspondence.

And early on the markup didn't fully integrate with the code behind. I found C# to be very flexible and sensible whatever you might think of DotNet.

I think what's relevant to the Rust community is you will start locking yourself into a paradigm no matter what you do. You must. Which should be standards based ideally. I think how abstract it ends up will create many issues to resolve.

17

QML Applications:

QML refence:

18

While I agree that it's nice, I don't think that it combines that well with the basic concepts of Rust (type safety, compile-time checking of as many potential sources of programming mistakes as possible, etc). Note how Qt is in C++, but in QML they use JavaScript instead, because it just doesn't fit C++ either.

1 Like
19

I'd really break that down into two questions:

  1. Which Rust concepts should be carried over into the UI language?

For instance, when it gets to GUI code, being explicit about things like incrementing and decrementing a shared pointer count is often cycles wasted on something that is not that important. Lifetimes and generics are also more debatable, since lifetimes can be abstracted away by some kind of GC (even just RC) and generics can in many cases use polymorphism instead. All of this has runtime cost, but on the scale of high-level GUI elements, it's not that big of a deal.

Things like null protection, optionals, match statements, and in general runtime logic -> compile-time flow control seem way more useful to me, as does type safety and traits.

I would also point out that you would not want to carry Rust compile times over to the UI. Ultimately, when you're iterating, you may be tweaking the way something looks - adjusting positioning or somesuch. So something has to get dropped regardless. Being able to immediately tweak and launch a QML application was amazingly freeing.

  1. Could QML be type-safe (or have the characteristics above)?

I think you could do this. Yeah, it's based on javascript, but last I read it used its own interpreter (V4 IIRC) in order to improve the C++ <-> QML binding speed. It can still execute normal javascript though, at least to some extent, at least as of a couple years ago.

But I think you could upgrade it to typescript. This would introduce compile-time type-safety. For the most part, when you're writing little QML bindings, it's not even going to be obvious; you'd just get a compiler error rather than a runtime error if there was a mismatch. The code itself would look identical. This would break compilation for existing programs if they introduced it with C++, but for Rust, backwards compatibility is not a problem.

I could also see something that looks similar, but is somewhat more Rusty. However, I'd be wary of shooting yourself in the foot with something like this. TypeScript or JavaScript allows for the direct usage of an enormous amount of libraries. Something different will create a niche language that has a catch-22 issue. I think TypeScript would be a happy medium between full Rust semantics and Wild Wild Web javscript.

Also, coding styles matter here. Some people dump everything into a QVariantMap and hand that off to QML. I think that's a bad idea. I much prefer to hand strongly-typed objects off to QML, because it's an interface boundary and it makes it MUCH easier to read.

One other interesting alternative I've never heard suggested would be a declarative language that uses Go rather than Javascript for the bindings. This would give you type-checking and fast compile times while still giving you pretty good performance. And Go is generally pretty cross-platform too. In QML, this would replace JS/C++ with Go/Rust.

1 Like
20

My answer would be none because there shouldn't be a separate UI Language. The more I ponder it and work with Rust, the more I'm convinced that just using Rust directly with a fluent API is the right answer.

3 Likes