Splitting a module in multiple files

About 50% of that crossbeam file is comments, and the rest are functions/methods that are only a handful of lines each. This is just glue code. You can see that the channel implementations are split into a number of other modules, named flavors::<flavor name here>::Channel (As I also do). Yes, glue code can be verbose, especially when properly commented, but there is little reason to split these into modules when they aren't doing all that much on their own.

Looking through bytes, I would certain split this into more than a few modules. All of the Debug/From/PartialEq/etc. impls can be put into a separate module that so that it doesn't obscure the core. This is doubly important when dealing with unsafe. Inner, Bytes, and BytesMut may be in separate modules depending on how the previous refactor goes. That should take care of the vast majority of size problem. This refactor would also make it easier to reason about the safety of this module, because BytesMut wouldn't be able to access Bytes.

I don't think these examples are good ones for modules that can't be split up.

No, I think you're the one who missed my point. I understand that you want to split a module across multiple files. I also understand that you have some strange ideas regarding what a module *ought" to be (e.g. some sort of a dependency tree), which doesn't coincide with what modules are. You for some reason want to organize your code in two separate and unrelated ways, spitting the functions into modules and files in two different ways. My point was that I don't like this, because it makes it harder to find a given function, and that I like having just one way in which the code is organized.

1 Like

What I haven't understood in this discussion/proposal is how the multiple files that constitute a single module are supposed to be ordered and linked. How does the compiler and other tooling identify which files are involved, and in what order? Presumably this requires linking declarations either within each file or within some hierarchical parent file. If so, how in practice does that differ from the current Rust module structure? Is it just the lack of hierarchy? Could that be achieved in a simpler way by addition of a cargo [unwrap] feature to a one-level-deep hierarchy, to flatten multiple files into one? Would you require a file naming scheme that dictates the concatenation order, such as foo_01, … foo_13?

2 Likes

Hey, everybody. Did you know that it actually is possible to split a single module, in Rust, into multiple files? It's easy. Use the include! macro.

// file1.rs
include!("other.rs");
// other.rs
struct Whatever {
}

Sorry, @steveklabnik. You apparently forgot.

Edit: My mistake. It was mentioned. I just missed it.

7 Likes

This was noted at the very start:

1 Like

Yes, they said that that didn’t count.

(Also, I agree with them: it’s a bad idea and you shouldn’t do that.)

3 Likes

I'm sorry. I was thought that you were arguing that the way you split code into files should be different than the way you split it into files.

Sorry. Reading comprehension failure.

1 Like

I don't have much trouble with it. I do it like I'm doing it with Clojure, Put functions/structs with related depencies/functions together, and have one, or multiple if really big, to tie them together.

It seems like you're not asking the right question. Your question is not really about modules, but about working with non-trivial quantities of code and the resulting non-small files. This is not a language-level question but an IDE-level question, code-folding and easy navigation within a file largely solve your problem. I personally use and highly recommend a rather esoteric, but highly powerful editor called Emacs.

1 Like

I find code folding a very poor substitute

@OvermindDL1
@motoras

The Dart language provides an easy way to split a library into multiple files using part and part of keywords.

New rust user here. I'm just learning, and am coming late to this thread. Like @motoras, I also am "surprised" at the requirement that a rust module fit in a single source file.

Let's take Java as a counter-example.

My mental model roughly equates a rust module with a java package. Multiple rust modules can be bundled into a crate, and multiple java packages can be bundled into a jar. In both languages many different constructs (structures, classes, constants, etc.) can be defined in the scope of that rust module / java package.

In general, java chose to map a package to a file system directory. I can have many .java files in the directory, and they together form that package's compilation unit. I can put my java code in a single file, and when it gets too large for my own taste or organization best practices, I can split it into separate files. I could organize the files by class, or maybe split my documentation in a separate file, whatever. Subpackages become subdirectories, and themselves can comprise multiple .java files.

Let's take Go as a counter-example.

My mental model roughly equates a rust module with a go package. I can likewise split my package implementation across multiple .go files in the same directory. I can likewise bundle multiple go packages and subpackages into a go module for distribution and sharing akin to a crate.

Rust has chosen to map a module to a file system file, full stop. I have the choice of putting the code of module foo into the file foo.rs, or I can create subdirectory foo and put the source into foo/mod.rs. But it's a single source file. That is a choice made by the language, so with respect to @mrcnski, I believe it is a language-level question.

Rust gives me the ability to re-export identifiers so that they appear to "live" in a different module. Thus I can choose to break my implementation up into multiple submodules, and "re-assemble" them by re-export.

That being said, as a rust newbie coming from java and go that just feels... odd. As in, it feels odd to be constrained that way, when file system subdirectories could have been used as in other languages to let me as a developer choose to split my module code into different source files without having to spill over module boundaries.

This won't be a perfect analogy: I write a new language that requires any function definition to be in a single source file line. When I'm asked, "why don't you let people decide where to put their line breaks to organize their code?" I can't understand why they care about 4K line length in their source. If they don't like it, they can split their function into multiple subfunctions, one per line, and simply call them from the top function... problem solved! I know... it's not a perfect analogy. :wink:

Bottom line: This feels to me that a singe source file per rust module is an odd language design decision. However, it's the choice that was made; that's the way it works, and if I want to use the language I'll figure out the idiomatic ways to organize my code with the contstraints I'm given.

1 Like

@djk Java packages correspond to Rust crates.

Java jars correspond to Rust compiled binaries.

But these are extremely rough analogies, they don't really match perfectly. Rust is very different from Java (in many ways), your Java intuition will not apply. You are better off going in fresh, without any preconceived ideas.

Also, Rust's module system is similar to other languages like JavaScript or Python (which have "one module per file"). JavaScript or Python developers would find Java's modules to be very odd!

Different languages have different module systems, just because Java does things one way doesn't mean that Rust's choices are bad, Rust takes inspiration from many languages. You just need to keep a fresh perspective when learning a new language.

5 Likes

I take your point that it's a different language, and that it will take me a time writing real code to develop a sense of its particular idioms.

Thanks for the quick reply!

1 Like

To me, the defining feature of a rust module is that it creates a privacy barrier. After all, minor name resolution nits aside, that's the only difference between having things in the parent module versus putting them in a submodule with pub use child::*.

Java classes, too, create a privacy barrier. Hence why I agree with others here that Java classes—not packages—are the equivalent of rust modules.

4 Likes

Created a proposal here:

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.