Modules: What am I doing wrong?

Once again I find myself fighting for an hour to get 4 source files in the same src directory to build.

I've read the book, seen the movie, read questions and responses here. I conclude there is something deep going on that I just don't understand yet.

I have 4 source files: main.rs, tatami.rs, primes.rs and constants.rs.

main.rs uses tatami.rs and primes.rs.

tatami.rs uses primes.rs

primes.rs uses nothing.

I'll come to constants.rs later.

After a prolonged discussion with the compiler I end up with a build by having this at the top of main.rs:

mod primes;
use primes::Primes;
mod tatami;
use tatami::Tatami;  

But to use primes from tatami I have to have this at the top of tatami.rs:

use crate::primes::Primes;

Why is this different ? What am I missing? Am I just trying to do things the wrong way?

Along the way, compiler error messages kept suggesting I introduce directories for tatami and primes and name all the sources "mod.rs". What is that about? I really don't want all my editor tabs to show the same filename everywhere.

Meanwhile my constants.rs contains only:

pub const PNUM: usize = 1300;
pub const FNUM: usize = 10;
pub const SMAX: i32 = 100_000_000;

So far I have failed get the compiler to tell me any way to use those constants from all my other files (I have duplicated them locally to get the thing to build!)

What to do?

I'm going to assume you're following the normal convention for laying out a project. If you used cargo new my-project this is what you get:

  • my-project/
    • src/
      • main.rs
      • primes.rs
      • tatami.rs
      • constants.rs
    • Cargo.toml

The contents of main.rs is the entrypoint to your program, and as such it needs to let the compiler know which modules it contains.

// src/main.rs

mod primes;
mod tatami;
mod constants;

It looks like you also want to re-export primes::Primes so they're accessible from the top level (i.e. consumers of your crate would write my_project::Primes instead of my_project::primes::Primes), so adjust main.rs to publicly use Primes and Tatami.

// src/main.rs

mod primes;
mod tatami;
mod constants;

pub use primes::Primes;
pub use tatami::Tatami;

Now in tatami.rs we want to import Primes. There are actually two valid import paths you can use, because tatami.rs and primes.rs are in the same crate you can refer to Primes directly using crate::primes::Primes. Because we've publicly re-exported Primes at the top level, the second import path you can use is crate::Primes (this is what a consumer of your crate would use).

// src/tatami.rs

use crate::Primes;

// everything else

The reason it compiled with use crate::primes::Primes is because that's an import path that tatami.rs can use to reference Primes.

That's because there are actually two ways to declare modules, depending on how you want to lay your files out on disk. The first way is to write code in tatami.rs like you've done, but you can also use directories with a mod.rs to create nested modules. A feature that's really useful feature for larger crates.

For example, imagine a directory structure like this:

  • my-project/
    • src/
      • nested/
        • primes.rs
        • tatami.rs
      • main.rs

If you tried to import my-project::nested what would be accessible? The compiler can't naively concatenate primes.rs and tatami.rs because you could end up with duplicate names or seemingly magical behaviour where you tatami.rs can use an item it hasn't directly imported because it was actually imported in primes.rs.

A mod.rs file is used to resolve this. Your mod.rs declares exactly how the my_project::nested can be used and which files are included under the my_project::nested module (e.g. mod tatami in src/nested/mod.rs would give you access to my_project::nested::tatami). It's a special file that Rust treats as a way to tell the compiler how a directory maps to a namespace.

A good analogy is with HTML documents. You know how you can usually type in http://google.com/whatever/index.html and it'll also be accessible using http://google.com/whatever/? Well think of mod.rs as the Rust version of index.html.

Does that make things any clearer? Or does it just confuse things even more?

I'm sorry working with modules has been so painful/unintuitive! I originally came from the Python world which uses a very similar way for structuring packages (mod.rs is the equivalent of __init__.py), but how easy modules are to pick up depends very much upon what languages you've used in the past and the unconscious assumptions you make about how the module system works.

2 Likes

Note that naming a file mod.rs isn't compulsory to create submodules. The module file could be src/nested.rs instead of src/nested/mod.rs. The submodules of nested would still go in src/nested/. The mod.rs option complies with what was expected in the 2015 edition, while the nested.rs option avoids moving and renaming the file when adding submodules (so it is the approach described in the book).

2 Likes

Thanks for the detailed explanation Michael-F-Bryan. jameseb7 too. That all starts to make sense and works like a treat. I think I'm going to stick it on the wall here.

Can we replace the chapter in the book with that post, it would save a lot of trouble :slight_smile:

No wonder I was confused. In a former life when source referred to other source, in various languages, the reference was a directory path with a fie name on the end. Perhaps with the help of some external path prefix. But when I see use "crate::primes::Primes" I'm looking at a crate name, a file name and an object name all mixed up together. And the file name ends up being a module name.

Secondly it seems that in order to get to use something in a file other than main.rs I need to add a reference to that thing in main.rs. For example, in my primes.rs I have:

use crate::constants::{FNUM, PNUM, SMAX};

Which produces the error:

error[E0432]: unresolved import `crate::constants`
 --> src/primes.rs:5:12
  |
5 | use crate::constants::{FNUM, PNUM, SMAX};
  |            ^^^^^^^^^ maybe a missing crate `constants`?

Which does not help at all. How can it be missing crate 'constants' when I have said 'crate::constants::...' right there!

I have to add:

mod constants;

to my main.rs to fix an error in primes.rs. Which, unless I'm a dumb ass, is not entirely obvious.

Anyway, thanks again.

1 Like

Well... It's not wrong... But the hint isn't the greatest either.

It's complaining because rustc is looking for a constants item inside the root namespace (which we refer to using crate) and couldn't find anything. In this case the first bit, "unresolved import crate::constants", is probably more useful than the "maybe a missing crate constants?" hint at telling you what's going wrong.

I think this is a by-product of the fact that if you explicitly declare that your crate uses another by putting extern crate some_library at the top of main.rs. When that happens, it'll add a some_library item to the crate namespace.

Can you think of a better hint as to how to fix the problem? How about something like this?

5 | use crate::constants::{FNUM, PNUM, SMAX};
  |            ^^^^^^^^^ maybe a missing crate or module definition for `constants`?

Hmm, not sure if that error message change would have helped me much.

The confusion is that a problem in one file, 'primes' or 'tatami' has to be fixed by a change to a different file, 'main'.

This is especially odd at first sight as 'main' does not use those constants and has no need to know or care about 'constants.rs' otherwise. It's just weird.

I think what you were (are?) missing is that the module hierarchy for a crate is explicitly defined, starting at the crate root. (which is typically main.rs or lib.rs)

You can think of crate:: like / in a unix filesystem path -- crate::constants refers to a mod constants in main.rs. Rust, for better or worse, does not automatically create sub-modules based on the presence of files in the filesystem. It may also be that the constants module doesn't even refer to a file at all, and is instead declared inline:

mod constants {
    pub const PNUM: usize = 1300;
    pub const FNUM: usize = 10;
    pub const SMAX: i32 = 100_000_000;
}

Does that help?

This seems to be a recurring source of confusion... I wonder if it could be helped by smarter diagnostics? I think the confusing part is that module paths usually map to directory and file names/paths in the filesystem, which can easily lead people to believe it should "just work" -- often people seem to be missing the fact that modules must be declared explicitly.

Except it sort of, kind of, does.

I have a file called 'tatami.rs'. There is no module called 'tatami' anywhere. I tell main.rs that there is a module called 'tatami' out there with 'mod tatami;' and boom, as if by magic the content of my file 'tatami.rs' is now a module called 'tatami'.

Just to confuse myself further I wrapped the code in tatami.rs up as a module: 'pub mod tatami {...}'

So now in main.rs I need:

mod tatami;
use tatami::tatami::Tatami;

And now my 'use' statement has a meaning like:

use <file>::<module>::<something>

Where as before it was:

use <file>::<something>

Where magically became a module !

Now I'm not sure why I even need the 'mod tatami;'. That 'use' clause tells exactly what file to look at and in there is the 'mod tatami {...}' After all, I don't need any 'mod' statement when 'use'ing things from external crates.

Forgive my confusion bu this is all confusing.

Yep. Thanks. I think I'm good to go.

That's exactly what we mean by "Rust does not automatically create modules." For every module in your crate, you must create it using the mod keyword.

This code:

mod tatami;

Expands to this when you compile your program:

mod tatami {
   // contents of tatami.rs
}

There's no different from the compiler's point of view between these two ways of creating a submodule in the current module.

OK.

Why do I need 'mod tatami;' in 'main.rs' when I have:

mod tatami {
   // implementation of tatami
}

in the file 'tatami.rs' which it knows about because of the 'use tatami::tatami::Tatami;' statement?

I don't mind, it just seem redundant.

That's because if your files are laid out like so:

//main.rs
mod tatami;
//tatami.rs
mod tatami {
    //implementation
}

Then the compiler will see it as something like this:

//main.rs
mod tatami { //Originally `mod tatami;`
    //tatami.rs
    mod tatami {
        //implementation
    }
}

So, if you have another mod tatami in tatami.rs you'll be creating another module.

A thing to note which seems to confuse some people; rust modules are not like C++ headers (Where everything is essentially concatenated into a huge file) or C# source files (Where each file must declare its scope with a namespace statement), in rust, every file is a separate module already, and to use it you must declare so with a mod name; statement which signals to the compiler to look for a file named name using the module-system rules.

1 Like

You should not have mod tatami inside of tatami.rs (unless for some reason you want to create a second tatami module inside of the first tatami module).

Here's how I like to think about it, module ownership (I'll use this fiile structure for the rest of this post)

src/
|- main.rs
|- tatami.rs
|- constants.rs
|- primes.rs
|- tatami/
    |- helper.rs
    |- tatami
        |- foo.rs

If we want to describe the ownership of modules: main.rs and lib.rs owns all of the modules in the same directory. In this case, tatami.rs, constants.rs, and primes.rs.

Every other module owns the modules in the directory of the same name in the same directory. In this case tatami.rs owns helper.rs, because helper.rs is in the directory tatami. But it does not own foo.rs because that is in the tatami directory that is not in the same directory as tatami.rs.

So, how to we declare this ownership scheme to Rust? Because Rust can't figure this out on it's own.

We use the mod keyword! Just list every module that the current module owns in their file.

// main.rs

mod tatami;
mod constants;
mod primes;

// tatami.rs

mod helper;

Ok, now how to use the stuff from other modules?

There are a few relevant keywords

use
crate
super

The use keyword will import things from other modules/crates. By default it will look into submodules.

So in tatami.rs if we did

use helper::helper_func;

It would look in the file src/tatami/helper.rs for the item helper_func.

The crate keyword tells the mod keyword to look in the crate root instead of submodules.

So in tatami.rs if we did

use crate::constants::*;

It would import everything from the src/constants.rs

Finally, the superkeyword tells the mod keyword to look in the module that owns the current module.

For example, in helper.rs if we did

use super::tatami_func;

It would look in the file src/tatami.rs for the item tatami_func


So when you did

// tatami.rs

mod tatami {
    // ...
}

You are telling that the src/tatami.rs module has a sub-module which is also named tatami. Normally that would be put in src/tatami/tatami.rs

OK. Sounds reasonable.

But why the need for 'mod tatami;' in main.rs?

I don't recall using 'mod anything' when using external crates. I would have though that they are full of modules and that I'd have been using some already.

When Cargo compiles your program, it passes just one file to the compiler. It runs a command like this:

rustc main.rs  # (plus some other flags based on your Cargo.toml)

The Rust compiler does not search through the whole filesystem looking for source files. It just reads the file that you pass it (main.rs). When it encounters a mod item, that is its signal to include an additional file.

The mod keyword appears not where you use a module but where you create a module. Therefore, each module needs only one mod keyword, which always appears in its parent module. For a module in an external crate, the mod keyword is somewhere in that crate's source code. For example, your program might use a type like std::io::BufWriter which is in the io submodule of the std crate. Therefore, this module appears at the top level of the std crate:

https://doc.rust-lang.org/1.39.0/src/std/lib.rs.html#461

In your own crate you can refer to this type by its absolute path std::io::BufWriter. Or you can write use std::io::BufWriter; to import its name into the current module, and then refer to it as just BufWriter.

Each function, module, type, etc. is defined in just one place. Just like fn foo defines a function, mod foo defines a module. But these functions, modules, types, etc., can be used in many places, without repeating their definitions.

2 Likes

Thanks everybody. Just now this is starting to make sense. If we discuss it further it might just get confusing again.

I think the best thing now is if I go back and read the book again, in the light of this discussion that will probably make sense too :slight_smile:

1 Like

You may also want to check out the modules page from the Rust Reference. The Rust Reference is more of a technical document explaining concepts formally, so it may be able to word things in a way which just click.

Also, if you come across bits of documentation in the book or the reference that you feel are confusing or ambiguous, please create an issue against the relevant repository so we can improve things for the next person! :slightly_smiling_face: