I'm new but I believe there are things wrong

I don’t remember if I did read the book from doc.rust-lang.org/book or from doc.rust-lang.org/stable/book. However there are important things that they do not say, took me many days to try to figure out and I’m still not certain I got them correctly.

One simple example is how to be sure that the code I write (I’m currently using RustRover trial and the trial will end in 5 days, far less than what will make sense for me) will work on this laptop and that I can access the correct libraries.

Technically I’m using a MacBook Pro 14-inch with Apple M5 (obviously now with Tahoe 26.3) and it was hard to figure out to add libc = "1.0.0-alpha.3" after [dependencies] in my project Cargo.toml file. As a fact I’m also not certain it’s fully correct. With my C code (using Xcode) is easy to call lstat in code I write in a .c file that did #include <sys/stat.h> which declares int lstat(const char *, struct stat *) __DARWIN_INODE64(lstat);

However the biggest problem I’m having is with RadDir from fs::read_dir (also was hard to figure out that use std::fs; is not enough and try something adding use std::os::unix::fs::DirEntryExt;) because what is most important for me simply doesn't happen (or happens in the wrong way).

Limited example of my C code (obviously the dots stand for parts of the source I’m not copying as is not useful here):

....
#include <dirent.h>
....
DirEntries* readDir(const char* dirPath __attribute__((nonnull))) {
  DIR *dir = opendir(dirPath);
  ....
  DirEntries* dep = calloc(1, sizeof(DirEntry*));
  ....
  struct dirent *ent = readdir(dir);
  while (ent != NULL) {
    // add a new entry to *dep
    .... = newDirEntry(ent->d_name, ent->d_ino, ent->d_type);
    ....
    ent = readdir(dir);
  }
  closedir(dir);
  ....
  return dip;

}

I’m not getting all elements of struct dirent but it should be clear that I’m getting, and later using, d_type as a value (technically it’s a uint8_t with values from 0 to 14) and getting it as an unsigned integer value in rust … didn’t even figured out how to. It’s also clear that I’m getting all the childs (not sorted) of the directory and this is also impossible in rust.

Here is a list of 2 things (likely limited given I’m new and didn’t check everything) that I consider wrong in rust:

  • A readable directory has at least two entries, the first is named “.” and refer to the same directory and the second is named “..” and refer to the parent. This is all not always true. It’s possible that they are not named like that, possible that they do not refer to that, possible that a directory is readable and has less than two entries. What is extremely common is not necessary always true.
  • The d_type value is useless as a value because it identify alternatives and is different depending on the system therefore is much better to give access to checks of which alternative is there. This is also quite stupid. It is a value, just 4 bits but it is an unsigned integer. Give me an integer (even signed if you want as long as it’s >= 0 and, currently in my system, <= 14). I will take care of it. Just like I will take care of st_mode in struct stat (please leave it as uint16_t so it’s easy to shift it right by 12).

Basically making things like managing pointers better is a good idea but making impossible to access what really would work with no troubles like getting d_type as an integer value and getting info to first elements of directories (most of the times “.” and “..”) is definitely one of the reasons it may be currently useless to study rust. And I haven’t yet started to verify a path is managed correctly in all cases (including . and .. elements anywhere in the path which is then passed to a C function as a char*) and also not started to verify things like listxattr and getxattr (etc etc to setxattr …)

If I simply lost some info and what I wrote is wrong I hope I’ll find it in some reply.

1 Like

One thing of note is that the standard library APIs are aiming to be platform independent. If you want (safe) interfaces around libc that are more platform specific, you could try using crates such as rustix[1] or nix. The type in place of dirent there would be:

(obtained from the rustix::fs::Dir iterator)

edit: or maybe this one(?)

(obtained from the rustix::fs::RawDir iterator)


and

(obtained from iterating over a nix::dir::Dir object)

respectively.


  1. actually still somewhat more platform-independent, perhaps just closer in flavor AFAICT

    Edit: Nevermind, re-reading the top-level docs of the crate, it seems that the majority of the API – including the fs module in question – is in fact platform-specific to Unix-family platforms ↩︎

5 Likes

I believe that most of the problem comes down to Rust needing to work on Unix, Windows, and more. The standard library’s DirEntry struct only exposes information which is (mostly) cross-platform. E.g., Windows doesn’t have inode numbers (even though it has something similar), thus the need for std::os::unix::fs::DirEntryExt if you want the inode number.

Conversely, I find it incredibly convenient that std::fs usually does what I want, instead of forcing me to always think about platform-specific details. For instance, syncing the contents of a file via std::fs will use the appropriate fcntl(F_FULLSYNC) on MacOS instead of providing a useless literal translation to fsync. And when I want to iterate over the children of a directory, chances are I’m not thinking about . and ...

It’s rare that I need to depend on rustix or libc, but when I do, that means I really need to think about each operating system.

4 Likes

I think it's important to remember that Rust is not C, nor is it trying to be. It's likely that you've heard people call Rust "the C-killer" or replacement, but that's not its goal.

With that in mind, you absolutely will find a lot of things that differ between the two languages. There are going to be workflows and patterns that you're used to from C, that don't translate cleanly to Rust. The same is true of C#, or JavaScript, or Nim. You have to learn to do things differently, and calling any of those languages "useless" because they're not C is unhelpful to everyone, including yourself.

7 Likes

That standard library APIs are aiming to be platform independent is not a problem. That there is an incredible number of crates, so many that finding something that may be useful to access some specific platform standard is a problem.

For example what is standard in Unix but does not exist in Windows should be fully available simply by adding one line somewhere (the project Cargo.toml or the current .rs file) and also the opposite for what is standard in Windows but does not exist in Unix. Obviously the project would then either become itself specific or have multiple .rs files separating alternative system code or an .rs file will have part of the code for one system and another part for another system.

This should also be easy to find in the rust book.

Anyway thank you for suggesting rustix and nix that I’ll go to check.

Rust is not C. I agree. It’s actually even far more than a programming language. But there is a programming language that is part of Rust and the fact that there are a lot of things that differ between the two languages is not a problem.

However I’m not saying that Rust is useless as a language. I’m saying that it’s useless for me and for doing something easy and standard in the operating system that I’m using (actually even in other operating systems because Tahoe is definitely not the only UNIX). The reason I showed C code is multiple and includes the fact that Rust actually uses the C compiler and library that I have on my laptop and also calls the C functions I showed I call in my C code.

Why? Some people really want to write code that only works on one specific platform… and that's possible with Rust — but why that should be easy?

I rather like the fact that most crates that are developed solely on one platform “just work”, when you use them on a different one. That's pretty important adavantage over C where it's PITA, most of the time, except when someone already done porting work.

But you have shown nonstandard C. That's the issue here. Try to find all these things that you demand to have on cppreference or in the relevant standard text — and you'll find nothing. Try to show us how one can standard filesystem library — and you'll find that it's not really possible.

You have to find and use entirely different set of functions… you couldn't “just add one line somewhere”!

Why should Rust do something that C/C++ don't bother to offer? Heck, C doesn't even bother to offer any interface that allows one list files in a directory so it, essentially, sends you to look for a nonstandard way of doing that without offering anything at all!

And C++ acts more-or-less like Rust…

1 Like

Yep. I know the problem comes from being possible to work on multiple OS. However the definition of standard is more complicated. Standard in the sense of identical in all OS is different than standard in a specific OS. And managing this is what Rust is still trying to do. Since I’m a beginner I’ll use terms which are common in C but it should be easy to understand what it means in the Rust universe.

Rust certainly needs what is standard everywhere and it also changes and grows over time. Just like C, the C standard and the C standard library (K&R C … ISO, ANSI, POSIX … C89 {C90, C94/C95}, C99, C11, C17, C23 …). It also needs what is not standard in that sense but is standard in a specific OS. Likely it also has what can be considered not standard but somehow pretty standard, like GNU. However some of these things exist and likely some still do not. Unfortunately it’s still pretty hard to figure out what exists and what doesn’t because what exists either it’s not fully accepted as being standard or it’s mixed with a big group of crates most of which definitively have nothing to do with the standard API of any OS.

Conversely, I find it obvious that you consider incredibly convenient that std::fs usually does what you want, instead of forcing you to always think about platform-specific details. However I find it quite negative that it does that without making possible what is part of the system. I believe that it should be easy and standard to get all the elements of a directory that there are in the system the code is running. That means that “.” and “..” should be there if it’s running in a unix OS and accessing a directory that has them even if it the code was developed in Windows: either the executable is basically an .exe file and cannot run in most unix systems or it gets them. However it makes sense that it is possible to parse a directory in multiple ways: full and not sorted, full but sorted, not full by ignoring what starts with a dot, including what starts with a dot except “.” and “..”, etc etc etc. Basically implementing a good part of ls in command line. Just read man ls in a linux based OS and in Apple macOS and you will find what is extremely common and what is different.

The fact that depending on rustix (I didn’t even know it existed and I’ll study it) or libc means you need to think about each operating system is definitively correct. What I miss is a way to cargo --unix new a project. Or cargo --only_local new a project that would automatically work on the OS you are using (could be Windows or an Unix or whatever) and also automatically fill all dependencies in the Cargo.toml it creates so you directly get all you need.

Generally speaking any project may be specific (one OS) or general (multiple OS), may start being planned as specific and later become general or start being planned general but then become just specific. Actually may even start as generic (basically still no idea which OS and even not a serious plan on what the project is). I think that the cargo command (and the whole Rust project) should seriously consider these alternatives as something important to manage: stop considering every new project as generic and make it easy to start them as specific or general.

1 Like

The real question is why it should be harder than just specifying which platform when you cargo new the project.

That’s because they are developed by someone that does not know other platforms or is not really developing something that is specific for a platform.

I showed code that is fully valid and, as far as I know, it basically use what is in POSIX. It is not an ISO standard but it definitively is valid for a standard because POSIX is a standard (at least for the meaning of it: STANDARD Definition & Meaning - Merriam-Webster ).

Some context you may not have is that the Rust project maintainers do not have the available capacity to:

  • maintain a full set of API bindings for each operating system supported by Rust, or
  • maintain a current and trustworthy list of third-party bindings. (An unofficial list of recommended libraries including Unix bindings may be found at https://blessed.rs/crates.)

So, in order for there to be some meaningful kind of “starting as specific”, you would need to define something achievable that “starting as specific” would do.

If you’d like to be able to easily create Cargo projects pre-configured for specific use cases, then maybe all you need is cargo-generate which is a tool for using project templates.

6 Likes

I just read about this batteries included initiative which might be what the OP is looking for.

Rust's standard library is not POSIX. It's Rust's own. It's a thin wrapper around OS APIs, so it's somewhat similar and has "escape hatches" like access to raw fd, but it's not meant to be used like C. It's not meant to cover the whole OS. It's the lowest common denominator for the basics (barely larger than C's libc, not as large as entire POSIX).

Rust tries to avoid having single-platform software (and doesn't have a concept of single-threaded programs either).

There's no selector for a preferred platform. Rust's library ecosystem tries to be portable too, and most libraries try to support all major platforms by default.

In C it's common to choose one platform, especially a side in POSIX vs Windows, because portability in C is tedious and annoying. You have time-wasting nonsense of MSVC, fragmented build systems, mess of inconsistent include headers, include paths to set, a complete chaos of dependencies if you don't stick to one OS with a package manager, and so on.

Rust doesn't have these problems. There's one compiler and one build system that works the same across all platforms. Dependencies are easy and portable. You don't need to touch include paths.

Projects that actually target a single platform are a minority. When they are platform-specific, it's usually due to targeting very unique Windows/Apple/Linux APIs, which aren't supported by Rust's minimal standard library, so a config option for the standard library wouldn't make a difference anyway.

I ended up maintaining a build tool for making Debian packages, but I'm a macOS user, so most of my development of a Linux-only tool is done on macOS. The tool can also run natively on Windows (packaging win32 exe, not just cross-compiling). Running it like that doesn't make any sense, but getting it to run on Windows was very easy, so I could do it just to laugh at the absurdity of Windows-only .deb packages :slight_smile:

10 Likes

I have 6 machines with different OSes and processors in my garage, to make sure that Rust has no any problems everywhere. It was a bit easier when I used Java, but JVM guys used C.

2 Likes

I'm all in favour of standards. Perhaps not always "official" standards like ISO and such but recognised and widely adopted standards like POSIX.

What has been discussed here is at least two standards. One being the standard definition of the C language and it's standard library The other being POSIX, a standard for operating system facilities and API's

These are two different standards. Even if they have been joined at the hip for a long time in for example Unix and Linux, later Windows. There is no POSIX in my Rust, especially for embedded systems.

I see no reason for Rust to incorporate POSIX into its own definition. The IEEE already maintains POSIX. I'm pretty sure all of POSIX is available in crates by now,

1 Like

Doesn't happen. That's why we play Windows games on all platforms, even on macOS or Linux they come with code that emulates Windows and thus hides the fact that it's not a native app from the end user.

Creating cross-platform code is not too hard, if you don't have cargo --only_local option, but becomes a problem if people can do that without thinking.

No, that's because writing platform-specific code takes special effort. And not just a trivial line added somewhere.

People are lazy, but most of them are not malicious. You give them shortcut — and they would use them, you make writing non cross-platform code hard — and they would only do that when they need that.

Yes but POSIX is not C. If and when someone would produce special version of Rust that only works with POSIX systems I would know how to avoid it — and that's the main thing that I need, really.

It absolutely is a standard, it's just not a standard for C language.

Note that none of language have done that. Not even C. C was born with POSIX API (well… subset of it), then it have become splintered (almost like BASIC that have hundreds of incompatible dialects) and then “standrd API” was created to make it possible to write cross-platform programs.

C# did something similar and it's extremely painful there, too.

Most modern languages simply do the right thing: make non crossplatform code possible but hard.

It's just the best way of doing things, experience have taught us that. It's the same story as with x86: it could be one of the most popular platforms but it's also a weirdo, similarly with C: it's one of the most popular languages but it's also a weirdo, while Rust does things that most other popular languages do, too.

2 Likes

I like your reply.

Most of what you say is something I already figure out. I’m not fully sure I already knew the link to blessed.rs but I’ll now try to have a local file (maybe even not fully local if I do it with Notes.app) with a list of links. Same logic about cargo-generate.

In general terms we could also connect “maintainers do not have the available capacity to” and “you would need to”. There are obviously many things that can be done even without being a maintainer (except that, if it becomes public, you should actually be the maintainer of that thing). Public, related to Rust, not managed by a Rust maintainer … there’s already a lot! For example W3Schools, JetBrains, GeeksForGeeks, How-To Geek, wasmCloud etc etc

Theoretically I can do almost anything. Practically my reality is that I first need to learn Rust (figured out it existed some years ago but really started understanding what it really is approximately 2 months ago and started really reading something about it approximately one month ago).

How fast is learning something new does not depends only on what it’s already known but also on you age: approximately 7 months from now I'll be 60 years old (definitively far slower than when I was 12 and self-learned 6502/6510 assembly in hexadecimal, also slower than at 16 and learned Pascal, Fortran, the basics of Lisp reading what McCarthy was thinking, again slower than a few years later reading Ritchie about C).

Not only I’m slow at learning. I’m also slow at thinking and comparing. Because my mind, when I’m learning something new, automatically compares it with what I already know (both what I know extremely well and what I simply know decently). Just try to compare anything new for you about Rust to Assembly, Basic, Pascal, Lisp, Fortran, Clipper, C, Smalltalk, Prolog, C++, Modula, Oberon, RPG, COBOL, Objective C, Ruby, Java, JavaScript, Scheme, Forth, Python, C#, Haskell, Racket, Swift, Go, Julia, Clojure. I just listed them (and likely not all) in the order in which I started to learn them. Ignored how much I did learn them and how much I used them (and also how much I actually remember them).

That’s why I could do anything as well as I can do almost nothing.

Ok. I now know that Zulip exists and will also need time to learn something about it.

Obviously … also the battery pack is something I should check (and, maybe, load :slightly_smiling_face:).

Really? (I’m not talking about threads). If it really want to avoid having anything which is single-platform it wants to avoid being used by anyone that wants to do something which really doesn't make sense in other platforms (at the moment, the future depends on how platforms change and how a platform copies what another platform has already done).

Rust really basic fully standard library is basically rust/library at main · rust-lang/rust · GitHub . When you actually install Rust you already get more. We may very well consider standard what you get whenever you just install Rust. And both rustup and cargo (among command line commands) could very well be organized to manage multiple possibilities.

Yep. If you consider standard the really minimal fully standard one. But a config option for rustup and/or cargo as command line commands still makes sense (and makes many cases easy given that would basically automatically just download and configure more, not less).

I’m not saying that making my code running on windows should not be done. I simply consider it totally useless.

Given you are a macOS user figure out something. Pretty easy to write C code (just using Xcode) and with the path of a directory read it, find a sub directory (get both d_name and d_ino) then use lstat to compare the actual st_ino to the assumed one. Also compare them to the st_ino from the subdirectory “.”

For example read “/” and find “private” and compare the d_ino to the st_ino you get for “/private” and the one you get from “/private/.”. In this case the will be all three identical. But this is not always the case.

An easy way to perform a simple check that things may change a lot is using Terminal.app (basically the command line). Use it an try these commands: ls -lfi / then ls -lfi /System/Volumes/Data

You will simply get the minimalistic info on how APFS works. Then figure out what a volume really is and which volumes are currently active (just try ls -lfi /Volumesthenls -lfi /System/Volumes/Data/Volumesthen ls -lfi /System/Volumes).

Try to visually compare what you get with ls -lfi to what you actually get as d_ino reading a directory and you’ll find differences (for sub directories what you get with ls -lfi is fare more likely the st_ino of the “.” of the subdirectory than the d_ino of the subdirectory entry in the parent).

Is it really a problem if I write code that processes multiple technical data of multiple files/directories from multiple volumes (may also say multiple partitions if you also consider thunderbolt or usb-c external storage) and consider it makes sense to use it only with macOS?

Is it really something Rust should try to avoid happening?

But that should make things easier, isn't it? Rust doesn't bring anything new to the table, except for affine type system, everything else is implemented in one language or another that you have listed[1].

Yes, of course.

Depends of you goals, ultimately.

So far we have the classic The patient says, “Doctor, it hurts when I do this.” The doctor says, “Then don't do that!” dialogue.

What kind of “business need” are you solving that you need such an obscure details that most users of the macOS don't even know exist? How would that map to Linux containers? To BSD jails? Do you even need all that complexity in your app? What kind of app is that?

Rust goal is pretty consistent on the levels of design: make “crazy things” possible, but hard… and that's really an appealing approach to me: it's not “full jail with escape into other language”[2], similar to C#, Java, or Haskell, but it's also not “you just have to hold it right” approach of C/C++. Lots of people find it a good balance, but, perhaps, in your case it's not a good fit, depending on your actual goals…


  1. Is Clipper even a language, though? It's in the dBase language handbook thus I would consider it a dBase language, or maybe an xBase↩︎

  2. Where everyone pretends language is fully “safe”, but it can be easily forced to made “bad thing” by bringing C module into the mix… ↩︎

1 Like

Exactly! Either Rust really wants to avoid being used or, simply, it’s still growing and there are cases where it’s still not fully ready or is almost ready but still makes them too complicated to manage.

As a newborn I still have to figure out which is the situation.