let fmt = std::ffi::CString::new("hello.as_ptr() as *const _ == %s\n".as_bytes()).unwrap();
let hello = "Hello!";
/// SAFETY: fmt is a null-terminated string and requires 1 stringy variadic arg
/// that arg is hello, which is a pointer to a Rust string, which WILL CAUSE UB
unsafe { libc::printf(fmt.as_ptr() as *const _, hello.as_ptr() as *const _) };
Now, my mental model of str is that it looks something like this,
struct str {
len: usize,
start: *mut char,
}
which would suggest that the above would print this,
hello.as_ptr() as *const _ == \x{6}Hello!{overread nonsense}
but what it actually shows is this,
"Hello!".as_ptr() as *const _ == Hello!{overread nonsense}
Even for longer strings where '\x{N}' would be a printing character, it doesn't do anything. So what's happening under the hood? I looked through the source code and I didn't find any definition for str, at least in core/str/mod.rs
First, there is already the improper_ctypes, which is a rustc lint and will warn you against using &str etc. in FFI.
Second, you seem to expect that the length will be embedded in the string. It's not; hello.as_ptr() gives you a pointer to the string contents. Since it's not NUL-terminated, printf() does not know when to stop and therefore continues to print.
str is a dynamically sized type. It doesn't have a fixed size of its own. In exchange, references and pointers to str are double the usual size. That is, it's as if you have this struct, if it were legal:
struct &str {
len: usize,
start: *const u8,
}
Note that it's &str, not str.
Alternatively, you can also view str as being the same as [u8] but with extra invariants:
struct str([u8]);
In any case, given that &str stores the length and the pointer next to each other inline, text data (which the pointer points to) is nowhere near where the length is stored. The .as_ptr() method returns this start pointer.
Also, note that I wrote a *const u8, not a char. In rust, a char is 4 bytes large, as it's stored as UTF-32, where each unicode codepoint is exactly 4 bytes large. In contrast, a str is stored as UTF-8, where each unicode codepoint has variable size, from 1 byte (to represent ascii, which is a subset of unicode) to 4 bytes.
So you're saying that str is merely an undefined number of UTF-8 bytes and that &str is what contains the length. Would there be a way then that I could cause the fat pointer with the length to be given to printf?
Because you're not pasting a &str, you're passing a *const c_char which is entirely fine. Note that &str-as-ptr is also fine if the string ends with a NUL char.
the "internal representation" of str is just a bunch of bytes in utf8 encoding.
rust uses what's called "fat" pointers for dynamically sized types, so the additional information (the length of the str in bytes, in this case) is NOT stored inline with the data, but with the pointers (references and raw pointers have the exact same representation). that's also the reason the pointers are not FFI safe, because they are #[repr(Rust)] types and the exact representation is not guaranteed to be stable.
str is a primitive type built into the compiler, just like bool and i32, it is NOT a "struct" defined in the standard library. however, methods on the str type are defined in the standard library, such as str::len() and str::as_ptr(). this is a privillege of the standard library. user code cannot define inherent methods for primitive types.
the heap allocated String on the other hand, is a library type (it's a lang item, so not a "normal" library type either), defined in the alloc crate, which is really just a wrapper of Vec<u8>:
How do you mean "fine", though? Like clearly the *const c_char I gave isn't fine and it's quite untypical to be putting '\0' into your strings in Rust anyway. My lint basically detects adjusted_to_str.as_ptr() as *const _ in an extern "C" fn and assumes that that's usually wrong and that you should use CString instead.
I guess the question for me now is where (how?) is str defined. If I try a grep across the rust repo for '(struct|enum|union|type) str\b', I come up with nothing substantial aside from some stuff in gcc and llvm. Does Rust maintain its primitive symbols there? Or is it even that the symbols have ethereal definitions due to bootstrapping?
Let's start with slices. A slice is one of the two (?) basic unsized types (the other is "trait object"). Unsized types are types whose size is unknown at compile time. Roughly speaking, a slice is an array but with unknown length.
struct [T]([T; unknown at compile time]);
A pointer to [T] is a "fat pointer", a type that carries the length in addition to the address:
struct *const [T] {
address: usize,
len: usize,
}
A reference is a wrapper around a pointer:
struct &T(*const T);
so a reference to a slice &[T] is also a fat pointer with address and length:
struct &[T](*const [T]);
A str is a wrapper around [u8]:
struct str([u8]);
Since it contains an unsized type inside, str itself is an unsized type. A pointer to any struct containing an unsized member contains the metadata for that member, so a pointer to str contains the length of the [u8]:
It'll definitely have false positives then since it's a common practice to do that with nul-terminated string literals (and was even more common practice before c"..." literals were stabilized), so it will probably have to be allow-by-default, although the Clippy maintainers will have the final say in that.
Interesting. Off-topic, but do you have a notion of if it's usually string literals or if guys are appending '\0' to their strings? Because the former case I could detect in the lint.
you're describing Pascal strings with a length inline immediately before the data. Rust doesn't do that. Rust strings have length outside of the data, in the fat pointer.
Consider that &str[1..5] is possible to do on immutable data. You can't do that with Pascal strings and you can't do that with NUL-terminated strings which both store length/terminator in the data. Storing length externally allows Rust to share a long string of characters and create any number of pointer+length pairs to any subset of that data without modifying the data.
