Does write! build up a string, or writes everything piece by piece?

1

Consider this:

write!(buffer, "{:?}, {:?}, {:?}", a, b, c);

does write! build up an intermediate string representing format!("{:?}, {:?}, {:?}", a, b, c) then writes that string ... or does it do:

write a
write ", "
write b
write ", "
write c

I.e. does it build up some intermediate string, or does it optimize that intermediate string away ?

If the former, is there a way to avoid this incremental string construction ?

Context: code generator, lots of nested / incremental writes

2

No. It just call the actual write many times piece by piece.

3

It is not obvious to me how to interpret

This macro accepts a ‘writer’, a format string, and a list of arguments. Arguments will be formatted according to the specified format string and the result will be passed to the writer. The writer may be any value with a write_fmt method; generally this comes from an implementation of either the fmt::Write or the io::Write trait. The macro returns whatever the write_fmt method returns; commonly a fmt::Result, or an io::Result.

Do you have a link proving your argument ?

4

Generally, the typical way to handle this is to wrap your output in a BufWriter, which will combine the many small writes into one large write.

Another possibility is to write into a Vec<u8>, then write the vector manually.

4 Likes
5

Well, yes, it's not that obvious. It only stated that it would call write_fmt. And the exact behaviour would be up to destination type's write_fmt implementation, which in turns calls fmt::write by default. And if you peek into the source, you can see it's indeed writing piece by piece under default behavior. An overrided io::Write::write_fmt or fmt::Write::write_fmt implementation may have different result, but it's not the case in most sane situation.

I'm not really suggesting Rust it's using source code as a specification. But I think you can safely assume Rust is doing the "better" way if there's not significant drawback.

2 Likes
6

Wait. I think I did not explain this well.

I do NOT want the intermediate format!(...) construction. I want write! to write in pieces, because I am writing lots of nested structs and I want to avoid every recursive call to construct an intermediate format!(...) string.

7

Thanks! I'm convinced. I just wanted something more reliable than "trust me bro" :slight_smile:

8

Another piece of evidence that it isn't allocating an intermediate string is that much of the formatting infrastructure (including the write! macro) is available in the core library, whereas allocation and the format! macro aren't included in core. The strings and arguments in the Arguments structure passed to write_fmt() are all provided via references.

4 Likes
9

If this is important to you, you can always check it for yourself by wrapping an existing std::fmt::Write implementation.

use std::fmt::{Error, Write};

fn main() {
    let mut buf = String::new();
    let mut writer = NoisyWriter(&mut buf);

    let a = "word";
    let b = 42;
    let c = 3.1415;

    write!(writer, "{a} {b} {c}").unwrap();
}

struct NoisyWriter<W>(W);

impl<W: Write> Write for NoisyWriter<W> {
    fn write_str(&mut self, s: &str) -> Result<(), Error> {
        println!("Writing {s:?}");
        self.0.write_str(s)
    }
}

(playground)

--- Standard Error ---
   Compiling playground v0.0.1 (/playground)
    Finished dev [unoptimized + debuginfo] target(s) in 0.53s
     Running `target/debug/playground`

--- Standard Output ---
Writing "word"
Writing " "
Writing "42"
Writing " "
Writing "3"
Writing "."
Writing "1415"
4 Likes
10

Yes, and by the way, I don't think any standard library of any programming language have specifications for every single item it provides. Thus, for the foreseeable future, reading the source code is going to remain the best way to see how exactly something works.

1 Like
11

Looking is fine. But it's not fine to then write code that relies on the implementation you saw.
The only things that are guaranteed are what's in the API contract. If you rely on something that's not in the contract a future Rust version may end up breaking your code.

2 Likes
12

The write! macro is in core and therefore cannot use a heap allocated buffer.

It might be however that format_args does some clever stack based optmizations. The builtin formatting machinery is not designed to be lightweight but for general purpose efficiency. If you need to minimize the formatting cost at runtime, f.i. because you are in an embedded context you might want to have a look at defmt.

13

In this particular case, I agree that one could try getting write! to guarantee the lack of internal buffering. But I wanted to note this is just barely within the boundary of reasonable things to document. At this level of nuances, it's quite possible that std::fmt might no longer be serving OP's use so well.

String formatting is mostly a solved problem, so in terms of stability, more and more guarantees can be made. However, a documentation can only hold so much details before it's no better than the source code. The requirements around code generation appear specific enough that these details might not always suffice.