Rust beginner notes & questions

Note that you borrow self mutably and then (presumably) return slice to a buffer inside self. This will not work nicely, as you’ll have to drop &[Self::Data] before calling peek again, otherwise borrow checker will rightfully yell at you. Borrow regions could probably help here, but your proposal has another problem. How do you think it will work with e.g. buffered file IO? Also it will not work if underlying buffer is not owned without GAT.

No one forbids you from prototyping such zero-copy Read trait in a separate crate. Unfortunately you will not be able to write impl<T: ZeroCopyRead> Read for T { .. } (I really hope we will be able to do it in future), but it’s not a huge problem. If your design will be good enough and will find enought traction, then the next step could be an RFC with proposal of this trait addition to std. But as you can see you’ll probably encounter a lot of problems and blockers. I agree with @BurntSushi that it’s not productive to discuss such hand-wavy proposals, with arguably weak motivation.

2 Likes

This seems like a higher level API than Read. Non destructive peeking means the source has a buffer (either naturally or manufactured), whereas Read is just a stream. If you want to add a buffer on top and allow peeking, you can do that yourself (or use BufReader or BufRead trait in the API requirements). BufRead has a fill_buf/consume duo that can be used to do buffering and peeking, and then advancement.

I’m not really seeing an issue with Read being the lowest level API. It allows others to build on top. Is the concern then that Rust stdlib doesn’t have a Pipeline-like API out of the box?

5 Likes

Slight tangent but the sdl2 case linked in that discussion is very compelling. The issue of abstraction via methods vs running into borrowck issues comes up on this forum often as well. I think it’s easier to work around it in private APIs or code but it makes it harder to design APIs that you can’t change willy-nilly after release. This added design axis is definitely a challenge IMO.

IIUC the main issue with Read trait highlighted by @peter_bertok is that it's not suited for zero-copy processing, e.g. when you use memory mapped files or have all data in memory already. It's indeed an issue, e.g. in rosbag crate I had to write (almost) zero-copy parser myself (though something like nom could've been probably usefull), but I think that solution should be prototyped outside of the std (and maybe even stay outside of it) and that it's currently blocked on implementation of the mentioned proposals.

1 Like

Incidentally, the last time this came up on internals, we failed to come up with any use cases where a "partial borrows" feature in the core language would've actually helped. Specifically, it seemed like all the use cases could be worked around with some refactoring, and any core language change that did not require such refactoring would amount to turning a borrow check error into a silent API compatibility hazard, which seemed like a net loss.

So if anybody does know of a compelling use case where that argument doesn't hold, please necro that thread!

4 Likes

I’ve read through this thread [edit: most of it; see below] and the blogpost about the Pipelines interface, and I’m not quite sure I understand your current position on the stream-vs-pipes issue, @peter_bertok.

In light of this addendum:

  • Do you still think the Pipeline implementation in C# is an example of what you’d prefer to see in Rust?
  • Since Pipelines are implemented in C# with byte-streams, do you still think the Pipe concept is incompatible with a byte-stream API in principle?
    • You mentioned downthread that you were “wrong…[that Read2 is] inherently incompatible with what’s already there,” but you went on to say that you would expect Read to be implemented using Read2 (and gave the code for this implementation), not the other way around.
    • Since the C# version of your preferred API is implemented using bytestreams, but you think that’s the wrong approach, can you provide an example of an IO pipeline implementation that doesn’t rely on bytestreams, or flesh out your ideas for how such a thing could exist? It doesn’t seem to me that you’ve fully addressed the point upthread (unfortunately I can’t find the quote) that everything in memory really is byte-based at the lowest level, nor @BurntSushi’s point that any implementation of the Pipeline API will necessarily require a large internal buffer that the client can’t control.

In short, do you still think that Rust has taken a “wrong turn” that will have permanent negative repercussions that can’t be solved without a major breaking change (comparable to e.g. the change in byte/string handling in Python 3), or are you just concerned that the standard library is too “thin” compared to more “batteries-included” languages?

[Edit: this is much the same as @vitalyd’s question a few posts up, which was written after I began composing this post.]

This is a fairly minor point, but I’m a bit confused by your “placement” of various languages.

  • It seems like you’re saying that Rust “skipped over” the first four issues, but hasn’t adequately addressed the fifth. So would it be more accurate to say that Rust is “in between” issues 4 and 5, rather than “on” step 4?
  • I’m familiar with C++14 but don’t know as much about C++17. What (if anything) is C++ doing to address this issue? (I agree that in the general case, C++ has a major “legacy garbage” problem, but I’m unaware of anything like the Pipeline API in the C++17 standard library.)
1 Like

If you have all data in memory and want zero copy, isn’t that what a &[u8] is for? It sounds to me like Pipelines (I didn’t read that much about it so I might be wrong) manages internal buffers for you, does whatever I/O to fill them, and then exposes slices into it while allowing you to indicate that you’re done with a slice. It sounds very much like ringbuffers used in networking. I agree it’s useful but it’s fundamentally a different API and usecase from Read, or so it seems to me.

4 Likes

The sdl blog I linked above has a compelling, IMO, case. They were also able to work around it with refactoring. And I agree that with sufficient thought and ability to move code around it’s probably doable. But this requires a lot of foresight to get right in a public API, where you can’t easily make breaking changes. This virtually guarantees breaking change churn or users needing workarounds, likely at a performance cost.

The fact you can do disjoint borrows of fields is virtually essential for making Rust usable. The problem is, of course, you don’t want to expose fields in public APIs. And knowing which combination of fields callers will want to borrow, so you can do fn get(&mut self) -> (&mut i32, &mut i32) type of thing, is also not obvious in all cases. That’s a hard issue to reconcile.

2 Likes

I don’t think C++ is doing anything about this (Pipeline) either. But as mentioned, this type of memory/buffer management isn’t novel as witnessed by fairly pervasive use of ringbuffers in networking code. Similar techniques are used in DMA or kernel bypass networking, where you get true zero copy all the way to the device. For example, you provide a NIC with a DMA buffer that it fills. Or the device has buffers that it exposes to you, and you send them back to the device when you’re done reading it. If one wants to see what that might look like in Rust, the netbricks lib might be interesting - it wraps dpdk and its mbuf abstraction.

These are somewhat highly specialized because there’s a buffer/memory management policy in use. I just don’t see a stdlib providing such things, especially in a thin one like Rust’s.

But, all that said, focusing so much on Read and using it as some sort of proxy on the state of Rust as a whole is almost a disservice.

3 Likes

That’s not a different trait; it’s a re-export of the Zero trait from num-traits.

It would be nice if rustdoc had some way to indicate this.

10 Likes

THIS definitely!

3 Likes

Let my try and explain why I’m focusing on it so much: because it’s a great example. It’s one of many. It’s not the reason I abandoned Rust, it’s just one paper-cut out of hundreds.

Based on all of the Rust std lib code that I’ve seen, and reading between the lines of statements like “I don’t think C++ is doing anything about this (Pipeline) either.” that Rust seems like a “safer C++ for people that at their heart just wanted C++ 2.0 without the C legacy”.

I learned C, used C++ productively for a decade, and then moved past it and I’m not going back. If someone handed me a “C++ 2.0” with safe memory management it would be still unproductive, repetitive, and fragile compared to any truly modern language.

This has nothing to do with being a platform language – people have written operating systems in C# – and Both Java and C# are used in constrained, embedded scenarios.

This has nothing to do with POSIX I/O conventions.

What I’m saying is that the Rust team have come up with a procedural language with most of Haskell’s wonderfully advanced and elegant language features and then decided not to use it. Instead of immediately embracing an abstraction that:

  • Is composable without incurring repeating costs
  • Is extensible to non-copy types, not just u8
  • Is more elegant to use for common, practical use-cases.

The arguments I’m hearing are along the lines of “oh no… we would have to allocate a buffer in the std lib. That’s just too much! Do it yourself!”. Well… no. I won’t. Why would I, when I can just pick up a language that has an API that does zero copy by default – quite often outperforming Rust out-of-the-box – and also has a bunch of stuff written around it so that I don’t have to write my own shims between whatever the netbricks guys did and the rest of the ecosystem.

The code @BurntSushi horrifies me. He had to write code to detect encoding, do the decoding for a stream, and he has his own code for handling decompression. These aren’t special cases. This is core to the type of programming Rust was originally designed for!!! Encoding detection, character encodings, switching between stream handlers (some of which decode to UTF-8 and some that pass-through), and decompression is all in common with HTML parsing. This is core stuff for something like Servo, the project for which the language was invented. But all of this had to be reinvented for a grep tool. What the …?

I admit that the C# guys also wrote their new I/O API to be u8 only, but that’s because in C# they have no choice. The template features of the language aren’t powerful enough to support the more general case because there’s no equivalent of traits.

Rust the language is not constrained in the same way, but its core developers act as if they are.

1 Like

I think what might help here (as others have said) is some concrete code demonstrating what you want. I know I’m confused. It seems you have pointed out examples of what Rust should be doing, like C# Pipes, then only to back-track and say that no, that isn’t what you thought it was. I think everyone (I know I am) confused by the insistence that it must be in core or std instead of on crates.io. I think the idea of a “Fat Standard Library” is wrong for a number of reasons (especially at this stage of the game). It is much more useful to iterate and experiment and provide alternatives in crates.io. It’s easy to use stuff from crates.io. It’s easy to create and publish stuff there. If something gains enough traction, it’s easy enough to make an RFC to have it added to the stdllb or core if that proves valueable/prudent…

I truly feel that this is nothing more than a difference of opinion on the matter and that you are not “wrong” so much as the Rust crates.io concept is just “wrong for you” (in the sense that you don’t like the idea of it). That’s OK.

If I’ve misunderstood you in any way, my apologies. I’ve tried to follow what is being said, but, I’m not an expert on Rust (more of a lurker at this point), so, I could be missing something blatantly obvious.

6 Likes

If you’re referring to the Midori project, no, they haven’t. The project fell apart, and well before that happened, the language used evolved quite a ways away from standard C#; so much so, in fact, that it was christened M#. (Joe Duffy’s blog has many interesting details.)

6 Likes

I’d like to hear about other papercuts. I think we’ve pretty much exhausted the Read topic and are going in circles now.

There’s a lot more to Rust than just C++ 2.0, unless 2.0 means an entirely different language. There’s naturally a reason to compare with C++, and occasionally borrow ideas from. The two languages are intended for the same domain with some similar goals.

As has been said, please provide some semi fleshed out code to demonstrate this. As you say, the language is powerful so you ought to be able to show some code using the features. I’m not asking for a full blown impl but something more than pseudocode.

std allocates buffers in some circumstances - that’s not the issue. I’d like to see C# outperforming Rust out of the box though - do you have examples?

The shims, at least the ones @BurntSushi wrote, are pretty small - as he said himself, it’s less than a day’s work. I don’t understand the complaint in that light. If I didn’t have any context, I’d think that Rust doesn’t have any stdlib based on your comments.

Again as he said, those things can be extracted out into separate crates and people can reuse them. Some people can be, for example, equally “appalled” that std has zero support for http out of the box. Rust std isn’t meant to be all encompassing.

Also, C# has just recently gotten the Pipeline API, and its std has been around much longer and was always richer than Rust’s. So prior to this, did you think C# was worthless?

5 Likes

OK, this is getting ridiculous. Please stop misrepresenting my code. I didn’t reinvent decoding or decompression. I wrote small shims that farm the primary task out to something else. Those shims can be put into crates if it’s worth doing and can be reused.

16 Likes

I’d like to point out that Microsoft has been heavily investing in the last few years in extracting things out of stdlib and into nuget packages, heavily because they were not happy being tied to monolithic .net framework releases just to iterate on an API. So I’m not even sure how much they would even agree that having a fat stdlib is good.

Afaik pipelines isn’t planned to be placed in stdlib, but will instead be kept in a nuget package that can be included if the application/library wants to utilize it.

6 Likes

Interestingly, things like “EntityFramework” are Nuget packages, not in the standard lib.

.Net core, the latest incarnation of .Net, is entirely made of up of small, modular Nuget Packages, not dissimilar from Rust’s philosophy with the stdlib. See the section “NuGet as a first class delivery vehicle” in this blog.

5 Likes

I can’t. That’s not the point.

Obviously, both Read and Read2 would use exactly 1 copy in this scenario, because this scenario is a user-to-kernel call. The difference is that the Read2 API is free not to make the copy in other scenarios, such as memory-mapped files or user-mode networking. This allows the same abstraction – the exact same trait – to be the core of a much richer set of “streaming” code, not just traditional POSIX/Win32 file I/O.

Believe it or not, traditional file I/O is not the common case, and will be less and less common over time. Right now, you can buy non-volatile storage that plugs into the DIMM sockets of a server and is mapped into memory. This is the future of storage. Nobody in their right mind would tie their fate to the 1990s POSIX way of things when memory-mapped I/O is pretty much going to be 100% of all local storage I/O, and user-mode RDMA will be pretty much 100% of all server networking real soon now. There just isn’t any reasonable way to process 100Gbps Ethernet via traditional sockets, and this is also something you can buy right now.

Anyway, as you said, it’s better to just “show the code” instead of waffling on about a bunch of vaguely related topics. I’m just going to park my general philosophical issue with Rust’s API design and try and demonstrate why Read2 really is lower level than Read, yet is more flexible for higher-level code too.

First: A problem statement
Lets get back to just memory mapped I/O, because that’s a very real scenario that’s going to be increasingly common (or ought to be), and if you don’t quite see my point, some others (@newpavlov) do.

Memory mapped I/O is not just an &[u8] as @vitalyd suggested. Last time I checked, files are allowed to be many TB in size, yet 32-bit processes can only memory map at most 2GB at a time. That’s a stretch though, because the heap can fragment the entire address space even when not fully “using” it per-se. This can be troublesome with 64-bit code as well due to issues like 2 MB “large pages” interfering with the kernel’s ability to map large contiguous segments. The page table of most CPU has much lower practical limits than the theoretical 2^64 address space. Most have an addressable range between only 2^36 and 2^40 bytes.

So the solution is to manage a “sliding window” of some reasonable size, such as 128 MB on 32-bit or say 1 GB on 64-bit. This beautifully maps to the Read2 trait, which can smoothly handle sliding “views” like this without ever having to copy anything from anywhere. The user (Rust developer) of this type of API would never have to deal with the complexities of page table entry exhaustion or heap fragmentation, and their client code will smoothly work without issues, because someone more experienced then them has made the std::io code robust on their behalf.

I cannot stress this enough. This is a real problem, and no matter how much @BurntSushi insists he thinks he’s solved them, the harsh reality is that he hasn’t. For example, here’s RipGrep’s 32-build simply barfing on a file it tried to memory-map, whereas the 64-bit built doesn’t:

PS C:\Projects\VM\Virtual Hard Disks> C:\Tools\RipGrep32\rg.exe test .\fs1.vhdx
file length overflows usize
PS C:\Projects\VM\Virtual Hard Disks> C:\Tools\RipGrep\rg.exe test .\fs1.vhdx

Oops.

This is not his fault. It’s a fault of an attitude of “isn’t this just a &[u8]” when it is well established, for decades, that it isn’t.

PS: The error comes from mmap, which I’m sure contains a bunch of other bugs considering that it blithely assumes that file lengths are at most 4GB on 32-bit platforms:

Sure, this is “patchable”, but… not really. On 32-bit, ripgrep as written (in the master branch at any rate) – and all similar Rust code written by developers less experienced than @BurntSushi – will always fall back to copying Read APIs after attempting to memory map files > 2GB and failing.

If Read2 was used, everyone could have their cake and eat it too. The API by default would support zero copy, support memory mapping terabyte-sized files on 32-bit, use the same code path for both memory-mapped and streamed files, and only a couple of Rust std lib developers would have to worry about corner-cases like page table fragmentation on Xeons with large-pages enabled.

Back to the code

Lots of people are having trouble seeing how Read2 is simpler than Read. This is likely due to assuming that it must heap-allocate a Vec<T> or something.

This just isn’t the case. The trait can flexibly return any buffer to the caller, even a stack-allocated fixed-size buffer or a buffer handed to it when the source is opened. This limits the maximum “required items” that can be requested, but this is the exact same limitation that Read always has anyway, so nothing is lost:

E.g.:

// Fits on the stack of an Arduino. You
// really won't get lighter-weight than this...
struct TinyBufferRead2<T> {
    buf: [T;32],
    start: usize,
    end: usize
}

impl<T: Clone> Read2 for TinyBufferRead2<T> {
    type Data = T;
    // ...

    fn read( &mut self, required_items: usize ) -> Result<&[Self::Item],Self::Error> {
        if required_items > 32 { return Err(()) } // you asked for too much!
        // ... straightforward implementation...

        // note that Clone is required to shuffle the data around inside the buffer.
        // the most general case is for Read2 to wrap an existing Vec<T> that never 
        // grows, which doesn't require even this constraint. This is the power of Rust's 
        // type system that I would love to see utilised more!
    }
}

Meanwhile, a sliding-window mmap implementation with Read2 is trivial, and allows all existing “streaming API code” to be layered on top without having to either:

  • Assume, incorrectly, that mmap-ing a file will fit into a &[u8].
  • Write two versions, one for memory buffers and one for streams.
  • Write one version that deals with both… somehow.

What I would like to have seen in Rust
Basically, my disappointment is that what I was hoping for was something like C#'s variety of stream-based libraries, but more efficient and compiled. This is what I saw in Iterator, it’s basically C#'s IEnumerable<T> but better.

I would love to see other parts of the standard library get the same treatment. I would love it if I could “compose” streaming APIs such that decoding something like a memory-mapped OOXML file would be short, elegant, fast, and correct. E.g.:

  1. FileReader | MemoryMappedReader <- the OOXML file coming from disk (potentially > 2 GB).
  2. Chain<ZipFile> <- OOXML files are split into “parts” that have to be assembled.
  3. XmlReader <- internally switching between UTF-8 or UTF-16 encodings, so there’s a TextDecoderReader layer behind the scenes here.
  4. Base64Reader <- decoding a huge chunk of embedded stuff.
  5. JpegStream <- or whatever embedded content we’re trying to stream out efficiently.

Now, imagine the scenario where the Zip “compression” is “store” (pass-through uncompressed) and the XML encoding is UTF-8. In this case, using Read2, the first 3 wrappers up to XmlReader could just pass through the data with zero copy. The developer would have to do nothing special to enable this scenario.

4 Likes