Parquet and fs-err

1

Hello,

Using parquet 9.0.2, this compiles:

use parquet::file::reader::SerializedFileReader;
use std::{fs::File, path::Path};

fn reader() {
    let path = Path::new("/path/to/sample.parquet");
    if let Ok(file) = File::open(&path) {
        let _reader = SerializedFileReader::new(file).unwrap();
    }
}

Then, I'm adding fs-err 2.6.0, which promises to be "a drop-in replacement for std::fs that provides more helpful messages on errors." So, I'm trying:

use parquet::file::reader::SerializedFileReader;
use fs_err::File;
use std::path::Path;

fn reader() {
    let path = Path::new("/path/to/sample.parquet");
    if let Ok(file) = File::open(&path) {
        let _reader = SerializedFileReader::new(file).unwrap();
    }
}

This no longer compiles and now I get:

error[E0277]: the trait bound `fs_err::File: ChunkReader` is not satisfied
   --> src/experiment.rs:8:49
    |
8   |         let _reader = SerializedFileReader::new(file).unwrap();
    |                       ------------------------- ^^^^ the trait `ChunkReader` is not implemented for `fs_err::File`
    |                       |
    |                       required by a bound introduced by this call
    |
note: required by a bound in `SerializedFileReader::<R>::new`
   --> /home/oliverr/.cargo/registry/src/github.com-1ecc6299db9ec823/parquet-9.0.2/src/file/serialized_reader.rs:130:19
    |
130 | impl<R: 'static + ChunkReader> SerializedFileReader<R> {
    |                   ^^^^^^^^^^^ required by this bound in `SerializedFileReader::<R>::new`

error[E0277]: the trait bound `fs_err::File: ChunkReader` is not satisfied
   --> src/experiment.rs:8:23
    |
8   |         let _reader = SerializedFileReader::new(file).unwrap();
    |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `ChunkReader` is not implemented for `fs_err::File`
    |
note: required by a bound in `SerializedFileReader`
   --> /home/oliverr/.cargo/registry/src/github.com-1ecc6299db9ec823/parquet-9.0.2/src/file/serialized_reader.rs:125:36
    |
125 | pub struct SerializedFileReader<R: ChunkReader> {
    |                                    ^^^^^^^^^^^ required by this bound in `SerializedFileReader`

Am I doing something wrong, or is fs-err not a drop-in after all? Any fix?

Thanks!

Best, Oliver

2

The fs-err self-description is perhaps a little misleading. It's impossible for any type foo::File to be a drop-in replacement for std::fs::File in all cases, because some crate could implement stuff for std::fs::File but not for foo::File. Such is the case here: parquet has a trait ChunkReader that it implements for std::fs::File, but not for fs_err::File, which means you can't use the latter where parquet expects an implementor of ChunkReader, such as in SerializedFileReader::new.

Options for working around this include:

  • defining a wrapper struct FriendlyFile(pub fs_err::File) and implementing ChunkReader for FriendlyFile, probably just copying the code from parquet's impl ChunkReader for std::fs::File. (The wrapper type is necessary due to the orphan rules.)
  • asking parquet to implement ChunkReader for fs_err::File, probably behind a feature gate so that the dependency on fs-err is optional
1 Like
3

Ah, thanks, that makes sense.

4

fs_err::File also has a couple of possibly helpful methods for these cases, such as File in fs_err - Rust
This is common for wrappers like this, though naturally any errors that would bubble up from it through SerializedFileReader would be the std kind.

5

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.