Refactor a test case that relies on filesystem writing order

help
1

I have this test case for a function that allows sorting files by either created or modified date. I create files in sequence, but probably due to OS filesystem caching(?), it's not really creating them on the order I anticipated. This happens on the CI server, but not locally.

I am looking for alternatives to make this test outcome consistent. I thought of introducing a delay, but not quite sure how to even do that, but I think the best idea would be to maybe flush/commit pending writes to disk? Not sure, but open to ideas :slight_smile:

Below's the test case in question, and repo.save is where I actually write the file to disk:

#[test]
fn test_sorting_list() {
    let tmpdir = tmpdir("test_list").unwrap();
    temp_env::with_var("REPO_ROOT", Some(tmpdir), || {
        let repo = Repo::new("user", "1".to_string()).unwrap();
        let items = repo.list::<Item>(None).unwrap();
        let exp: Vec<String> = vec![];
        assert_eq!(items, exp);

        for n in (1..=10).rev() {
            let item = common::Item {
                name: format!("t{}", n),
                content: "content".to_string(),
            };

            repo.save(&item).unwrap();
        }

        assert_eq!(
            repo.list::<Item>(Some(ListSort::new(SortField::Name, SortDirection::Asc)))
                .unwrap(),
            vec!["t1", "t10", "t2", "t3", "t4", "t5", "t6", "t7", "t8", "t9",]
        );

        assert_eq!(
            repo.list::<Item>(Some(ListSort::new(SortField::Name, SortDirection::Desc)))
                .unwrap(),
            vec!["t9", "t8", "t7", "t6", "t5", "t4", "t3", "t2", "t10", "t1",]
        );

        assert_eq!(
            repo.list::<Item>(Some(ListSort::new(SortField::Created, SortDirection::Asc)))
                .unwrap(),
            vec!["t10", "t9", "t8", "t7", "t6", "t5", "t4", "t3", "t2", "t1",]
        );

        assert_eq!(
            repo.list::<Item>(Some(ListSort::new(SortField::Created, SortDirection::Desc)))
                .unwrap(),
            vec!["t1", "t2", "t3", "t4", "t5", "t6", "t7", "t8", "t9", "t10",]
        );

        assert_eq!(
            repo.list::<Item>(Some(ListSort::new(SortField::Modified, SortDirection::Asc)))
                .unwrap(),
            vec!["t10", "t9", "t8", "t7", "t6", "t5", "t4", "t3", "t2", "t1",]
        );

        assert_eq!(
            repo.list::<Item>(Some(ListSort::new(
                SortField::Modified,
                SortDirection::Desc
            )))
            .unwrap(),
            vec!["t1", "t2", "t3", "t4", "t5", "t6", "t7", "t8", "t9", "t10",]
        );

        let item = common::Item {
            name: "t4".to_string(),
            content: "content".to_string(),
        };

        repo.save(&item).unwrap();

        assert_eq!(
            repo.list::<Item>(Some(ListSort::new(SortField::Modified, SortDirection::Asc)))
                .unwrap(),
            vec!["t10", "t9", "t8", "t7", "t6", "t5", "t3", "t2", "t1", "t4",]
        );

        assert_eq!(
            repo.list::<Item>(Some(ListSort::new(
                SortField::Modified,
                SortDirection::Desc
            )))
            .unwrap(),
            vec!["t4", "t1", "t2", "t3", "t5", "t6", "t7", "t8", "t9", "t10",]
        );
    });
}

And here's the failure:

failures:

---- test_sorting_list stdout ----
thread 'test_sorting_list' panicked at 'assertion failed: `(left == right)`
  left: `["t10", "t9", "t8", "t7", "t5", "t6", "t4", "t2", "t3", "t1"]`,
 right: `["t10", "t9", "t8", "t7", "t6", "t5", "t4", "t3", "t2", "t1"]`', tests/repo.rs:141:9

And here's line 141 where the error occurs:

assert_eq!(
    repo.list::<Item>(Some(ListSort::new(SortField::Created, SortDirection::Asc)))
        .unwrap(),
    vec!["t10", "t9", "t8", "t7", "t6", "t5", "t4", "t3", "t2", "t1",]
);

EDIT: Adding the relevant code for the repo.save method:

let file_path = self.path_for::<T>()?.join(&file_name);
let file = fs::File::create(&file_path)?;
let mut writer = BufWriter::new(file);
serde_json::to_writer_pretty(&mut writer, &record)?;
writer.flush()?;
2

The problem is probably that you are creating files faster than the time resolution with which last-modified timestamps are stored on the file system you are using — it can be as low as 1 second.

So, if you want to test "sort by last modified" behavior, you must

  • wait long enough between creating files for the timestamps to be distinct, or
  • find or write a filesystem abstraction layer that lets you test your application logic against a fake filesystem (and hope there are no bugs in that additional code).
4 Likes
3

Now that I think about it, it makes total sense. I was here thinking it was about maybe filesystem commit/flushing timing but didn't think that maybe the files are written so close to eachother, that the resolution of the timestamp is not enough.

I will either try to figure out a mocking library or just skip this test case altogether for now.

Thank you very much.

4

Is it possible to set the created/modified dates on the filesystem you're using? Some filesystems let you modify (some of) those times (for example, Linux lets you modify atime and mtime with touch, and I bet there are ways to do it directly from Rust). If you do that, then you can make your test more resilient, by forcing the files to have correct (relative) times.

If you didn't want to muck around with breaking encapsulation in your test cases, you could even add an Option<DateTime> to the save method, and have it modify the file time if present.

2 Likes
5

I love the idea, worth researching and giving a shot. I'll do just that and reply with my findings! :slight_smile:

1 Like
6

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.