RwLock<HashMap> no entry found!

help
1

Hi all,

I have created the analog to rust-gc crate library:

with support by default Concurrent GC !!

But I have faced with the issue that when I access HaspMap through RwLock:

let mut trs = self.trs.write().unwrap();
dbg!("trs.len = {}", trs.len());
let del = trs[&tracer]; // Crashed here, no entry is found
dealloc(del.0, del.1);
trs.remove(&tracer);

https://github.com/redradist/RustGC/blob/master/gc/src/gc/sync.rs#L505

But if I iterate HashMap I am able to find the appropriate key ... but indexer for HashMap cannot find it ...

Is it an issue in standard library ?

Example code:

use easy_gc::gc::{Gc, Trace, Finalizer, GcCell};
use easy_gc::gc::sync::Gc as GlobalGc;
use easy_gc::gc::sync::GcCell as GlobalGcCell;
use easy_gc_proc_macro::{Trace, Finalizer};
use core::time;
use std::thread;
use std::time::Duration;
use std::path::Path;
use std::fs::File;
use std::error::Error;
use std::io::Write;

#[derive(Trace)]
struct MyStruct {
    jh: u32,
}

impl Drop for MyStruct {
    fn drop(&mut self) {
        println!("MyStruct in drop !!");
    }
}

impl Finalizer for MyStruct {
    fn finalize(&self) {
        // println!("MyStruct in finalize !!");
        let mut file = File::create("foo.txt");
        match file {
            Ok(mut f) => { f.write_all(b"Hello, world!") },
            Err(e) => { Err(e) },
        };
    }
}

struct MyStruct3 {
    jh: u32,
}

#[derive(Trace, Finalizer)]
struct MyStruct39(#[unsafe_ignore_trace] MyStruct3, u16);

#[derive(Trace)]
struct MyStruct2 {
    jh: Gc<u32>,
}

impl Drop for MyStruct2 {
    fn drop(&mut self) {
        println!("MyStruct in drop !!");
    }
}

impl Finalizer for MyStruct2 {
    fn finalize(&self) {
        // println!("MyStruct in finalize !!");
        let mut file = File::create("foo2.txt");
        match file {
            Ok(mut f) => { f.write_all(b"Hello, world!") },
            Err(e) => { Err(e) },
        };
    }
}

fn main() {
    println!("Hello, world!");
    match std::thread::current().name() {
        None => println!("Current Thread name is <unnamed>"),
        Some(name) => println!("Current Thread name is {}", name),
    }
    {
        println!("Before gf");
        let gf = Gc::new(2);
        println!("After gf");
        let gf1 = Gc::new(MyStruct { jh: 3 });
        let gf2 = GcCell::new(MyStruct { jh: 3 });
        gf2.borrow_mut().jh = 2;
        println!("Before gf3");
        let gf3 = Gc::new(MyStruct2 { jh: gf });
        println!("After gf3");
    }
    {
        // NOTE: Error happens !!
        // let gf2 = GlobalGc::new(MyStruct { jh: 3 });
    }
    let gf3 = Gc::new(MyStruct { jh: 3 });
    let ten_secs = time::Duration::from_secs(20);
    thread::sleep(ten_secs);
}
2

What are you using for the hashmap key? It may be that the Hash and Eq implementations don't match up, or the key gets mutated while it is in the map. Both of these will break your map in weird and wonderful ways.

(from the HashMap docs)

It is required that the keys implement the Eq and Hash traits, although this can frequently be achieved by using #[derive(PartialEq, Eq, Hash)] . If you implement these yourself, it is important that the following property holds:

k1 == k2 -> hash(k1) == hash(k2)

In other words, if two keys are equal, their hashes must be equal.

It is a logic error for a key to be modified in such a way that the key's hash, as determined by the Hash trait, or its equality, as determined by the Eq trait, changes while it is in the map. This is normally only possible through Cell , RefCell , global state, I/O, or unsafe code.

3

For key I use tracer: *const dyn Trace it is simple pointer ...

4

The key has type *const dyn Trace, which is a fat pointer (data pointer + vtable pointer). Unfortunately, in some circumstances, two different fat pointers to the same trait object can have different vtable pointers, and thus different hashes:

https://github.com/rust-lang/rust/issues/46139

You could maybe work around this by using usize or a thin pointer type like *const () as the key in the hash map. (You can use as to cast from *const dyn Trace to *const (), discarding the vtable pointer.) This will work as long as all of your objects live at different addresses. (It could fail if some of your objects can have the same address but different concrete types, for example if one is stored in a field of another, or if they are different zero-sized types.)

Alternately, you might be able to fix this by setting incremental = false and codegen-units = 1 in the [profile] sections of your Cargo.toml, for any crate that depends (directly or indirectly) on this code.

1 Like
5

But the interesting thing ...

In HashMap only one object and it cannot find it ...

Okay, I will try ...

6

But ...

I think this is exactly what I need that two different objects have different trait object pointers ...

Maybe there are some issue with Hasher and PartialEq for trait object ?

7

If your objects all occupy disjoint locations in memory, then different objects will always have different pointers. The only problem is if you might have two objects occupying the same or overlapping locations. (For example, if one of your objects is a struct, and another object stored inline in one of the fields of that struct.) This would be a problem for both "thin" and "fat" pointers.

Since you are implementing a garbage collector, I would expect that each managed value lives in a separate memory allocation. If so, then you don't need to worry about this.

8

Yes, I think the same ...
But by some reason HashMap cannot find the key ... But when I iterate over HashMap I see this key inside of container ... very strange issue

9

How are you checking the key during iteration? If you are printing it out using dbg! or println! then you will only see half of the fat pointer, the "data" part. For example:

let x: *const dyn Debug = &"hello";
dbg!(x);
// prints "x = 0x000055a6d15c84c8"

// For debugging purposes only; don't do this transmute in production code.
let (data, vtable) = unsafe { transmute::<_, (*const (), *const ())>(x) };
dbg!((data, vtable));
// prints "(data, vtable) = (0x000055a6d15c84c8, 0x000055a6d15c84e0)"

The problem with comparing trait object pointers is that the vtable part might be different even for two pointers to the same object. You won't see this in normal debug output because it doesn't print the vtable part.

1 Like
10

I am checking using such technic:

let mut trs = self.trs.write().unwrap();
for trc in (&*trs).into_iter() {
  dbg!("trc = {}", trc.0); // Key is found
}
let del = trs[&tracer]; // Crash with panic
11

The interesting fact is that:

let mut trs = self.trs.write().unwrap();
dbg!("tracer = {}", tracer);
dbg!("trs.len = {}", trs.len());
for trc in (&*trs).into_iter() {
   dbg!("trc = {}", trc.0);
   if *trc.0 == tracer {
     dbg!("Key is found, trc = {}", trc.0);
   } else {
      dbg!("tracer{} != trc{}", tracer, trc.0);
   }
}

Produce the following result:

"tracer = {}" = "tracer = {}"
tracer = 0x0000020ee3a28be0
"trs.len = {}" = "trs.len = {}"
trs.len() = 1
"trc = {}" = "trc = {}"
trc.0 = 0x0000020ee3a28be0
"tracer{} != trc{}" = "tracer{} != trc{}"
tracer = 0x0000020ee3a28be0
trc.0 = 0x0000020ee3a28be0

This mean that tracer = 0x0000020ee3a28be0 != trc.0 = 0x0000020ee3a28be0 !!

How it is possible ?

12

tracer is a "fat" pointer: It contains two pointers. But dbg! only prints out one of the two pointers, so you can't use dbg! to see whether two fat pointers are equal. See the code above for a way to print out both parts of the fat pointer.

13

Yeah, I have printed in new and in drop method the pointer value:

"Gc<T>::new()" = "Gc<T>::new()"
(data, vtable) = (
    0x000001ad86196030,
    0x00007ff60d9f0fe8,
)
Gc::drop
(data, vtable) = (
    0x000001ad86196030,
    0x00007ff60d9f0438,
)

I cannot get why vtable is different for new and drop methods ...

14

The link above explains that different pointers to the same trait object are not guaranteed to be equal because the compiler may generate multiple copies of the vtable. This is just an unfortunate fact of how trait objects are compiled currently. See my earlier comments for how to work around this problem.

15

Okay, I got the idea ...

But will be it fixed later ?
... because it is very strange behavior ... :frowning:

16

It's not clear yet whether this will be fixed. I agree that it's very surprising. If it isn't fixed, I hope it can at least be made less surprising, maybe by adding new compiler warnings:

https://github.com/rust-lang/rust/issues/69757

17

Thank you !!

I successfully fixed the issue ... :wink:

18

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.