How to visualise this HashMap using Natvis?

help
1

Hi Guys,

I am checking the amethyst/legion crate and it is really great. However, I have a challenge in debugging it. In my code, there are many places I will have below if let,

if let Some(entry) = world.entry(entity) {
        let position = entry.into_component::<Position>().unwrap();
        println!("{}", position.x);
    }

If I visualise entry using debugger, it shows the raw data like below and I have to declare a temperory variable position to know the values in this entry. This is quite challenging because I am working on a huge simulator and using println! is less productive.
image

Hence, I thought of writing my own Natvis for visualisation but it is not working :frowning:

<Type Name="legion::internals::entry::Entry">
    <DisplayString>{{ Selva }}</DisplayString>
    <Expand HideRawView="true">
            <CustomListItems>
                  <Variable Name="i" InitialValue="0" />
                  <Variable Name="n" InitialValue="world->components.storages.base.table.items" />
                  <Size>world->components.storages.base.table.items</Size>
                  <Loop>
                        <Break Condition="n == 0" />
						<!-- This if condition need to be modified to show only items of this entry -->
                        <If Condition="(world->components.storages.base.table.ctrl.pointer[i] &amp; 0x80) == 0">
                              <Exec>n--</Exec>
                              <Item>((tuple&lt;$T1, $T2&gt;*)world.components.storages.base.table.ctrl.pointer)[-(i + 1)].__0.name.type_id.t</Item>
                              <!-- <Item>((tuple&lt;$T1, $T2&gt;*)world->components.storages.base.table.ctrl.pointer)</Item> -->
                        </If>
                        <Exec>i++</Exec>
                  </Loop>
            </CustomListItems>
    </Expand>
</Type>

I think the storages stored as HashMap is not visualised correctly (with or without my Natvis). I feel it is due to custom hasher they use, HashMap< ComponentTypeId, Box<dyn UnknownComponentStorage>, BuildHasherDefault<ComponentTypeIdHasher>.
image

Can someone help me on how I can modify this Natvis to show this HashMap?

Note: My custom natvis file is based on the natvis in Rust git repo.

2

The custom hasher is fine (I think.) There are three problems:

  • Your natvis references $T1 / $T2, which refer to the generic/template parameters of legion::internals::entry::Entry. Except, Entry has no generic/template parameters.
  • Box<dyn UnknownComponentStorage> causes... problems.
  • world.components "works" but gives different (and incorrect) results vs world->components in VS Code.

I started out by commenting out the entire natvis and uncommenting bits until I narrowed down the breakage to:

<Item>((tuple&lt;$T1, $T2&gt;*)world.components.storages.base.table.ctrl.pointer)[-(i + 1)].__0.name.type_id.t</Item>

And then further narrowed down the problem to the cast - e.g. this "works":

<Item>(world.components.storages.base.table.ctrl.pointer)</Item>

This doesn't:

<Item>((tuple&lt;$T1, $T2&gt;*)world.components.storages.base.table.ctrl.pointer)</Item>

I recall that tuples only successfully visualize if that specific tuple type was instantiated somewhere. E.g. tuple<u64, u64> might succeed when tuple<u32, u32> doesn't. So to further subdivide the problem and test replacements for $T1/$T2, it's easier to test if the (sub)types resolve without the tuple. $T1's replacement is easy enough:

<Item>((legion::internals::storage::component::ComponentTypeId*)0)</Item>

$T2 is more problematic. Rolling over the watch window in VS Code, this seems like one of these "should" work, but it doesn't:

<Item>((alloc::boxed::Box&lt;UnknownComponentStorage&gt;*)0)</Item>
<Item>((alloc::boxed::Box&lt;legion::internals::storage::UnknownComponentStorage&gt;*)0)</Item>

UnknownComponentStorage in general seems problematic, these don't work either:

<Item>((UnknownComponentStorage*)0)</Item>
<Item>((legion::internals::storage::UnknownComponentStorage*)0)</Item>

Also, if I box my own traits:

trait Foo : std::fmt::Display {}
impl Foo for i32 {}

fn main() {
    let foo : Box<dyn Foo> = Box::new(42);

The resulting type names are weird - it matches this:

    <Type Name="legion_natvis_test::Box&lt;Foo&gt;">
        <DisplayString>ASDF</DisplayString>
    </Type>

I have no idea why my test crate name is doing in the debug info for Box when used on my traits, but not when used on legion types. Fixing this properly might involve changes to rust-lang/rust. Fixing this improperly... well, you can always make wild assumptions about type layout and abuse the heck out of pointer math. A complete and working example introducing a stride:

<?xml version="1.0" encoding="utf-8"?>
<AutoVisualizer xmlns="http://schemas.microsoft.com/vstudio/debugger/natvis/2010">
    <Type Name="legion::internals::entry::Entry">
        <DisplayString>{{ Selva }}</DisplayString>
        <Expand HideRawView="true">
            <CustomListItems>
                <Variable Name="i" InitialValue="0" />
                <Variable Name="n" InitialValue="world->components.storages.base.table.items" />
                <Variable Name="s" InitialValue="sizeof(legion::internals::storage::component::ComponentTypeId) + 2*sizeof(void*)" />
                <Size>world->components.storages.base.table.items</Size>
                <Loop>
                    <Break Condition="n == 0" />
                    <If Condition="(world->components.storages.base.table.ctrl.pointer[i] &amp; 0x80) == 0">
                        <Exec>n--</Exec>
                        <Item>(legion::internals::storage::component::ComponentTypeId*)((char*)world->components.storages.base.table.ctrl.pointer - s*(i+1))</Item>
                    </If>
                    <Exec>i++</Exec>
                </Loop>
            </CustomListItems>
        </Expand>
    </Type>
</AutoVisualizer>

This makes terrible assumptions - that (ComponentTypeId, Box<dyn UnknownComponentStorage>) starts with ComponentTypeId in memory (AFAIK this is not guaranted), and that Box<dyn ...> has the same layout as a generic vtable+data fat pointer (will break if legion ever switches to a non-ZST allocator for it's Box) - but it seems to work:
image

3

And here's a version that makes the PackedStorage s in question visible. Caveats:

  • Assumes more details about fat pointer layout (data before vtable)
  • Forced to hardcode every type to enumerate
  • strstr on non-\0 terminated Rust strings is badwrong. A 'correct' implementation likely uses a length check + strncmp (memcmp isn't supported for natvis conditionals...)
  • I'm too lazy to figure out the correct indicies to use, so I'll leave that to you :wink: 
<?xml version="1.0" encoding="utf-8"?>
<AutoVisualizer xmlns="http://schemas.microsoft.com/vstudio/debugger/natvis/2010">
    <Type Name="legion::internals::entry::Entry">
        <DisplayString>{{ Selva }}</DisplayString>
        <Expand HideRawView="true">
            <CustomListItems>
                <Variable Name="i"      InitialValue="0" />
                <Variable Name="n"      InitialValue="world->components.storages.base.table.items" />
                <Variable Name="stride" InitialValue="sizeof(legion::internals::storage::component::ComponentTypeId) + 2*sizeof(void*)" />
                <Variable Name="id"     InitialValue="(legion::internals::storage::component::ComponentTypeId*)0" />
                <Variable Name="vtable" InitialValue="(void*)0" />
                <Variable Name="data"   InitialValue="(void*)0" />
                <Size>world->components.storages.base.table.items</Size>
                <Loop>
                    <Break Condition="n == 0" />
                    <If Condition="(world->components.storages.base.table.ctrl.pointer[i] &amp; 0x80) == 0">
                        <Exec>n--</Exec>
                        <Exec>id     = (legion::internals::storage::component::ComponentTypeId*)((char*)world->components.storages.base.table.ctrl.pointer - stride*(i+1))</Exec>
                        <Exec>data   = ((void**)(id+1))[0]</Exec>
                        <Exec>vtable = ((void**)(id+1))[1]</Exec>
                        <If Condition="data &amp;&amp; (char*)id->name.data_ptr == strstr((char*)id->name.data_ptr, &quot;legion_natvis_test::Position&quot;)">
                            <Item Name="{id->name}">*(legion::internals::storage::packed::PackedStorage&lt;legion_natvis_test::Position&gt;*)data</Item>
                            <Exec>data = 0</Exec>
                        </If>
                        <If Condition="data &amp;&amp; (char*)id->name.data_ptr == strstr((char*)id->name.data_ptr, &quot;legion_natvis_test::Velocity&quot;)">
                            <Item Name="{id->name}">*(legion::internals::storage::packed::PackedStorage&lt;legion_natvis_test::Velocity&gt;*)data</Item>
                            <Exec>data = 0</Exec>
                        </If>
                        <If Condition="data">
                            <!-- Unknown T for PackedStorage<T> -->
                            <Item Name="{id->name}">data</Item>
                        </If>
                    </If>
                    <Exec>i++</Exec>
                </Loop>
            </CustomListItems>
        </Expand>
    </Type>
</AutoVisualizer>
4

Thanks a lot @MaulingMonkey. I wouldn't have thought about your solution for years :slight_smile:. I copy pasted your natvis and I am able to visualise the components. But the components always show incorrect Entry. I played with indices as you suggested but not able to figure it out. I am sorry, I am a novice in C++, Rust and Natvis, so I am not able to figure it.

Below is my code,

    let entity: legion::Entity = world.push((
        Position { x: 30.0, y: 50.0 },
        Velocity {
            dx: 310.0,
            dy: 510.0,
        },
    ));
    let entity2: legion::Entity = world.push((
        Position { x: 31.0, y: 51.0 },
        Velocity {
            dx: 311.0,
            dy: 511.0,
        },
    ));

When I set a debugger at below line,

if let Some(entry) = world.entry(entity2) {
        let position = entry.into_component::<Position>().unwrap();
        println!("{}", position.x);
}

I expected Position{x:31, y:51} but I am getting Position{x:30, y:30}. Please see below pic,

image
image1135×314 27.3 KB

Any suggestion? Sorry to trouble you.

5

slices is a Vec of pointer+length combinations, which no debugger will automatically realize it should visualize as a slice - instead, it displays the first element of said slice. This is missing bounds checks and will display bogus missing components as a result, but will at least select the correct components when they're present:

<?xml version="1.0" encoding="utf-8"?>
<AutoVisualizer xmlns="http://schemas.microsoft.com/vstudio/debugger/natvis/2010">
    <Type Name="legion::internals::entry::Entry">
        <DisplayString>{{ Selva }}</DisplayString>
        <Expand HideRawView="false">
            <Item Name="[archetype]">location.__0.__0</Item><!-- use to index "index" -->
            <Item Name="[component]">location.__1.__0</Item><!-- use to index "slices[index[archetype]]" -->
            <CustomListItems>
                <Variable Name="i"      InitialValue="0" />
                <Variable Name="i2"     InitialValue="0" />
                <Variable Name="n"      InitialValue="world->components.storages.base.table.items" />
                <Variable Name="stride" InitialValue="sizeof(legion::internals::storage::component::ComponentTypeId) + 2*sizeof(void*)" />
                <Variable Name="id"     InitialValue="(legion::internals::storage::component::ComponentTypeId*)0" />
                <Variable Name="vtable" InitialValue="(void*)0" />
                <Variable Name="data"   InitialValue="(void*)0" />
                <Size>world->components.storages.base.table.items</Size>
                <Loop>
                    <Break Condition="n == 0" />
                    <If Condition="(world->components.storages.base.table.ctrl.pointer[i] &amp; 0x80) == 0">
                        <Exec>n--</Exec>
                        <Exec>id     = (legion::internals::storage::component::ComponentTypeId*)((char*)world->components.storages.base.table.ctrl.pointer - stride*(i+1))</Exec>
                        <Exec>data   = ((void**)(id+1))[0]</Exec>
                        <Exec>vtable = ((void**)(id+1))[1]</Exec>
                        <If Condition="data &amp;&amp; (char*)id->name.data_ptr == strstr((char*)id->name.data_ptr, &quot;legion_natvis_test::Position&quot;)">
                            <Exec>i2 = ((legion::internals::storage::packed::PackedStorage&lt;legion_natvis_test::Position&gt;*)data)->index.buf.ptr.pointer[location.__0.__0]</Exec>
                            <Item Name="{id->name}">((legion::internals::storage::packed::PackedStorage&lt;legion_natvis_test::Position&gt;*)data)->slices.buf.ptr.pointer[i2].__0.pointer[location.__1.__0]</Item>
                            <Exec>data = 0</Exec>
                        </If>
                        <If Condition="data &amp;&amp; (char*)id->name.data_ptr == strstr((char*)id->name.data_ptr, &quot;legion_natvis_test::Velocity&quot;)">
                            <Exec>i2 = ((legion::internals::storage::packed::PackedStorage&lt;legion_natvis_test::Velocity&gt;*)data)->index.buf.ptr.pointer[location.__0.__0]</Exec>
                            <Item Name="{id->name}">((legion::internals::storage::packed::PackedStorage&lt;legion_natvis_test::Velocity&gt;*)data)->slices.buf.ptr.pointer[i2].__0.pointer[location.__1.__0]</Item>
                            <Exec>data = 0</Exec>
                        </If>
                        <If Condition="data">
                            <!-- Unknown T for PackedStorage<T> -->
                            <Item Name="{id->name}">data</Item>
                        </If>
                    </If>
                    <Exec>i++</Exec>
                </Loop>
            </CustomListItems>
        </Expand>
    </Type>
</AutoVisualizer>

I believe:

  • i2 should be bounds checked against ((legion::internals::storage::packed::PackedStorage&lt;COMPONENT_TYPE_HERE&gt;*)data)->index.len?
  • location.__1.__0 should be bounds checked against ((legion::internals::storage::packed::PackedStorage&lt;COMPONENT_TYPE_HERE&gt;*)data)->slices.buf.ptr.pointer[i2].__1
  • there are some component versions I'm completely ignoring which probably need checking too
6

Thanks a lot @MaulingMonkey. It worked like a charm. Below is small modification I did to check for i2.

<?xml version="1.0" encoding="utf-8"?>
<AutoVisualizer xmlns="http://schemas.microsoft.com/vstudio/debugger/natvis/2010">
    <Type Name="legion::internals::entry::Entry">
        <DisplayString>{{ Selva }}</DisplayString>
        <Expand HideRawView="true">
            <Item Name="[archetype]">location.__0.__0</Item><!-- use to index "index" -->
            <Item Name="[component]">location.__1.__0</Item><!-- use to index "slices[index[archetype]]" -->
            <CustomListItems>
                <Variable Name="i"      InitialValue="0" />
                <Variable Name="i2"     InitialValue="0" />
                <Variable Name="n"      InitialValue="world->components.storages.base.table.items" />
                <Variable Name="stride" InitialValue="sizeof(legion::internals::storage::component::ComponentTypeId) + 2*sizeof(void*)" />
                <Variable Name="id"     InitialValue="(legion::internals::storage::component::ComponentTypeId*)0" />
                <Variable Name="vtable" InitialValue="(void*)0" />
                <Variable Name="data"   InitialValue="(void*)0" />
                <Size>world->components.storages.base.table.items</Size>
                <Loop>
                    <Break Condition="n == 0" />
                    <If Condition="(world->components.storages.base.table.ctrl.pointer[i] &amp; 0x80) == 0">
                        <Exec>n--</Exec>
                        <Exec>id     = (legion::internals::storage::component::ComponentTypeId*)((char*)world->components.storages.base.table.ctrl.pointer - stride*(i+1))</Exec>
                        <Exec>data   = ((void**)(id+1))[0]</Exec>
                        <Exec>vtable = ((void**)(id+1))[1]</Exec>
                        <If Condition="data &amp;&amp; (char*)id->name.data_ptr == strstr((char*)id->name.data_ptr, &quot;my_legion::Position&quot;)">
                            <Exec>i2 = ((legion::internals::storage::packed::PackedStorage&lt;my_legion::Position&gt;*)data)->index.buf.ptr.pointer[location.__0.__0]</Exec>
                            <If Condition="i2 != -1">
                                <Item Name="{id->name}">((legion::internals::storage::packed::PackedStorage&lt;my_legion::Position&gt;*)data)->slices.buf.ptr.pointer[i2].__0.pointer[location.__1.__0]</Item>
                            </If>
                            <Exec>data = 0</Exec>
                        </If>
                        <If Condition="data &amp;&amp; (char*)id->name.data_ptr == strstr((char*)id->name.data_ptr, &quot;my_legion::Velocity&quot;)">
                            <Exec>i2 = ((legion::internals::storage::packed::PackedStorage&lt;my_legion::Velocity&gt;*)data)->index.buf.ptr.pointer[location.__0.__0]</Exec>
                            <If Condition="i2 != -1">
                                <Item Name="{id->name}">((legion::internals::storage::packed::PackedStorage&lt;my_legion::Velocity&gt;*)data)->slices.buf.ptr.pointer[i2].__0.pointer[location.__1.__0]</Item>
                            </If>
                            <Exec>data = 0</Exec>
                        </If>
                        <If Condition="data">
                            <!-- Unknown T for PackedStorage<T> -->
                            <Item Name="{id->name}">data</Item>
                        </If>
                    </If>
                    <Exec>i++</Exec>
                </Loop>
            </CustomListItems>
        </Expand>
    </Type>
</AutoVisualizer>

With this condition, I am seeing only the components there in the Entry as expected.
image

Only thing I don't know is how to remove the Unable to display collection. But that should be fine as it is a small niggle I can live with :wink:

Again, thanks a lot :slight_smile:

7

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.