Toward Better Crate Dependency Graphs

community
#1

Judging from the search hits in google and on crates.io (below), many people seem to think dependency graphs of rust crates are useful. Here's a recent blog post about coming to a dependency graph, from the other, manual, human luddite direction:

Important Crates Graph

Have β‰₯5 inter-dependent crates? You need a graph.

(I'd include the SVG but its apparently not supported here, so please click and have a look.)

Graph Automation Crates

Automation is good, if it can help yield similar results. Just searched a little in this vast landscape:

deprecated in favor of cargo-graph

Doesn't build (via cargo install) for me, per below: Victim of cargo patch feature?

error: failed to compile cargo-graph v0.3.1, intermediate artifacts can be found at /tmp/cargo-installWtM0pq

Caused by:
failed to select a version for the requirement clap = "~2.11.3"
candidate versions found which didn't match: 2.33.3, 2.33.2, 2.33.1, ...
location searched: crates.io index
required by package cargo-graph v0.3.1

⸘git.sr.htβ€½

Installed. Made fun of its default output in my blog postβ€”no offense intended, just as a form of hopefully constructive criticism.

I'm pretty sure there is more than what I've found and listed above.

1st question: Who is or wants to become the "heir apparent" in this fertile space?

2nd question: Who might be interested in finding a way to add configuration or flags to such a cargo plugin, in order to be able to automatically arrive at similar quality?

Adding features like:

  • Allowed/disallowed list of deps and/or maximum (e.g. transitive 1Β° 2°…) distance setting from root
    of project/workspace.

  • Filtering deps by number of connections (e.g. filtering out leaf nodes that are wholly owned by another closer crate dependency.

  • Providing a means to group or otherwise color code crate (like I did in the above).

  • Distinguish between private and public (and default or non-default feature) dependencies, via edge styles or other markers (like mine above).

  • Any other interesting statistical/heuristic improvements to default graph output, without additional settings.

Also what about some kind of output mode flag for alternatives to graphviz dot like:

Anyone interested in working with me on this? I think I'd want "+1" or commit bit to a github hosted source tree, CI, etc. Or lacking that, maybe, I'll just fork and forget, if I can find the time? Thanks for your code under permissive license!

1 Like
Can't install binary due to yanked dependency?
#2

@cuviper pointed me at cargo-deps being worth an install. This is approaching usable (but not great IMO) if I don't use the --all-deps or any other add-more-crates type flags:

yah
yah2404Γ—827 242 KB

Sadly this is missing the whole hyperium-tokio-rs nexus which is important for reasons described in the blog post. Here is --depth 5 --optional-deps which becomes unwieldy as expected:

yah2
yah27690Γ—1019 1.14 MB

#3

Tried again with cargo-depgraph (the other repo that looks alive). The main problem I see with the output (as well as cargo-deps) is that they use incrementing integers for node names and thus its very unfriendly for hand customization:

#! cargo depgraph --focus=body-image --all-features

digraph {
    0 [ label = "body-image" shape = box]
    1 [ label = "barc" shape = box]
    2 [ label = "body-image-futio" shape = box]
    3 [ label = "barc-cli" shape = box]
    1 -> 0 [ ]
    2 -> 0 [ ]
    3 -> 1 [ ]
    3 -> 0 [ ]
    3 -> 2 [ style = dotted]
}

This is what lead me to just do itβ„’ by text editor and scan of Cargo.toml files, initially. In my hand roled version, node 0 is named bodyimage (dot doesn't like '-' or '_' in the names), but then all the edge associations are intuitive.

Note in theory one could find a collision by simply removing special characters from the crate name. A better strategy for the tool would be to append an incremented integer value, in the unlikely event that a collision occurs (if and only if).

cc: @jplatte

Rendered:

depgraph-focus

#4

Hi! I'd be stoked to help you find your way around cargo-depgraph to implement any of the features you are missing.

One note about the the numbers in the output though: this is the output of petgraph's graphviz export and I'm not sure whether the node names are customizable. I don't exactly understand why this would complicate customization too. (are you talking about editing it by hand, or passing cargo-depgraph's output to a customization script before dot?)

If it helps, I can mirror the repository on GitHub. I'm not a huge fan of GitHub because it's closed-source and a big point of centralization in Open Source, but also haven't been super happy with the rejection of any social features (stars, reactions, ...) in sourcehut so I think for now the best solution in many cases may just be to use multiple platforms.

1 Like
#5

Do you need it to be visual? An ASCII tree can be generated by cargo tree (built-in command). IMHO it works well. It has some useful features like cargo tree -d that shows duplicate crates.

1 Like
#6

Thanks Jonas! Your hosting choice seems well reasoned. I was mostly just being a jerk about the git.sr.ht host name, which I literally assumed was either a joke or a typo when I first saw it, coming from the repository link of crates.io, your crate.

I think I could work directly with git.sr.ht, if you don't mind patch emails with long commit messages? (Old school that.)

I hadn't really looked at your code yet, but the fact that the graph is represented via petgraph means that I could offer my own tree-walker-style output mode, right?

And yes, if you look at my hand done graph's dot source code (I'll share an excerpt below), you will see that I'm customizing down at the level of individual nodes (e.g. xhtml label, colors) and edges (e.g. weights):

  taolog [ label=<tao-log>, fillcolor="#eed35b" ];
  log [ fillcolor="#e93d19" ];

  taolog -> log [ weight=2 ];

I'm super excited to try and dive in!

#7

Yeah I use that all the time as well, because its so immediate and easy for small sub-graphs. However for the use cases I describe in the above linked blog post, I think its inferior, primarily because of the duplication (below as (*)).

Included below the whole cargo tree output. How do we tell this renderer to not assume every code block (e.g. ```) is rust?

barc-cli v2.2.0 (/home/david/src/body-image/barc-cli)
β”œβ”€β”€ barc v2.2.1 (/home/david/src/body-image/barc)
β”‚   β”œβ”€β”€ body-image v2.2.1 (/home/david/src/body-image/body-image)
β”‚   β”‚   β”œβ”€β”€ bytes v1.0.1
β”‚   β”‚   β”œβ”€β”€ http v0.2.3
β”‚   β”‚   β”‚   β”œβ”€β”€ bytes v1.0.1
β”‚   β”‚   β”‚   β”œβ”€β”€ fnv v1.0.7
β”‚   β”‚   β”‚   └── itoa v0.4.7
β”‚   β”‚   β”œβ”€β”€ memmap v0.7.0
β”‚   β”‚   β”‚   └── libc v0.2.82
β”‚   β”‚   β”œβ”€β”€ olio v1.4.0
β”‚   β”‚   β”‚   β”œβ”€β”€ libc v0.2.82
β”‚   β”‚   β”‚   └── memmap v0.7.0 (*)
β”‚   β”‚   β”œβ”€β”€ remove_dir_all v0.5.2
β”‚   β”‚   β”œβ”€β”€ tao-log v1.0.1
β”‚   β”‚   β”‚   └── log v0.4.8
β”‚   β”‚   β”‚       └── cfg-if v0.1.10
β”‚   β”‚   └── tempfile v3.2.0
β”‚   β”‚       β”œβ”€β”€ cfg-if v1.0.0
β”‚   β”‚       β”œβ”€β”€ libc v0.2.82
β”‚   β”‚       β”œβ”€β”€ rand v0.8.2
β”‚   β”‚       β”‚   β”œβ”€β”€ libc v0.2.82
β”‚   β”‚       β”‚   β”œβ”€β”€ rand_chacha v0.3.0
β”‚   β”‚       β”‚   β”‚   β”œβ”€β”€ ppv-lite86 v0.2.10
β”‚   β”‚       β”‚   β”‚   └── rand_core v0.6.1
β”‚   β”‚       β”‚   β”‚       └── getrandom v0.2.1
β”‚   β”‚       β”‚   β”‚           β”œβ”€β”€ cfg-if v1.0.0
β”‚   β”‚       β”‚   β”‚           └── libc v0.2.82
β”‚   β”‚       β”‚   └── rand_core v0.6.1 (*)
β”‚   β”‚       └── remove_dir_all v0.5.2
β”‚   β”œβ”€β”€ brotli v3.3.0
β”‚   β”‚   β”œβ”€β”€ alloc-no-stdlib v2.0.1
β”‚   β”‚   β”œβ”€β”€ alloc-stdlib v0.2.1
β”‚   β”‚   β”‚   └── alloc-no-stdlib v2.0.1
β”‚   β”‚   └── brotli-decompressor v2.3.1
β”‚   β”‚       β”œβ”€β”€ alloc-no-stdlib v2.0.1
β”‚   β”‚       └── alloc-stdlib v0.2.1 (*)
β”‚   β”œβ”€β”€ bytes v1.0.1
β”‚   β”œβ”€β”€ flate2 v1.0.19
β”‚   β”‚   β”œβ”€β”€ cfg-if v1.0.0
β”‚   β”‚   β”œβ”€β”€ crc32fast v1.2.1
β”‚   β”‚   β”‚   └── cfg-if v1.0.0
β”‚   β”‚   β”œβ”€β”€ libc v0.2.82
β”‚   β”‚   └── miniz_oxide v0.4.3
β”‚   β”‚       └── adler v0.2.3
β”‚   β”‚       [build-dependencies]
β”‚   β”‚       └── autocfg v1.0.1
β”‚   β”œβ”€β”€ http v0.2.3 (*)
β”‚   β”œβ”€β”€ httparse v1.3.4
β”‚   β”œβ”€β”€ memmap v0.7.0 (*)
β”‚   β”œβ”€β”€ mime v0.3.16
β”‚   β”œβ”€β”€ olio v1.4.0 (*)
β”‚   └── tao-log v1.0.1 (*)
β”œβ”€β”€ body-image v2.2.1 (/home/david/src/body-image/body-image) (*)
β”œβ”€β”€ body-image-futio v2.2.1 (/home/david/src/body-image/body-image-futio)
β”‚   β”œβ”€β”€ blocking-permit v1.3.0
β”‚   β”‚   β”œβ”€β”€ bytes v1.0.1
β”‚   β”‚   β”œβ”€β”€ futures-core v0.3.12
β”‚   β”‚   β”œβ”€β”€ log v0.4.8 (*)
β”‚   β”‚   β”œβ”€β”€ num_cpus v1.13.0
β”‚   β”‚   β”‚   └── libc v0.2.82
β”‚   β”‚   β”œβ”€β”€ parking_lot v0.11.1
β”‚   β”‚   β”‚   β”œβ”€β”€ instant v0.1.9
β”‚   β”‚   β”‚   β”‚   └── cfg-if v1.0.0
β”‚   β”‚   β”‚   β”œβ”€β”€ lock_api v0.4.2
β”‚   β”‚   β”‚   β”‚   └── scopeguard v1.1.0
β”‚   β”‚   β”‚   └── parking_lot_core v0.8.2
β”‚   β”‚   β”‚       β”œβ”€β”€ cfg-if v1.0.0
β”‚   β”‚   β”‚       β”œβ”€β”€ instant v0.1.9 (*)
β”‚   β”‚   β”‚       β”œβ”€β”€ libc v0.2.82
β”‚   β”‚   β”‚       └── smallvec v1.6.1
β”‚   β”‚   └── tokio v1.0.2
β”‚   β”‚       β”œβ”€β”€ libc v0.2.82
β”‚   β”‚       β”œβ”€β”€ mio v0.7.7
β”‚   β”‚       β”‚   β”œβ”€β”€ libc v0.2.82
β”‚   β”‚       β”‚   └── log v0.4.8 (*)
β”‚   β”‚       β”œβ”€β”€ num_cpus v1.13.0 (*)
β”‚   β”‚       └── pin-project-lite v0.2.4
β”‚   β”‚       [build-dependencies]
β”‚   β”‚       └── autocfg v1.0.1
β”‚   β”œβ”€β”€ body-image v2.2.1 (/home/david/src/body-image/body-image) (*)
β”‚   β”œβ”€β”€ brotli v3.3.0 (*)
β”‚   β”œβ”€β”€ bytes v1.0.1
β”‚   β”œβ”€β”€ flate2 v1.0.19 (*)
β”‚   β”œβ”€β”€ futures-core v0.3.12
β”‚   β”œβ”€β”€ futures-sink v0.3.12
β”‚   β”œβ”€β”€ futures-util v0.3.12
β”‚   β”‚   β”œβ”€β”€ futures-core v0.3.12
β”‚   β”‚   β”œβ”€β”€ futures-macro v0.3.12 (proc-macro)
β”‚   β”‚   β”‚   β”œβ”€β”€ proc-macro-hack v0.5.19 (proc-macro)
β”‚   β”‚   β”‚   β”œβ”€β”€ proc-macro2 v1.0.24
β”‚   β”‚   β”‚   β”‚   └── unicode-xid v0.2.1
β”‚   β”‚   β”‚   β”œβ”€β”€ quote v1.0.8
β”‚   β”‚   β”‚   β”‚   └── proc-macro2 v1.0.24 (*)
β”‚   β”‚   β”‚   └── syn v1.0.58
β”‚   β”‚   β”‚       β”œβ”€β”€ proc-macro2 v1.0.24 (*)
β”‚   β”‚   β”‚       β”œβ”€β”€ quote v1.0.8 (*)
β”‚   β”‚   β”‚       └── unicode-xid v0.2.1
β”‚   β”‚   β”œβ”€β”€ futures-sink v0.3.12
β”‚   β”‚   β”œβ”€β”€ futures-task v0.3.12
β”‚   β”‚   β”‚   └── once_cell v1.5.2
β”‚   β”‚   β”œβ”€β”€ pin-project-lite v0.2.4
β”‚   β”‚   β”œβ”€β”€ pin-utils v0.1.0
β”‚   β”‚   β”œβ”€β”€ proc-macro-hack v0.5.19 (proc-macro)
β”‚   β”‚   β”œβ”€β”€ proc-macro-nested v0.1.7
β”‚   β”‚   └── slab v0.4.2
β”‚   β”œβ”€β”€ http v0.2.3 (*)
β”‚   β”œβ”€β”€ http-body v0.4.0
β”‚   β”‚   β”œβ”€β”€ bytes v1.0.1
β”‚   β”‚   └── http v0.2.3 (*)
β”‚   β”œβ”€β”€ httparse v1.3.4
β”‚   β”œβ”€β”€ hyper v0.14.2
β”‚   β”‚   β”œβ”€β”€ bytes v1.0.1
β”‚   β”‚   β”œβ”€β”€ futures-channel v0.3.12
β”‚   β”‚   β”‚   └── futures-core v0.3.12
β”‚   β”‚   β”œβ”€β”€ futures-core v0.3.12
β”‚   β”‚   β”œβ”€β”€ futures-util v0.3.12 (*)
β”‚   β”‚   β”œβ”€β”€ http v0.2.3 (*)
β”‚   β”‚   β”œβ”€β”€ http-body v0.4.0 (*)
β”‚   β”‚   β”œβ”€β”€ httparse v1.3.4
β”‚   β”‚   β”œβ”€β”€ httpdate v0.3.2
β”‚   β”‚   β”œβ”€β”€ itoa v0.4.7
β”‚   β”‚   β”œβ”€β”€ pin-project v1.0.4
β”‚   β”‚   β”‚   └── pin-project-internal v1.0.4 (proc-macro)
β”‚   β”‚   β”‚       β”œβ”€β”€ proc-macro2 v1.0.24 (*)
β”‚   β”‚   β”‚       β”œβ”€β”€ quote v1.0.8 (*)
β”‚   β”‚   β”‚       └── syn v1.0.58 (*)
β”‚   β”‚   β”œβ”€β”€ socket2 v0.3.19
β”‚   β”‚   β”‚   β”œβ”€β”€ cfg-if v1.0.0
β”‚   β”‚   β”‚   └── libc v0.2.82
β”‚   β”‚   β”œβ”€β”€ tokio v1.0.2 (*)
β”‚   β”‚   β”œβ”€β”€ tower-service v0.3.0
β”‚   β”‚   β”œβ”€β”€ tracing v0.1.22
β”‚   β”‚   β”‚   β”œβ”€β”€ cfg-if v1.0.0
β”‚   β”‚   β”‚   β”œβ”€β”€ pin-project-lite v0.2.4
β”‚   β”‚   β”‚   └── tracing-core v0.1.17
β”‚   β”‚   β”‚       └── lazy_static v1.4.0
β”‚   β”‚   └── want v0.3.0
β”‚   β”‚       β”œβ”€β”€ log v0.4.8 (*)
β”‚   β”‚       └── try-lock v0.2.3
β”‚   β”œβ”€β”€ hyper-tls v0.5.0
β”‚   β”‚   β”œβ”€β”€ bytes v1.0.1
β”‚   β”‚   β”œβ”€β”€ hyper v0.14.2 (*)
β”‚   β”‚   β”œβ”€β”€ native-tls v0.2.7
β”‚   β”‚   β”‚   β”œβ”€β”€ log v0.4.8 (*)
β”‚   β”‚   β”‚   β”œβ”€β”€ openssl v0.10.32
β”‚   β”‚   β”‚   β”‚   β”œβ”€β”€ bitflags v1.2.1
β”‚   β”‚   β”‚   β”‚   β”œβ”€β”€ cfg-if v1.0.0
β”‚   β”‚   β”‚   β”‚   β”œβ”€β”€ foreign-types v0.3.2
β”‚   β”‚   β”‚   β”‚   β”‚   └── foreign-types-shared v0.1.1
β”‚   β”‚   β”‚   β”‚   β”œβ”€β”€ lazy_static v1.4.0
β”‚   β”‚   β”‚   β”‚   β”œβ”€β”€ libc v0.2.82
β”‚   β”‚   β”‚   β”‚   └── openssl-sys v0.9.60
β”‚   β”‚   β”‚   β”‚       └── libc v0.2.82
β”‚   β”‚   β”‚   β”‚       [build-dependencies]
β”‚   β”‚   β”‚   β”‚       β”œβ”€β”€ autocfg v1.0.1
β”‚   β”‚   β”‚   β”‚       β”œβ”€β”€ cc v1.0.66
β”‚   β”‚   β”‚   β”‚       └── pkg-config v0.3.19
β”‚   β”‚   β”‚   β”œβ”€β”€ openssl-probe v0.1.2
β”‚   β”‚   β”‚   └── openssl-sys v0.9.60 (*)
β”‚   β”‚   β”œβ”€β”€ tokio v1.0.2 (*)
β”‚   β”‚   └── tokio-native-tls v0.3.0
β”‚   β”‚       β”œβ”€β”€ native-tls v0.2.7 (*)
β”‚   β”‚       └── tokio v1.0.2 (*)
β”‚   β”œβ”€β”€ hyperx v1.3.0
β”‚   β”‚   β”œβ”€β”€ base64 v0.13.0
β”‚   β”‚   β”œβ”€β”€ bytes v1.0.1
β”‚   β”‚   β”œβ”€β”€ http v0.2.3 (*)
β”‚   β”‚   β”œβ”€β”€ httparse v1.3.4
β”‚   β”‚   β”œβ”€β”€ httpdate v0.3.2
β”‚   β”‚   β”œβ”€β”€ language-tags v0.2.2
β”‚   β”‚   β”œβ”€β”€ mime v0.3.16
β”‚   β”‚   β”œβ”€β”€ percent-encoding v2.1.0
β”‚   β”‚   └── unicase v2.6.0
β”‚   β”‚       [build-dependencies]
β”‚   β”‚       └── version_check v0.9.2
β”‚   β”œβ”€β”€ memmap v0.7.0 (*)
β”‚   β”œβ”€β”€ olio v1.4.0 (*)
β”‚   β”œβ”€β”€ tao-log v1.0.1 (*)
β”‚   └── tokio v1.0.2 (*)
β”œβ”€β”€ clap v2.33.3
β”‚   β”œβ”€β”€ bitflags v1.2.1
β”‚   β”œβ”€β”€ term_size v0.3.2
β”‚   β”‚   └── libc v0.2.82
β”‚   β”œβ”€β”€ textwrap v0.11.0
β”‚   β”‚   β”œβ”€β”€ term_size v0.3.2 (*)
β”‚   β”‚   └── unicode-width v0.1.8
β”‚   └── unicode-width v0.1.8
β”œβ”€β”€ http v0.2.3 (*)
β”œβ”€β”€ hyper v0.14.2 (*)
β”œβ”€β”€ piccolog v1.0.2 (/home/david/src/body-image/piccolog)
β”‚   └── log v0.4.8 (*)
└── tao-log v1.0.1 (*)
#8

How do we tell this renderer to not assume every code block is rust?
like this: ```text

1 Like
#9

(Thanks, corrected. I've been using txt all along. Isn't that kind of sad it can't interpret that? Also for the dot files, its best to use ruby for formatting. :slight_smile: )

#10

Honestly the whole email workflow thing is the feature I like least about sourcehut. I don't think I've even set up a mailing list for any of my projects there. Since I haven't set anything else up feel free to send patches to my email address from the git log, create todos with a link to your fork at cargo-depgraph β€” sourcehut todo or whatever else might be convenienct for you. I'll probably set up a GitHub mirror soon and make issues from the current todos.

1 Like
#11

Sure, that should be possible.

1 Like
#12
1 Like
#13

Forked! (In the github sense, not "I'm going to publish cargo-depgraph2" sense!)