Hi everyone!
I'm trying to write a command line utility which I'd like to be easily extensible, and the model that cargo and git both use - using third-party binaries for executing certain subcommands - seems like the way I'd like to go. I'm hoping to use clap for structuring the top-level binary's argument parsing. I see that clap claims to be able to delegate unmatched subcommands to external subcommands, but I'm unable to find much detail on how that interaction works. I'd like to see how the top-level command and sub-commands interact.
I'm trying to sift through cargo's source to understand how it implements its subcommand interface, and I've found this spot in the code where it seems to find the subcommand executable in the PATH and forward its arguments to it. Cargo uses its own subcommand forwarding, however (i.e. it doesn't use clap), but it seems that the packages that are subcommands to cargo can manage to use clap to receive the invocation from the top-level cargo binary (such as cargo-update).
My ultimate goal is to make a pair of binaries which can both be used independently, but where the top-level binary behaves like cargo, forwarding calls of a specific subcommand to the other binary. If anybody has any insight on this, I'd greatly appreciate it!
3 Likes
I would love insight into this as well!
1 Like
Be warned that Cargo's implementation of CLI interface is far from ideal!
I see that clap claims to be able to delegate unmatched subcommands to external2 subcommands, but I'm unable to find much detail on how that interaction works.
I haven't used clap much, but it seems that it does not execute external sub commands, it just says: "Hey, I have not found foo among build-in sub commands, so it is probably an external subcommand, and it was called with these arguments". It's then your responsibility to turn subcommand name into binary name (it would be cargo-foo for Cargo's case), and execute the binary, passing through arguments received from Clap.
3 Likes
clap's maintainer here. @matklad's answer is exactly correct. Check out the docs for AllowExternalSubcommands for some examples (for those that haven't seen them yet). The consumer binary will still be responsible for actually executing the external subcommand because dealing with all the case-by-case issues, security, oddities, etc. isn't something I think clap should be responsible for 
While I agree it would be nice and ergonomic for end consumers if clap handled it automagically, I think it'd be quite hard to actually implement correctly. Unfortunately, that'd be a ton of work to make generic and secure enough for all users and something I just don't have the time to do right now. This would be a great case for something built on top of clap that does this, but I don't have the bandwidth to maintain a feature like that personally.
If someone wanted to make an external crate that did this, built on clap, I'd be more than happy to provide assistance or answer questions though.
7 Likes
Thank you all for your replies!
Since this is still a problem I'll be needing to solve in the near future, I may look into how I might make a crate like you mentioned that extends clap with that functionality. @kbknapp I may take you up on your offer for advice once I get into the thick of that process, but I'm still new to rust and it may take me some time to put together an implementation of that.
3 Likes
Anytime! Also feel free to stop by the Gitter channel if you have any questions since I get those alerts quicker than urlo or github posts (usually 
)
3 Likes
If anybody's interested, I've just started building subcommand-dispatch as a means to solve the problems I brought up in this thread. It's a work in progress and I'm not nearly ready to publish it to crates.io or anything yet, but any feedback or contributions would definitely help speed things along and be much appreciated!
Edit: It seems that I jumped the gun on this crate idea. I've since discovered that process::Command is a thing, and serves this use case just about perfectly, so I'll be dropping the crate I started. ¯\_(ツ)_/¯
3 Likes
@matklad, I was wondering if you could elaborate on what you meant by
Are there certain practices that Cargo uses that strike you as bad practice specifically, or do you have examples of what you would consider to be a good CLI strategy? I'm trying to feel out the landscape because I'll be needing to write several CLI applications in the near future and I'd like to build a good mental model of how to go about it.
2 Likes
Are there certain practices that Cargo uses that strike you as bad practice specifically
The main problem is that there is a lot of duplication in most of the build-in commands. Here's a recent pull request that demonstrates the problem: Add --exclude flag by torkleyy · Pull Request #4031 · rust-lang/cargo · GitHub. One has to add the same flags and the same configuration logic to several commands, with all the perks of code duplication: boring, difficult to refactor and hard to test 
I think one of the reason for the current situation is that Cargo uses docopt, which is absolutely awesome when one writes a simple command-line utility with a moderate number of flags, but which does not support subcommands really well. If you going to use a lot of internal sub commands, I would suggest to look at clap, which has first-class support for them. Note though, that clap only solves argument parsing problem, and to handle common configuration across different subcommands you'll need to write something custom.
BTW, @kbknapp, do you have an example for clap, where different subcommands share some, but not all configuration? I've once tried to rerail Cargo from docopt to clap, but I've failed to find a satisfactory solution for sharing common configuration code (different variations of this preamble). All the solutions I've came up with were overly complex (trait object with trait inheritance and such)
5 Likes
Sorry for the long wait in a reply, I've been crazy busy lately! I don't have an example presently, but could pretty easily throw one together. Are you talking about large swaths of common args? If so, marking args as global could work, otherwise simple creating a Vec<Arg> of common args and adding them via App::args to all applicable subcommands would work too.
1 Like
No worries, being busy myself
Are you talking about large swaths of common args?
Yeah. In cargo, there are various data structures like Worspace, PackageSpec, CompileOptions, which more or less correspond to different subsets of options of various subcommands. So, the implementation function of subcommand would look like this
fn buld(ws: Worskpace, package_spec: PackageSpec, options: CompileOptions) { }
fn run(ws: Worskpace, package_spec: PackageSpec, options: CompileOptions, runOpts: RunOptions) { }
fn metadata(ws: Workspace, metadataOpts: MetadataOptions) { }
fn new(newOptions: NewOptions) { }
And the question is, how to connect these implementaion functions (which all have different signatures) with command line arguments? That is, how to specify the arguments themselves and how to deserialize them without duplication which is present currently in Cargo. I was able to code something like this:
trait CargoCommand {
fn options(&self) ->clap::SubCommand;
fn exec(&self, args: &clap::ArgMatches);
}
trait CargoWorkspaceCommand {
fn options(&self) ->clap::SubCommand;
fn exec(&self, ws: Workspace, args: &clap::ArgMatches);
}
impl <T: CargoWorkspaceCommand> CargoCommand for T {
fn options(&self) -> { /* add workspace options and then call <T as CargoWorkspaceCommand>::options */ }
fn exec(&self, args: &clap::ArgMatches) -> { /* parse Workspace from args and delegate to <T as CWC>::exec */ }
}
but this is just so more complicated than it needs to be
2 Likes