Docopt and clap: feedback

Continuing the discussion from Crate of the Week:

Thank you so much for your answer @BurntSushi ! : )

That is indeed a simple stat worth noting.

I'm particularly interested in a comparison in terms of:

  • Is it easy to use ?
  • Is it hard to maintain/refactor ?
  • What situation is problematic and isn't handled properly by the library ?
    (@BurntSushi has given a good example of that with subcommand with docopt )

I'll just say what I know about Docopt. I haven't used clap before, so this should be taken as only one side of the comparison.

  • The docopt crate has API documentation, but the documentation for writing the usage string is part of the official Docopt project. So you'll need to look at that to determine what features docopt has: http://docopt.org/ It has most things you'd expect. Abbreviated short flags: -zxf. Specify flags more than once to get a list of values: --output VGA-1 --output HDMI-0. Greedy matching to support commands like cp src1 src2 src3 dst (actually, I think that is a docopt.rs special ;-)). Flags can come before or after positional arguments (toggleable). Auto-responses to --version and --help. Default values. etc etc
  • Automatic deserialization of values using rustc-serialize. (This is one of my favorite features. No fiddling with integer conversions and such. You either get the right type of values, or docopt shows an error to the user.)
  • There is a docopt! compiler plugin that will generate your struct for your from the usage string, which I think is wicked awesome.

That's pretty much it. The set of allowed CLI invocations is controlled by your usage string. I don't know where clap fits in with the above, so don't take that as a list of things docopt has that clap doesn't. It's just a list of things that docopt has. :slight_smile:

Some negatives of Docopt:

  • As I said, I have no good answer for sub-commands yet. Both Cargo and xsv do use sub-commands with docopt, but it's done by having a top-level Docopt usage string with multiple docopt usage strings for each sub-command. I actually think this is a fine thing to do, but the Docopt crate won't really help you do it. You have to wire it all up yourself. It's not bad, but it's not automatic.
  • Some failure modes can be especially bad. Docopt is pretty good about saying whether you passed a non-existent flag or committed some other syntax error, but if you passed a valid invocation that simply didn't match one of the usage patterns, then docopt doesn't say why. It just says, "Invalid flags." If your command has lots of complex usage patterns, then it may not be obvious to the user why their command isn't working.

Ignoring sub-commands, I do think Docopt is easy to use, and I think programs written with it can be easy to maintain. If you follow along in my case study that uses Docopt, it's pretty easy to add a new argument. Of course, it's a pretty trivial example, and I imagine it's just as easy as clap. I think docopt's story will get much better when we get stable compiler plugins (or equivalent functionality).

My general view of "which to use" I think is mostly focused on how you feel about either one. I don't think either are missing anything too major. By far, the most common objection I hear about Docopt is something along the lines of "writing the usage string is annoying." But then again, one of the most common praises is, "writing the usage string is awesome."

Anyway, as I wrote (and use) docopt.rs, take the above with appropriate biases. And if I've missed something, please let me know!

Also, one last thing: I've seen lots of development activity on clap which I think is great. For the most part, I think of the docopt crate as mostly done. There are some problems I've outlined above that I'd like to fix, but otherwise, I don't see much more being added. Most of the meat is in the Docopt spec itself.

2 Likes

Another point that should not be neglected is that docopt's convenience comes with a runtime cost (at least on stable Rust). For binaries that are called lots of times, and only run for a few milliseconds, this adds up.

This gist by the clap author is a comparison of the runtime for racer, with the current hand-parsing, docopt and clap.

1 Like

Thanks again for providing another interesting answer ! :sunny:

One of the reason I really enjoyed using docopt in the first place was the freedom to format your usage string as you want. It is usally tricky to do with a generated usage. Besides maintaining the usage string separately from his handling is cumbersome, and thus not a solution attractive. In my opinion, docopt really shines here by merging the two.

I guess we can hope for a natural solution with time. I perfectly understand that the project is not fully ready yet. It needs its time to mature.

This part can be annoying, is it a problem intern to the implementation or a problem that can't be sovled because of the usage string grammar?

This comparison is quite interesting! I have doubt however about the value "performance" has for arguments parsing.

To be honest, I think that if your binary is going to be called really often, then it will waste a lot of resources anyway by recreating the ones used on each call (assuming your binary does something useful). For that case, I tend to think that it is much more natural to opt for a binary that can be run as a daemon where it will process messages in a queue.

Javascript developers are quite familiar with the concept, because the code of the script itself is recompiled at each program call, so you always have a "watch" argument option or similar to handle cases were you would need to call the program multiple times.

So, in my opinion performance is often handled better by providing a more refined interface of your usage string by understanding your users needs rather than switching to a more optimized argument parsing library (which would still not solve the problem of "resources wasted by your own code").

Anything else on clap that make it attractive?

The problem is a somewhat more specific instance of "why doesn't this text (the passed arguments) match this regex (a usage pattern)?" Perhaps there are heuristics to make failure modes better. I haven't put much time or energy into thinking of a solution, but one doesn't seem readily apparent to me.

On the other hand, it should be as fast as possible because there's no good reason for it not to be. I've actually never profiled/benchmarked/optimized docopt. I'm sure there are several regex compilations and allocations that could be removed with some effort.

Sure, that is another mode of operation. But Racer aims to provide a completion backend for any editor, and not every one of those may have a scripting facility powerful enough to support long running subprocesses sanely.

Another example where startup time matters is binaries used in shell programming, and user command-line programs, although I don't think 20ms are a problem there.

Personally I'm a fan of the docopt compiler plugin (and hope we get plugins on stable rust soon); it guarantees well-formedness of the docopt string and can shift a lot of the processing to compile time as well.

I'm writing a small tar-like tool for manipulating a specific archive format. I evaluated both docopt and clap-rs, and ultimately I went with clap-rs.

docopt is interesting, but ultimately I prefer to create my argument syntax in as much code as possible so that the compiler can tell me when things are not right. docopt was irritating to me because I had to refer to some rather vague instructions about how to format my docopt strings. To know if I got it right, I had to compile and run the whole program again, whereas clap-rs usually only requires a compile to tell if you've done it correctly.

My other teething issue was that I was attempting to duplicate as closely as possible tar's syntax, which neither docopt or clap-rs really have thought about. (To their defense, I hadn't thought about it either when I wrote my argument parser for Objective-C). I ended up departing from tar's invocation and using subcommands, which made clap-rs the clear winner.

Frankly, I like both, but I like clap-rs just a little bit more.

Thanks ! That's a very insightful feedback ! :sunny: