Wednesday, November 18, 2009

Unix Options: a lisp cli parser

I recently spun off a sub project from another that I was working on recently. Index is an effort at an application-level file tagger (written in Common Lisp) and, being largely CLI driven, needed a decent CLI option parser to go with it. A quick search on Google, Cliki, and the CL-Directory didn't turn up the hits I'd like so I wrote my own for Index. Eventually, Unix Options, as I call it, grew large enough to be spun off into its own library, with unit tests and ASDF file so I did and here it is.

Unix Options attempts to support reading in options in the most idiomatic fashion possible. It supports parsing options both in short option style (single dash: '-a') long option style (double dash: '--alpha'). It recognizes grouped short options ('-abc') and recognizes file and argument parameters to options. Both '--option=parameter' and '--option parameter' syntax is recognized and '--' ends the parser and treats all remaining tokens as free tokens, as is usually expected at a Unix CLI.

Unix Options was meant to have the flexibility of the Perl GetOptions command. That is, rather than act as a simple getopt replacement for sorting tokens and making them easy to parse by the rest of the program, Unix Options provides mechanisms for automatically binding options and handling details like unsupported options and invalid input. In additions, things like usage printouts (from a '-h') options and document generation hooks are planned.

To this end, I split the 'back-end' or parser part of Unix Options, from the various 'front-end' parts which handle the values found. The function map-parsed-options is the back-end. It accepts a list of the tokens passed in on the CLI as passed in on the CLI, as well as lists of valid options. It takes two callbacks, one to handle valid options, and one to handle free tokens. It processes the list of tokens, splitting it into option-value pairs, for each option found and splitting out free tokens, running the appropriate callback for each. For options which take parameters, the parameter is passed as value; options that don't take parameters simply receive T for the value. map-parsed-options has the prototype:

(map-parsed-options cli-options bool-options param-options opt-val-func free-opt-func) 

bool-options lists all options that take no parameters ('bool' implies that they are either true or false, passed in or not) and param-options lists all that do.

While map-parsed-options is very flexible, it is also more complex than is convenient for normal use. To remedy this, there are a two 'front-ends' available with Unix Options getopt and with-cli-options. (In addition, the user can construct his own front-end using map-cli-options as the back-end, but this is unnecessary.) The full usage for either of these is better explained in the README.

getopt is a simple replacement for the Unix 'getopt' command. It's syntax is similar to that of the Python 'getopt' command and simply converts a list of CLI tokens into one more easily parsed.

The with-cli-options macro attempts to be a more comprehensive solution for binding values passed in on the CLI. The macro is best explained with a simplistic example:

(with-cli-options () 
(option &parameters file)
(if option
(print file)))

with-cli-options binds values passed in on the cli to a list of names provided by the user. Each name generates a 'spec' for binding options to it: a long option of the same name, and a short option of the first letter of the name. If more than one name is listed of the same first letter, the second listed uses the capital for the short option, and any more only use the long option. Names listed after a &parameters symbol take parameters. A list of free tokens is generated and is bound to free unless a single name listed after a &free symbol exists, in which case the list is bound to it instead. The prototype for with-cli-options is such:

(with-cli-options (&optional (cli-options *cli-options*)) option-variables &body body) 

(*cli-options* is bound to the list of cli tokens provided the Lisp implementation.)

As such, with-cli-options reduces the entire process of dealing with CLI options to about a single line of code, which I thought was neat.

Posted via email from astine's posterous

Am working right now

Watch me go.

Posted via email from astine's posterous