Simpleton Digest

Trials and tribulations of a professional computer nerd

Monthly Archives: December 2011

Implementing macros in bash

Note: I’ve written a follow up to this post here that talks about a better implementation and goes into some more detail. It’s still worth it to read this post to get an introduction to what I’m doing.

This work is part of a little project I’ve started call lib.sh. My hope with this project is to develop a set of libraries for bash so that one can develop medium size programs in bash without abandoning good development practices. It grew out of a desire to be able to implement new syntax in bash without resorting to source code translation. What I’ve ended up doing is creating a small macro system for bash. The approach I develop here seems to work very well though it still does have it’s limitations.

One of the earliest features I wanted for lib.sh was to have a simple, lightweight argument parsing system.  Typically in bash when you have a function or a script the “right” thing to do is something like:

myfunc()
{
  if [ $# -ne 2 ]
  then
     echo 1>&2 "Usage: $FUNCNAME arg1 arg2";
     return 1
  fi
  local arg1="$1" arg2="$2"
}

This will accomplish a few things:

  1. The number of arguments is checked, and an informative “usage” message is printed.
  2. The arguments are assigned to named variables instead of $1, $2 …
  3. The arguments are held in local variables so that they won’t overwrite other variables, or be overwritten by subsequent function calls.

Still, it’s a lot of code, and it get’s very tedious to do this for every function you write (You are using functions, right?). Wouldn’t life be wonderful if you could write something like this:

myfunc()
{
   @args arg1 arg2
}

The big hurdle to accomplishing this is that to get access to local variables like $# and $FUNCNAME and to create local variables we need to execute code inside the myfunc function, and we have pretty much no way of doing this.

I’m going to spare you the details of the many false starts I had and go straight into the solution I found.  It involves a trick involving three separate features of bash: the DEBUG trap, aliases, and eval.

Eval

Our first stop is, of course, “eval”. As with most versions of eval, bash’s takes a string and executes it as a bash command.  This is how we’re going to execute in the calling function’s context.  Allowing for a slightly uglier syntax we can do something like this:

myfunc()
{
  eval "$(args arg1 arg2)"
}

We would then have the “args” function generate the desired argument parsing code and echo it to standard out, which would then be executed by eval.  Easy!

args ()
{
    # @args ...
    local i=1 argv=( "$@" )

    echo -n "local argv=( \"\$@\" )"
    while [ "$#" -ne 0 ]
    do
        echo -n " $1=\"\$$i\""
        let ++i
        shift
    done
    echo ';'
    cat <<END
if [ \$# -ne $(( i - 1 )) ]
then
    error "Usage: \$FUNCNAME ${argv[*]}"
fi
END
}

If you’re the type of uncaring programmer who could put up with that sort of syntactic monstrosity (kidding!), you can stop now. However if you are, like me, uncompromising when it comes to such things, we must push forward to find ways of improvement.

The @macro Alias

The first, and most obvious, step for improving bash syntax is using aliases.  A bash alias is a very limited sort of pre-processor macro.  For example:

$ alias hw='echo "Hello World!"'
$ hw
Hello World!
$

It’s two big limitations are

  • It can only do basic search and replace (no arguments like the C pre-processor)
  • It is only applied on the command name, not any of the arguments (this is not precisely true but close enough for what we’re doing).

Using aliases we could imagine doing something like:

alias @macro='eval "$($MACRO_COMMAND)"'

The only remaining problem is how do we take

@macro args arg1 arg2

and turn it into

MACRO_COMMAND="args arg1 arg2" @macro

The DEBUG Trap

The answer is the most obscure part of bash we’re using for this trick: the DEBUG trap. The DEBUG trap is a special quasi-signal provided by bash that allows debugger like functionality.  For those that don’t know, “trap” is a command in bash that allows you to have a piece of bash code run whenever a particular signal is raised (like Kill or Interrupt).  In addition to the standard signals, it also can take a set of quasi-signals which are executed when various things inside the bash program happen.

The DEBUG trap is executed before every single command (all of em).  To get a sense of how this works you can pull up your bash prompt and type in

trap 'echo "Running $BASH_COMMAND"' DEBUG

Then start typing random stuff into your shell prompt.  Before each command you run, bash will run the code in the 2nd argument above, setting BASH_COMMAND to the (mostly) exact text of your command.  You may begin to already see how this might be used to accomplish our goal.

There are a couple of little things to note about the DEBUG trap:

  1. All aliases are expanded before the trap is triggered, and
  2. All comments are removed from the end of the line.

The basic idea is going to be that we’re going to inspect each command before it’s run, and if it’s a @macro command, we’re going to pull the arguments out of BASH_COMMAND and stick them in MACRO_COMMAND.  To make this a bit easier we’re going to put this into a function and then call the function from the trap

MACRO_ALIAS='eval "$($MACRO_COMMAND)"'

MACRO_ALIAS_LENGTH="${#MACRO_ALIAS}"
alias @macro="$MACRO_ALIAS"

filter_macro_command()
{
  local command="$1"
  # Check if the start of command is the same as $MACRO_ALIAS
  if [ "${command:0:$MACRO_ALIAS_LENGTH}" == "$MACRO_ALIAS" ]
  then
    # If so, assign MACRO_COMMAND to everything after it.
    MACRO_COMMAND="${command:$MACRO_ALIAS_LENGTH}"
  fi
}

trap 'filter_macro_command "$BASH_COMMAND"' DEBUG

There are a couple of things here.  One is that instead of checking lines that begin with “@macro” we have to check for what @macro expands to.  This is because aliases are expanded before DEBUG is triggered.  By just storing the expanded value in a variable and then using bashes substring operators (${VAR:START:LENGTH}) it’s not too hard to do this.  The code will work like this

@macro args arg1 arg2

# Expanding alias
eval "$($MACRO_COMMAND)" args arg1 arg2

# Call DEBUG trap, match command and set MACRO_COMMAND=" args arg1 arg2"
eval "$( args arg1 arg2 )" args arg1 arg2

So, this is pretty close.  It mostly looks like what we had in the Eval section except we still have those annoying parameters sticking off the end of the command.  Luckily, eval comes to the rescue. By adding a ‘#’ to the end of the eval, we start a comment after the command is executed, which will cause bash to ignore the rest of the command.

MACRO_ALIAS='eval "$($MACRO_COMMAND) #"'
@macro args arg1 arg2

# Expanding alias
eval "$($MACRO_COMMAND) #" args arg1 arg2

# Call DEBUG trap, match command and set MACRO_COMMAND=" args arg1 arg2"
eval "$( args arg1 arg2 ) #" args arg1 arg2

Finally

Last but not least, we add a special alias for @args to give us that added layer of convenience.

alias @args='@macro args'
myfunc()
{
  @args arg1 arg2
}

In fact, we can go further and use @macro in a generic way, giving us a way to turn any function into a macro which will have it’s output evaluated inside the context of the calling function. To support this we can just create a simple function call macroify which, when given the name of a function, generates an appropriately named “macro alias” (which is just @ + name of the function).

macroify()
{
    @macro args funcname
    alias "@$funcname=@macro $funcname"
}

macrofiy args
# We now have @args

All of this is done, in a much more fleshed out manner, in macro.sh in the lib.sh project (and args.sh for the @args implementation).

Advertisements

Introduction

So, I’m starting a blog.  This will likely just be a personal/technical blog focusing mainly on my trials and tribulations as a computer nerd.  I’ll be adding a post right after this one with actual technical content but I figure I should start off with an introduction.

I am, at the time of writing, an IT consultant with xforty technologies living in Albany, NY with my wonderful wife Liz and our adorable Shih-Tzu, Lala.  Beyond the wide variety of things I do at work I spend a good chunk of my free time working on various side projects.  I’m not sure there is much involving computers that wouldn’t fall under my “interests” so I’m not going to bother listing them out.

Ok, enough of that.