Simpleton Digest

Trials and tribulations of a professional computer nerd

Implementing macros in bash

Note: I’ve written a follow up to this post here that talks about a better implementation and goes into some more detail. It’s still worth it to read this post to get an introduction to what I’m doing.

This work is part of a little project I’ve started call My hope with this project is to develop a set of libraries for bash so that one can develop medium size programs in bash without abandoning good development practices. It grew out of a desire to be able to implement new syntax in bash without resorting to source code translation. What I’ve ended up doing is creating a small macro system for bash. The approach I develop here seems to work very well though it still does have it’s limitations.

One of the earliest features I wanted for was to have a simple, lightweight argument parsing system.  Typically in bash when you have a function or a script the “right” thing to do is something like:

  if [ $# -ne 2 ]
     echo 1>&2 "Usage: $FUNCNAME arg1 arg2";
     return 1
  local arg1="$1" arg2="$2"

This will accomplish a few things:

  1. The number of arguments is checked, and an informative “usage” message is printed.
  2. The arguments are assigned to named variables instead of $1, $2 …
  3. The arguments are held in local variables so that they won’t overwrite other variables, or be overwritten by subsequent function calls.

Still, it’s a lot of code, and it get’s very tedious to do this for every function you write (You are using functions, right?). Wouldn’t life be wonderful if you could write something like this:

   @args arg1 arg2

The big hurdle to accomplishing this is that to get access to local variables like $# and $FUNCNAME and to create local variables we need to execute code inside the myfunc function, and we have pretty much no way of doing this.

I’m going to spare you the details of the many false starts I had and go straight into the solution I found.  It involves a trick involving three separate features of bash: the DEBUG trap, aliases, and eval.


Our first stop is, of course, “eval”. As with most versions of eval, bash’s takes a string and executes it as a bash command.  This is how we’re going to execute in the calling function’s context.  Allowing for a slightly uglier syntax we can do something like this:

  eval "$(args arg1 arg2)"

We would then have the “args” function generate the desired argument parsing code and echo it to standard out, which would then be executed by eval.  Easy!

args ()
    # @args ...
    local i=1 argv=( "$@" )

    echo -n "local argv=( \"\$@\" )"
    while [ "$#" -ne 0 ]
        echo -n " $1=\"\$$i\""
        let ++i
    echo ';'
    cat <<END
if [ \$# -ne $(( i - 1 )) ]
    error "Usage: \$FUNCNAME ${argv[*]}"

If you’re the type of uncaring programmer who could put up with that sort of syntactic monstrosity (kidding!), you can stop now. However if you are, like me, uncompromising when it comes to such things, we must push forward to find ways of improvement.

The @macro Alias

The first, and most obvious, step for improving bash syntax is using aliases.  A bash alias is a very limited sort of pre-processor macro.  For example:

$ alias hw='echo "Hello World!"'
$ hw
Hello World!

It’s two big limitations are

  • It can only do basic search and replace (no arguments like the C pre-processor)
  • It is only applied on the command name, not any of the arguments (this is not precisely true but close enough for what we’re doing).

Using aliases we could imagine doing something like:

alias @macro='eval "$($MACRO_COMMAND)"'

The only remaining problem is how do we take

@macro args arg1 arg2

and turn it into

MACRO_COMMAND="args arg1 arg2" @macro

The DEBUG Trap

The answer is the most obscure part of bash we’re using for this trick: the DEBUG trap. The DEBUG trap is a special quasi-signal provided by bash that allows debugger like functionality.  For those that don’t know, “trap” is a command in bash that allows you to have a piece of bash code run whenever a particular signal is raised (like Kill or Interrupt).  In addition to the standard signals, it also can take a set of quasi-signals which are executed when various things inside the bash program happen.

The DEBUG trap is executed before every single command (all of em).  To get a sense of how this works you can pull up your bash prompt and type in

trap 'echo "Running $BASH_COMMAND"' DEBUG

Then start typing random stuff into your shell prompt.  Before each command you run, bash will run the code in the 2nd argument above, setting BASH_COMMAND to the (mostly) exact text of your command.  You may begin to already see how this might be used to accomplish our goal.

There are a couple of little things to note about the DEBUG trap:

  1. All aliases are expanded before the trap is triggered, and
  2. All comments are removed from the end of the line.

The basic idea is going to be that we’re going to inspect each command before it’s run, and if it’s a @macro command, we’re going to pull the arguments out of BASH_COMMAND and stick them in MACRO_COMMAND.  To make this a bit easier we’re going to put this into a function and then call the function from the trap


alias @macro="$MACRO_ALIAS"

  local command="$1"
  # Check if the start of command is the same as $MACRO_ALIAS
  if [ "${command:0:$MACRO_ALIAS_LENGTH}" == "$MACRO_ALIAS" ]
    # If so, assign MACRO_COMMAND to everything after it.

trap 'filter_macro_command "$BASH_COMMAND"' DEBUG

There are a couple of things here.  One is that instead of checking lines that begin with “@macro” we have to check for what @macro expands to.  This is because aliases are expanded before DEBUG is triggered.  By just storing the expanded value in a variable and then using bashes substring operators (${VAR:START:LENGTH}) it’s not too hard to do this.  The code will work like this

@macro args arg1 arg2

# Expanding alias
eval "$($MACRO_COMMAND)" args arg1 arg2

# Call DEBUG trap, match command and set MACRO_COMMAND=" args arg1 arg2"
eval "$( args arg1 arg2 )" args arg1 arg2

So, this is pretty close.  It mostly looks like what we had in the Eval section except we still have those annoying parameters sticking off the end of the command.  Luckily, eval comes to the rescue. By adding a ‘#’ to the end of the eval, we start a comment after the command is executed, which will cause bash to ignore the rest of the command.

@macro args arg1 arg2

# Expanding alias
eval "$($MACRO_COMMAND) #" args arg1 arg2

# Call DEBUG trap, match command and set MACRO_COMMAND=" args arg1 arg2"
eval "$( args arg1 arg2 ) #" args arg1 arg2


Last but not least, we add a special alias for @args to give us that added layer of convenience.

alias @args='@macro args'
  @args arg1 arg2

In fact, we can go further and use @macro in a generic way, giving us a way to turn any function into a macro which will have it’s output evaluated inside the context of the calling function. To support this we can just create a simple function call macroify which, when given the name of a function, generates an appropriately named “macro alias” (which is just @ + name of the function).

    @macro args funcname
    alias "@$funcname=@macro $funcname"

macrofiy args
# We now have @args

All of this is done, in a much more fleshed out manner, in in the project (and for the @args implementation).


5 responses to “Implementing macros in bash

  1. Chris December 30, 2011 at 11:48
            read arg1 arg2 < <(echo $@)
    • medlefsen December 30, 2011 at 12:34

      Thanks for the feedback. That’s a neat trick, but it has a number of problems I wanted to solve with this solution.

      • 1. read doesn’t parse words like bash does, so if you call “myfunc 1\ 2 3” it will assign arg1 to 1 and arg2 to 2 instead of arg1 to “1 2” and arg2 to 3.
      • 2. It doesn’t check number of arguments or throw an error if they’re wrong.
      • 3. It doesn’t assign them to local variables so it will overwrite variables from other scopes.

      If you don’t care about #2, I would think local arg1=”$1″ arg2=”$2″ would be almost as terse but not have problems #1 or #3.

      My goal with is to provide more robust mechanisms for programming in bash. Writing terse bash code isn’t too hard, but to make it robust you have to add so much extra cruft most people don’t bother and so are either forced to deal with a brittle script or rewrite in another language.

  2. Malte Skoruppa September 19, 2014 at 22:13

    This was a very interesting read indeed! Very instructive 🙂 I also read your follow-up article. Really nice idea with the macro commands.

    However, while the @macro alias from your follow-up post works fine for me, the implementation described in this post does not. Specifically, when I execute a command with a comment at the end, the `$BASH_COMMAND` variable is apparently stripped of that comment:

    $ trap ‘echo “Running $BASH_COMMAND”‘ DEBUG
    $ echo 1 2 # 3
    Running echo 1 2
    1 2

    Hence, if I define a function `filter_macro_command()` as you did, and call it in the DEBUG trap with `”$BASH_COMMAND”` as argument, and then I execute a command with a comment, then the function `filter_macro_command()` does not get whatever appears after the `#` of that command:

    $ MACRO_ALIAS=’eval “$($MACRO_COMMAND)” #’
    $ alias @macro=”$MACRO_ALIAS”
    $ filter_macro_command() {
    local command=”Running $1″
    echo “$command”
    $ trap ‘filter_macro_command “$BASH_COMMAND”‘ DEBUG
    $ @macro args arg1 arg2
    Running eval “$($MACRO_COMMAND)”

    …and **not** `Running eval “$($MACRO_COMMAND)” #args arg1 args` as we would like. This is a problem, since we actually **want** the arguments after the `#` sign, they’re the whole point. Indeed, when I execute `@macro args arg1 arg2`, which expands to `eval “$($MACRO_COMMAND)” #args arg1 arg2`, already the comparison that you do in your function `filter_macro_command()`, namely,

    if [ “${command:0:$MACRO_ALIAS_LENGTH}” == “$MACRO_ALIAS” ]

    will fail, because what I get in the function as `$command` is only `eval “$($MACRO_COMMAND)`, without what follows. On the other hand, `”$MACRO_ALIAS”` is `eval “$($MACRO_COMMAND)” #`, with a ` #` at the end. So the strings are not equal. Even if we remedied that, we would still not get what we actually want, namely `MACRO_COMMAND=”${command:$MACRO_ALIAS_LENGTH}”`, because `$command` just doesn’t hold that information. In fact, `$command` is only `$MACRO_ALIAS_LENGTH` is 2 in length.

    Indeed, when I put your code into a script, (not forgetting to put `shopt -s expand_aliases` at the top — this cost me some time), and execute that script with `bash -x`, at the relevant point in time where the `@macro args arg1 arg2` is called and the comparison in the DEBUG trap happens, I see

    ++ ‘[‘ ‘eval “$($MACRO_COMMAND)”‘ == ‘eval “$($MACRO_COMMAND)” #’ ‘]’

    …which returns false, and so `$MACRO_COMMAND` stays empty, and the alias only ends up doing

    + eval ”

    …i.e., no macro code is executed.

    I realize this article is quite old. I just stumbled upon it. I was really fascinated and I played around with the commands while reading. But in the end it didn’t work, as explained above. Am I doing something wrong? Has bash changed? Is a comment following a given command not being put into `$BASH_COMMAND` variable when in a DEBUG trap any longer?

    As I said, the method described in your follow-up article works fine for me, and that was just as fascinating a read as this one! 🙂


    • Matt Edlefsen September 20, 2014 at 03:51

      Glad to hear there are others who get excited about these sorts of things :). The issue you’re having is due to a mistake on my part in the post. There is a typo in the example talking about adding the comment to the alias (I left off the ‘#’, which is the whole thing I was trying to demonstrate, whoops). I’ve fixed it in the post but to explain it a bit, the key is that the ‘#’ has to be inside the quoted argument to eval:

      MACRO_ALIAS=’eval “$($MACRO_COMMAND) #”‘

      I talk a bit more about this “eval comment” trick in the other post, but basically you get the best of both worlds. By having the ‘#’ as a quoted argument to eval, bash will treat the extra arguments like normal when it executes the line, so they will be included in BASH_COMMAND, but they won’t be passed in to the actual macro command because they are commented out inside of the eval.

      One thing that I realized while responding to this though is that I’m pretty sure that arguments with side effects will produce bugs in the current macro code because they will be evaluated twice. I’ll have to think it through a bit (I have some thoughts already), so I may have a new update.


      • Malte Skoruppa September 21, 2014 at 07:46

        Ah yes, I see! Of course. Thanks, that fixes it indeed. The solution in your follow-up post is better at any rate, but I was just wondering why this one wasn’t working for me.

        Btw, another thing that confused me while reproducing the ideas from this article was that you use a function `error` within your `args` function, which doesn’t exist in pure bash. I realize you probably implemented it in your project, however this post never declares or explains the `error` function. I replaced it with `echo 1>&2`, just thought I’d mention.

        Curious to see what you mean by “arguments with side-effects”, and a new update! 🙂

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: