Tips and Tricks

Reverse a Binary Stream Using Busybox

Today I had the need to reverse a binary stream using only bash and commonly-available command-line utilities. Not tac, sed, or rev, which are all line-oriented utilities that work best on ASCII data. I needed something that I could trust with binary data. This is what I came up with. Feel free to point out my weakness.

The first round was this:

reverse() {
    local i=0
    cat | xxd -c 1 | awk '{print $2}' | tac | \
        while read F; do
            printf "%06x: %s\n" $i $F; i=$((i+1))
        done | xxd -c 1 -r

I wasn't a huge fan of the while loop to prefix the lines with addresses for 'xxd -r'. The streams that I am using this for are only several kB max, so efficiency was not my first goal, but why not try to make it faster if you have the option? Some reading reveals that 'tac' is not available on every Unix platform. And 'xxd' is only available if you have vim installed. I swapped in 'hexdump' for 'xxd', but hexdump does not have a reverse, so I had to find a way to do that. This is where awk comes into play, doing and integer to character conversion for each line. This happens to run in about 6 times faster than the original version and uses stuff that even busybox has.

My final version was this:

reverse() {                                                                                                            
    cat | hexdump -v -e '/1 "%d\n"' | \
        sed -e '1!G;h;$!d' | \
        awk '{printf "%c", $0}'

You might use it like this:

$ reverse < file > file.reversed
# or
$ command -in -a | pipeline | reverse | process | reverse > some_output

Tame your bash history

I am a packrat, but I do like a bit of order. This makes maintaining my bash history difficult. There are some commands that I use frequently that seem to fill up my history file making it hard to keep some of the lesser used, yet very important commands in the history. Finally sick of the problem, I poured over the manpage for bash and found the section on HISTCONTROL. From the description there, I found that this along with HISTIGNORE, I can almost eliminate my problem of my bash history getting too full of stupid common commands.

I added this to my ~/.bash_profile:

export HISTIGNORE="&:ls:[bf]g:disown:cd:cd[ ]-:exit:^[ \t]*"
export HISTCONTROL=ignoredups:ignorespace:erasedups
export HISTFILESIZE=2000

Here is the snippet from the bash manual that corresponds to these controls:

              A colon-separated list of values controlling how commands are saved on the  history
              list.   If  the list of values includes ignorespace, lines which begin with a space
              character are not saved in the history list.  A value of  ignoredups  causes  lines
              matching  the  previous  history  entry  to not be saved.  A value of ignoreboth is
              shorthand for ignorespace and ignoredups.  A value of erasedups causes all previous
              lines  matching  the  current  line to be removed from the history list before that
              line is saved.  Any value not in the above list  is  ignored.   If  HISTCONTROL  is
              unset,  or  does  not include a valid value, all lines read by the shell parser are
              saved on the history list, subject to the value of HISTIGNORE.  The second and sub-
              sequent lines of a multi-line compound command are not tested, and are added to the
              history regardless of the value of HISTCONTROL.
              The maximum number of lines contained in the history file.  When this  variable  is
              assigned a value, the history file is truncated, if necessary, by removing the old-
              est entries, to contain no more than that number of lines.  The  default  value  is
              500.   The  history  file  is  also truncated to this size after writing it when an
              interactive shell exits.
              A colon-separated list of patterns used to decide which  command  lines  should  be
              saved  on  the history list.  Each pattern is anchored at the beginning of the line
              and must match the complete line (no implicit `*' is appended).   Each  pattern  is
              tested  against the line after the checks specified by HISTCONTROL are applied.  In
              addition to the normal shell pattern matching characters, `&' matches the  previous
              history  line.   `&'  may  be  escaped  using a backslash; the backslash is removed
              before attempting a match.  The second and subsequent lines of  a  multi-line  com-
              pound  command are not tested, and are added to the history regardless of the value
              of HISTIGNORE.