Be more intelligent with your `sleep`--or-- how to overengineer your scripts

The universal solution to waiting for something to be ready in shell scripting is the `sleep` command.

Here, we're waiting for a dir to be created (say from a `yum install httpd` going on in another terminal), so we can `ls` the contents, perhaps as part of a script that configures httpd.

sleep 6
ls /etc/init.d/httpd

But is there a better way? What if the directory exists almost immediately? You've wasted nearly 6 seconds unnecessarily, which if you do this a lot in your script, adds a bunch of time.

We need a way to continuously query if the resource exists. And it needs to be with a command that sets an error exit code if the resource is not found. Let's use `stat` to do this.

Leveraging the fact that `stat` will set a non-zero exit code on failure

while true; do
  stat /etc/init.d/httpd
  if [ $? -eq 0 ]; then # check if the stat was successful
    break
  fi
done
ls /etc/init.d/httpd

'true' is a command, whose output is nothing and whose exit status is 0. In terms of performance we'd probably like not to call a binary, but `true` is a shell built-in. The no-op operator `:` accomplishes the same thing, ex: `while :; do`.

We can simplify the if statement by using the && operator which executes the following command break, if stat exits without error (sets a 0 status)

while true; do
  stat /etc/init.d/httpd && break
done
ls /etc/init.d/httpd

Instead of stat, we can use the `test` command (aliased as `[`). Here we check for a file using -f.

while true; do
  if [ -f /etc/init.d/httpd ]; then break; fi
done
ls /etc/init.d/httpd

However, in these examples, if the file never exists, the loop will never exit.

So, instead we can define a timeout and a shorter `sleep` interval, and a counter (i) to track the iterations:

i=0
while [ "$i" -lt 6 ]; do
  if [ -f /etc/init.d/httpd ]; then break; fi
  sleep 1
  (( i++ )) # built-in arithmetic
done
ls /etc/init.d/httpd

Quote i for protection. Use the break keyword to escape the while loop.

Alternatively, use Bash's built-in arithmetic:
i=0
while (( i < 6 )); do
  if [ -f /etc/init.d/httpd ]; then break; fi
  sleep 1
  (( i++ ))
done
ls /etc/init.d/httpd

This is a good start. Calling test (even as a builtin) in an infinite loop is also wasteful. If you are in bash, you can use the [[ keyword, which has the added benefit of protecting you against unquoted variables in a comparison.

i=0
while (( i < 6 )); do
  if [[ -f /etc/init.d/httpd ]]; then break; fi

  sleep 1
  (( i++ ))
done
ls /etc/init.d/httpd

There is a bug here. ls will run no matter whether the file was found or not.

So now we exit the loop, but how do we notify the caller that it failed? The `break` statement does not return a non-zero. As far as the shell is concerned, the loop completed. We can use a RETVAL variable that we set explicitly to 0 when it succeeds, and 143 to mean "file not found".

i=0
while (( i < 6 )); do
  if [[ -f "$path" ]];then
      RETVAL=0
  else
      RETVAL=143
  fi
    
  [[ "$RETVAL" -eq 0 ]] && break

  sleep 1
  (( i++ ))
done
[[ "$RETVAL" -eq 0 ]] && ls /etc/init.d/httpd || echo "ERR: file never created"

I quote variables in the [[ ]] here even though it's not actually required.

Which works fine, but we can simplify by simply unsetting RETVAL if there's a success. The test is expressed with [[ -z $xxx ]]. Also it is best practice to send error messages to STDERR (file descriptor --or fd-- #2), using >&2.

i=0
while (( i < 6 )); do
  if [[ -f "$path" ]];then
    unset RETVAL
  else
    RETVAL=143
  fi
    
  [[ -z "$RETVAL" ]] && break

  sleep 1
  (( i++ ))
done
[[ -z "$RETVAL" ]] && ls /etc/init.d/httpd || echo "ERR: file never created" >&2

Now, let's abstract the variables, and set an exit status.

i=0; path="/etc/init.d/httpd"; timeout=6
while (( i < $timeout )); do
  if [[ -f "$path" ]]; then
    unset RETVAL
  else
    RETVAL=143
  fi
    
  [[ -z "$RETVAL" ]] && break

  sleep 1
  (( i++ ))
done
[[ -z "$RETVAL" ]] && ls "$path" || (echo "ERR: file never created" >&2; exit $RETVAL )

Notice that the `exit` is called from a subshell, as if it is called in the current context, it will exit your interactive shell which is annoying and undesirable. There is no way to use return in this context since this is not yet in a function.

Testing with a bogus file.

$ path=/etc/bogusfile

$ while (( i < $timeout )); do

> if [[ -f "$path" ]]; then
>   unset RETVAL
> else
>   RETVAL=143
> fi

> [[ -z "$RETVAL" ]] && break

> sleep 1

> (( i++ ))

> done

$ [[ -z "$RETVAL" ]] && ls "$path" || (echo "ERR: file never created"; exit $RETVAL)

ERR: file never created

$ echo $?

143

Now put it in a reusable function!

_testforfile () {
  local i=0
  local timeout="$1"
  local path="$2"

  while (( i < $timeout )); do
    if [[ -f "$path" ]];then
      unset RETVAL
    else
      local RETVAL=143
    fi

    [[ -z "$RETVAL" ]] && break

    sleep 1
    (( i++ ))
  done
  [[ -z "$RETVAL" ]] && ls "$path" || (echo "ERR: file never created" >&2; return "$RETVAL" )
}

Note the use of the `local` keyword so we don't have our custom variables pollute the invoking environment, and the change of exit to return. This function accepts two parameters as input. Call it like so:

_testforfile 6 /etc/init.d/httpd

Now, we can enhance this by making sure that at least the first argument is numeric, and even set that to a default value of 5 if it was not provided.

_testforfile () {
  local i=0
  local timeout="${2:-5}"
  local path="$1"

  local re='^[0-9]+$'
  if ! [[ $timeout =~ $re ]]; then
    echo "ERR: Timeout was not a number" >&2
    return 1
  fi

  while (( i < $timeout )); do
    if [[ -f "$path" ]];then
      unset RETVAL
    else
      local RETVAL=143
    fi
    
    [[ -z "$RETVAL" ]] && break
    
    sleep 1
    (( i++ ))
  done
  [[ -z "$RETVAL" ]] && ls "$path" || (echo "ERR: file never created" >&2; return "$RETVAL" )
}

re is a regular expression used in conjunction with the =~ operator.

In action:
$ _testforfile 47d /etc/hosts
ERR: Timeout was not a number

And maybe some tests in a future post.