Retrace - Configurable, elegant retrying¶

Dealing with some unstable code? Be it a bad connection or a system that often falls over, retrace is here to help. Simple, easy, elegant and configurable method retrying with a nice clean API.

Don't manually fudge around with exception retrying again!

Retrace supports Python 2.7 and 3.3+.

Installation¶

Install from pip¶

Installation from pip is simple, like so:

pip install retrace

Vendoring¶

If you don't want to add a new dependency for such a small tool, you are in luck! Retrace is designed to be easily vendor-able! Simply head to the GitHub repo, grab the retrace.py file and include it in your project tree. Then, for example, say you add it under myproject.utils.retrace then you just need to use that import path in the examples below.

Usage Examples¶

Retry all exceptions¶

If you want to retry a function call on any exception you can use the decorator with no arguments. By default this will retry 5 times.

import retrace

@retrace.retry
def unstable():
    # ...

Retry on a specific exception type¶

Retry when an IOError is raised or any subclasses of it.

import retrace

@retrace.retry(on_exeption=IOError)
def unstable():
    # ...

Delaying between retries¶

If you want to delay between retries you can pass in a number which is equal to the number of seconds to delay between retrying. For example, wait a second between attempts

import retrace

@retrace.retry(interval=1)
def unstable():
    # ...

Limit the number of attempts¶

By default retrace will retry 5 times, if you want to change that, pass in a new limit.

import retrace

@retrace.retry(limit=10)
def unstable():
    # ...

Gradually delay more between attempts¶

Here is a neat trick - if you want to delay between each try, and have that increase with each attempt you can pass time.sleep in as your interval function! This will mean after the first attempt it will sleep for one second, then two seconds, three seconds etc.

This works because you can use functions as the delay, they must accept one argument which is the current retry number. So with time.sleep, the code sleeps for a the number of seconds equal to the attempt number.

import time
import retrace

@retrace.retry(interval=time.sleep)
def unstable():
    # ...

Validating and retrying based on the results¶

Sometimes you will have functions that don't error, but return bad values or you might need to call it until you get a good value. You can achieve this with validators.

In The following example, we have a function that returns a good value half of the time, we want to retry unless the result matches what we expected.

import random
import retrace

@retrace.retry(validator='WANTED VALUE')
def unstable():
    if random.random() > 0.5:
        return 'WANTED VALUE'
    else:
        return 'BAD VALUE'

Custom Retry Handling¶

limits and intervals¶

Okay, we touched on this, but let's just state it here clearly. The retry decorator takes two different arguments for controlling it's behaviour, limit and interval. These are similar, but different. Limit controls how many times we should retry before giving up. Interval controls how much delay happens between retry attempts.

Controlling the interval between retries¶

Customising the interval, the delay between retries, is a breeze, if you have some specific logic you want to implement.

For example, here is a exponential backoff. It will increase the delay between each attempt. To do this, a method needs to be passed that accepts one argument. The argument is the the current attempt integer.

import time
import random

import retrace

def exponential_backoff(attempt_number):
    # Increase the delay between attempts each time it fails. This function
    # sleeps for the number of seconds equal to the attempt number plus a random
    # percentage of that time again. So, for example, after the first failure it
    # sleeps between 1 and 2 seconds, then between 2 and 4, then 3 and 6 etc.
    time.sleep(attempt_number + (random.random() * attempt_number))

@retrace.retry(interval=exponential_backoff)
def unstable():
    # ...

Limiting the number of reties¶

Similarly, the same approach can be used to limit the number of retries. In this artificial example, the retry limit is 10 in the afternoon, but only 5 in them morning.

import datetime
import retrace

def try_more_in_the_afternoon(attempt_number):

    now = datetime.datetime.now()
    if now.hour < 12 and attempt_number > 5:
        raise retrace.LimitReached()
    elif attempt_number > 10:
        raise retrace.LimitReached()

@retrace.retry(limit=try_more_in_the_afternoon)
def unstable():
    # ...

Custom Validators¶

Validators are used to verify that the result from the function passes a check.

If it isn't a callable, it can be any object that is then compared with the result. Check that the function returns the value "EXPECTED".

@retrace.retry(validator="EXPECTED")
def unstable():
    # ...

Provide a custom validator that checks for type, rather than a full match.

def validate_string(value):
    return isinstance(value, str)

@retrace.retry(validator=validate_string)
def unstable():
    # ...