Bite-sized Twisted: Deferred Errors

In this post, I'll talk about the Deferred callback chain, including error handling.

Exceptions

Boring, run-of-the-mill Exception:

def mouth(food):
    raise Exception("Don't eat %s" % food)
mouth('cannon balls')

err1.py

Traceback (most recent call last):
  File "err1.py", line 3, in <module>
    mouth('cannon balls')
  File "err1.py", line 2, in mouth
    raise Exception("Don't eat %s" % food)
Exception: Don't eat cannon balls

python err1.py

Exception raised inside a callback:

from twisted.internet.defer import Deferred

def mouth(food):
    raise Exception("Don't eat %s" % food)

d = Deferred()
d.addCallback(mouth)
d.callback('cannon balls')

err2.py

Unhandled error in Deferred:
Unhandled Error
Traceback (most recent call last):
  File "err2.py", line 8, in <module>
    d.callback('cannon balls')
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 361, in callback
    self._startRunCallbacks(result)
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 455, in _startRunCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 542, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "err2.py", line 4, in mouth
    raise Exception("Don't eat %s" % food)
exceptions.Exception: Don't eat cannon balls

python err2.py

It complains of an Unhandled error in Deferred. To handle it, an error callback (errback for short) is needed.

from twisted.internet.defer import Deferred

def mouth(food):
    raise Exception("Don't eat %s" % food)

def badfood(food):
    print "Bad food happened, but I'm okay"

d = Deferred()
d.addCallback(mouth)
d.addErrback(badfood)
d.callback('cannon balls')

err3.py

Bad food happened, but I'm okay

python err3.py

Errbacks are only called to handle errors. So, in this case, the errback is not called since there's no error:

from twisted.internet.defer import Deferred

def mouth(food):
    print "I ate %s" % food

def badfood(food):
    print "Bad food happened, but I'm okay"

d = Deferred()
d.addCallback(mouth)
d.addErrback(badfood)
d.callback('piano keys')

err4.py

I ate piano keys

python err4.py

Garden of forking paths

Callbacks will always result in either:

1. a value or
2. an error

If a value is returned, the next callback in line will get the value. If an error happens, the next errback in line will get the error.

Errbacks are the same -- they will always result in either a value or an error. And the result will be handled either by the next callback or the next errback. This snippet illustrates that:

from twisted.internet.defer import Deferred

def value(result, msg):
    print 'value %s' % msg
    return 'value'

def error(result, msg):
    print 'error %s' % msg
    raise Exception(msg)

d = Deferred()
d.addCallback(value, 'callback level 1')

d.addErrback(value, 'errback level 2')

d.addCallback(error, 'callback level 3')
d.addCallback(value, 'callback level 4')
d.addCallback(error, 'callback level 5')
d.addCallback(value, 'callback level 6')

d.addErrback(value, 'errback level 7')
d.addErrback(error, 'errback level 8')
d.addErrback(value, 'errback level 9')
d.addErrback(error, 'errback level 10')

d.addCallback(value, 'callback level 11')

d.callback('begin')

twopaths1.py

value callback level 1
error callback level 3
value errback level 7
value callback level 11

python twopaths1.py

What happened?

1. The Deferred is called back with 'begin'.
2. The first callback is called with 'begin' and returns 'value'.
3. Since there is no error, the errback on level 2 is skipped and the next callback (level 3) is called.
4. The callback on level 3 ends in error, so all the callbacks are skipped until the next errback (level 7) is found.
5. The errback on level 7 returns 'value', so all the errbacks are skipped until the next callback (level 11) is found.

Here's a diagram of what happened. Blue squares are callbacks. Red squares are errbacks. Time flows to the right:

Trapping specific exceptions

Error handlers can be made to only deal with specific exceptions using the trap() method of the passed-in error (the first argument). If the actual error doesn't match the kind being trapped, the next errback in line will be called with the error.

from twisted.internet.defer import Deferred

def add3(result):
    return 3 + result

def catchAttributeError(error):
    error.trap(AttributeError)
    print 'AttributeError happened'
    return 'Bad attribute'
    
def catchTypeError(error):
    error.trap(TypeError)
    print 'TypeError happened'
    return 'Bad type of value'

def printResult(result):
    print 'Result: %s' % result

d = Deferred()
d.addCallback(add3)
d.addErrback(catchAttributeError)
d.addErrback(catchTypeError)
d.addCallback(printResult)
d.callback('foo')

trap1.py

TypeError happened
Result: Bad type of value

python trap1.py

Without the .trap() calls in trap1.py, catchAttributeError would have handled the error instead of catchTypeError. Also, notice that catchAttributeError is called first, but since it's only trapping AttributeErrors, the error is passed to the next errback, catchTypeError.

Why can you call .trap() on the thing passed to errbacks? Short answer: it's a Failure object. Long answer: see the "Read more" section below.

Read more

Believe it or not, there's still more you can do with Deferreds. As features are needed for the Bomberman clone, I'll introduce them. In the meantime, read the very thorough, official Deferred Reference if you're interested.

Next time: Bombs away!

Using the things I've posted about so far, I plan for the next Bite-sized Twisted post to include the code for a playable version of boom. Lest you expect more than I'm planning: it will be single player, may or may not display a textual representation of the board, and will require pressing enter after every move. Future posts will improve the game further.