Friday, June 8, 2012

Bite-sized Twisted: Deferreds

In this post, we'll examine the Deferred.

The check's in the mail

A Deferred is kind of like an IOU. It's a promise. It's a forecast of things to come. A forerunner of sorts. A title bearer or trumpeteer.

Enough analogies: it's a class you can instantiate. Like this:

>>> from twisted.internet.defer import Deferred
>>> iou = Deferred()

If you are given a Deferred, use addCallback to register a function to handle the value when it arrives.

>>> def spend(value):
... print value + ' spent!'
>>> iou.addCallback(spend)
<Deferred at 0xb764bcccL>

If you are the one giving out the Deferred and are responsible for fulfilling the promise, use callback to send the value to the registered callback functions:

>>> iou.callback('$20.00')
$20.00 spent!

Callbacks can be chained

Multiple callbacks can be added to a Deferred. Each callback is not called with the original value. Each callback is called with the return value of the prior callback. For example:

from twisted.internet.defer import Deferred

def mouth(what):
print 'I am the mouth, I got ' + what
return 'chewed up ' + what

def stomach(what):
print 'I am the stomach, I got ' + what

d = Deferred()
d.callback('jelly beans')
I am the mouth, I got jelly beans
I am the stomach, I got chewed up jelly beans

Extra arguments

If you want to pass additional arguments to a callback, add them in addCallback:

from twisted.internet.defer import Deferred

def callback(value, arg, keyword_arg='foo'):
print 'value: %r' % value
print 'arg: %r' % arg
print 'keyword_arg: %r' % keyword_arg

d = Deferred()
d.addCallback(callback, 'apple', keyword_arg='banana')
value: 'gorilla'
arg: 'apple'
keyword_arg: 'banana'

Read the source

A Deferred works approximately like this PoorMansDeferred:

class PoorMansDeferred:

def __init__(self):
self.callbacks = []

def addCallback(self, function):

def callback(self, value):
for callback in self.callbacks:
value = callback(value)

def government(value):
portion = value / 2
print 'Government got %s' % portion
return value - portion

def vinny(value):
portion = value / 2
print 'Vinny got %s' % portion
return value - portion

def me(value):
print 'I got %s' % value

d = PoorMansDeferred()
Government got 15
Vinny got 7
I got 8

Vinny should study up on integer division. You should probably study up on self defense.

The real Deferred implementation is not that long and worth reading. It includes error handling, chaining, extra arguments, canceling, etc... Read the source here.

There's no reactor

Please notice a few things about the above examples:

  1. The reactor was not involved
  2. There was no call
  3. reactor wasn't used
  4. A Deferred does not need the reactor

In short, a Deferred does not depend on the reactor. It is made more useful by the reactor but it works independently from it.

Many of Twisted's APIs either return or work with Deferreds. It is time well-spent learning how they work.

Defer explosions!

Though a Deferred does not require the reactor, it can be used with it. Let's add an ignite method to our Bomb that returns a Deferred:

from twisted.internet import reactor
from twisted.internet.defer import Deferred

class Bomb:

def __init__(self, fuse, size):
self.fuse = fuse
self.size = size

def ignite(self):
d = Deferred()
reactor.callLater(self.fuse, d.callback, self)
return d

Then we can modify the Protocol we wrote a few posts ago to use the Deferred code:

from twisted.internet.protocol import Protocol
from twisted.internet import reactor, stdio

from import Bomb

def explode(bomb):
print 'BOOM! (magnitude %s)' % bomb.size

class BombProtocol(Protocol):

def dataReceived(self, data):
fuse = int(data)
bomb = Bomb(fuse, fuse)

BOOM! (magnitude 3)
BOOM! (magnitude 1)
BOOM! (magnitude 2)
BOOM! (magnitude 4)
BOOM! (magnitude 5)
BOOM! (magnitude 5)
BOOM! (magnitude 10)

Errors next time

Error handling deserves a post of its own. In fact, that's probably what the next post will be about. Until then, chew on this:

from twisted.internet.defer import Deferred

def toInfinity(value):
return value / 0

def errorHandler(error):
print 'There was an error: %s' % error.value

d = Deferred()

print "The error didn't kill the program"
There was an error: integer division or modulo by zero
The error didn't kill the program

Also consider reading Twisted's official, very thorough guide.

No comments:

Post a Comment