Bite-sized Twisted:Networking Triforce

This post begins a series of posts about networking -- an area where Twisted shines.

Network Input/Output == Bytes

In the second Bite-sized Twisted post we accepted input (bytes) from stdin asynchronously. In this post, we'll accept input (bytes) over the network. I'm intentionally emphasizing bytes (bytes). It's all just bytes.

It might be worth reviewing that second post, so the concepts are fresh.

Parts of the Triforce

In Twisted, dealing with networking (moving bytes) usually involves three things:

1. Protocol - determines fate of bytes
2. Transport - carries bytes between protocol and remote
3. Factory - creates protocols

Protocol

A Protocol's job is to do application-specific things with bytes. Code for deciding what bytes should be sent back across the wire is also in the Protocol. When it decides to send bytes across the wire, it will invoke the Transport's write method.

Here's an ObstinateProtocol which sends the bytes 'no\r\n' in response to all data received:

from twisted.internet.protocol import Protocol

class ObstinateProtocol(Protocol):

    def dataReceived(self, data):
        self.transport.write('no\r\n')

protocol1.py

Transport

As seen above, the Transport provides an interface for interacting with the remote side. It has a write method which sends bytes to the remote side. There is generally one Transport per Protocol. You can access the transport through a Protocol's transport attribute.

Factory

A Factory's job is to make a Protocol for a connection. In general, there is one Factory for many Protocol instances.

Standard Factories determine what Protocol to make based on their protocol attribute. This snippet shows a Factory that will make ObstinateProtocol protocols:

from twisted.internet.protocol import Factory
factory = Factory()
factory.protocol = ObstinateProtocol

factory1.py

Notice that factory.protocol is set to the class ObstinateProtocol and not an instance ObstinateProtocol().

Magic glue

One other component is needed for the Networking Triforce to work: magic glue. Here's a complete example that serves the ObstinateProtocol using TCP on port 8900 (magic glue is highlighted):

from twisted.internet.protocol import Factory, Protocol
from twisted.internet import reactor
from twisted.internet.endpoints import TCP4ServerEndpoint

class ObstinateProtocol(Protocol):

    def dataReceived(self, data):
        self.transport.write('no\n')

factory = Factory()
factory.protocol = ObstinateProtocol
endpoint = TCP4ServerEndpoint(reactor, 8900)
endpoint.listen(factory)
reactor.run()

glue1.py

Start glue1.py in one shell, then open another shell and start telnet with this command:

telnet 127.0.0.1 8900

You've just connected to the obstinate server. Type something then press enter; you should see 'no' written back. For example:

$ telnet 127.0.0.1 8900
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
Are you there?
no
Are you lying?
no
Are you sure?
no

If you have another computer nearby, start glue1.py here, then start telnet on the other computer, changing 127.0.0.1 to this computer's IP (you can even do this while the other telnet session is still going).

A thousand words to explain the magic glue

I don't like magical code. So, here's the pertinent code for hooking up the Triforce (from twisted/internet/tcp.py). First, the factory is asked to build a protocol:

protocol = self.factory.buildProtocol(self._buildAddr(addr))

portion of twisted/internet/tcp.py

Factory.buildProtocol looks like this:

class Factory:

    def buildProtocol(self, addr):
        p = self.protocol()
        p.factory = self
        return p

portion of twisted/internet/protocol.py

Then a transport is created:

transport = self.transport(skt, protocol, addr, self, s, self.reactor)

portion of twisted/internet/tcp.py

The transport is connected to the protocol:

protocol.makeConnection(transport)

portion of twisted/internet/tcp.py

which looks like this:

class BaseProtocol:

    def makeConnection(self, transport):
        self.connected = 1
        self.transport = transport

portion of twisted/internet/protocol.py

Here's a thousand words describing it:

Toward a better boom

Here's a Protocol and Factory that will let you telnet in to play.

from twisted.internet.protocol import Protocol, Factory
import string


from boom.game import Pawn, YoureDead, IllegalMove


class SimpleProtocol(Protocol):

    num = 0

    move_mapping = {
        'w': 'u',
        'a': 'l',
        'd': 'r',
        's': 'd',
    }

    def connectionMade(self):
        self.factory.protocols.append(self)
        name = string.uppercase[self.num % len(string.uppercase)]
        SimpleProtocol.num += 1
        self.pawn = Pawn(name)
        self.factory.board.insertPawn((0,0), self.pawn)


    def connectionLost(self, reason):
        self.factory.protocols.remove(self)
        self.factory.board.pawns.remove(self.pawn)


    def dataReceived(self, data):
        for k in data:
            if k in self.move_mapping:
                try:
                    self.pawn.move(self.move_mapping[k])
                except YoureDead, e:
                    pass
                except IllegalMove, e:
                    pass
            elif k == 'e':
                try:
                    self.pawn.dropBomb()
                except YoureDead, e:
                    pass
                except IllegalMove, e:
                    pass



class SimpleFactory(Factory):
    """
    A factory for making L{SimpleProtocol}s
    
    @ivar board: the game board on which I'll be playing
    @ivar protocols: A list of L{SimpleProtocol} instances currently
        in use.
    """
    
    protocol = SimpleProtocol
    
    
    def __init__(self, board):
        self.board = board
        self.protocols = []

boom/protocol.py

The code is available from GitHub in the tx-telnet-1 branch. Do this:

git clone -b tx-telnet-1 git://github.com/iffy/boom boom.git
cd boom.git
PYTHONPATH=. python run.py

Or if you don't have Git:

wget https://github.com/iffy/boom/tarball/tx-telnet-1
tar xf tx-telnet-1 
cd iffy-boom-f5c6433/
PYTHONPATH=. python run.py

Then connect with:

telnet 127.0.0.1 8900

Next time, we'll remove the need to press enter after each move. In the meantime, have a look at Twisted's guide to writing servers for more information.