Network Programming with Python

Tweet

Network Programming with Python contents 1 Overview 2 2 Our Example Client/Server Pair 2 2.1 Analysis of the Server Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.2 Analysis of the Client Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3 Role of the OS 6 3.1 Basic Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.2 How the OS Distinguishes Between Multiple Connections . . . . . . . . . . . . . . . . . . 7 4 The sendall() Function 7 5 More on the “Stream” Nature of TCP 8 5.1 Rememember, It’s Just One Big Byte Stream, Not “Lines” . . . . . . . . . . . . . . . . . . 8 5.2 The Wonderful makefile() Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 5.3 Getting the Tail End of the Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 6 Nonblocking Sockets 11 7 Advanced Methods of Polling 14 1 1 Overview The TCP/IP network protocol suite is the standard method for intermachine communication. Though originally integral only to the UNIX operating system, its usage spread to all OS types, and it is the basis of the entire Internet. This document will briefly introduce the subject of TCP/IP programming using the Python language. See http://heather.cs.ucdavis.edu/˜matloff/Networks/Intro/ NetIntro.pdf for a more detailed introduction to networks and TCP/IP. A TCP/IP application consists of a pair of programs, called a server and a client. If for example you use a Web browser to view www.yahoo.com, the browser is the client, and the Web server at Yahoo headquarters is the server. 2 Our Example Client/Server Pair As our main illustration of client/server programming in Python, we have modified a simple example in the Library Reference section of the Python documentation page, http://www.python.org/doc/ current/lib. Here is the server, tms.py: 1 # simple illustration client/server pair; client program sends a string 2 # to server, which echoes it back to the client (in multiple copies), 3 # and the latter prints to the screen 4 5 # this is the server 6 7 import socket 8 import sys 9 10 # create a socket 11 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 12 13 # associate the socket with a port 14 host = ’’ # can leave this blank on the server side 15 port = int(sys.argv[1]) 16 s.bind((host, port)) 17 18 # accept "call" from client 19 s.listen(1) 20 conn, addr = s.accept() 21 print ’client is at’, addr 22 23 # read string from client (assumed here to be so short that one call to 24 # recv() is enough), and make multiple copies (to show the need for the 25 # "while" loop on the client side) 26 27 data = conn.recv(1000000) 28 data = 10000 * data 29 30 # wait for the go-ahead signal from the keyboard (shows that recv() at 31 # the client will block until server sends) 32 z = raw_input() 33 34 # now send 35 conn.send(data) 2 36 37 # close the connection 38 conn.close() And here is the client, tmc.py: 1 # simple illustration client/server pair; client program sends a string 2 # to server, which echoes it back to the client (in multiple copies), 3 # and the latter prints to the screen 4 5 # this is the client 6 7 import socket 8 import sys 9 10 # create a socket 11 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 12 13 # connect to server 14 host = sys.argv[1] # server address 15 port = int(sys.argv[2]) # server port 16 s.connect((host, port)) 17 18 s.send(sys.argv[3]) # send test string 19 20 # read echo 21 i = 0 22 while(1): 23 data = s.recv(1000000) # read up to 1000000 bytes 24 i += 1 25 if (i yourself. 3 The two main points to note when you run the programs are that (a) the client will block until you provide some keyboard input at the server machine, and (b) the client will receive data from the server in rather random-sized chunks. 2.1 Analysis of the Server Program Now, let’s look at the server. Line 7: We import the socket class from Python’s library; this contains all the communication methods we need. Line 11: We create a socket. This is very much analogous to a file handle or file descriptor in applications involving files. Internally it is a pointer to information about our connection (not yet made) to an application on another machine. Again, at this point, it is merely a placeholder. We have not made any network actions yet. But our calls bind() etc. below will result in more and more information being added to the place pointed to by our socket. The two arguments state that we wish to the socket to be an Internet socket (socket.AF INET), and that it will use the TCP method of organizing data (socket.SOCK STREAM), rather than UDP (socket.SOCK DGRAM). Note that the constants used in the arguments are attributes of the module socket, so they are preceded by ‘socket.’; in C/C++, the analog is the #include file. Line 16: We invoke the socket class’ bind() method. Say for example we specify port 2000 on the command line when we run the server (obtained on Line 15). A port is merely an ID number, not anything physical. Since there may be many network connections on a machine at one time, we need a way to distinguish between them. The ID number serves this purpose. Port numbers 0-1023, the so-called well-known ports, are for standard services such as FTP (port 21), SSH (port 22) and HTTP (port 80).2 You cannot start a server at these ports unless you are acting with root privileges. When we call bind(), the operating system will first check to see whether port 2000 is already in use by some other process.3 If so, an exception will be raised, but otherwise the OS will reserve port 2000 for the server. What that means is that from now on, whenever TCP data reaches this machine and specifies port 2000, that data will be copied to our server program. Note that bind() takes a single argument consisting of a two-element tuple, rather than two scalar arguments. Line 19: The listen() method tells the OS that if any messages come in from the Internet specifying port 2000, then they should be considered to be requesting connection to this socket. The method’s argument tells the OS how many connection requests from remote clients to allow to be pending at any give time for port 2000. The argument 1 here tells the OS to allow only 1 pending connection request at a time. We only care about one connection in this application, so we set the argument to 1. If we had set it to, say 5 (which is common), the OS would allow one active connection for this port, and four other pending connections for it. If a fifth pending request were to come it, it would be rejected, with a “connection refused” error. 2On UNIX machines, a list of these is available in /etc/services. 3This could be another invocation of our server program, or a different program entirely. You could check this “by hand,” by running the UNIX netstat command (Windows has something similar), but it would be better to have your program do it, using a Python try/except construct. 4 That is about all listen() really does. We term this socket to be consider it a listening socket. That means its sole purpose is to accept connections with clients; it is usually not used for the actual transfer of data back and forth between clients and the server.4 Line 20: The accept() method tells the OS to wait for a connection request. It will block until a request comes in from a client at a remote machine.5 That will occur when the client executes a connect() call (Line 16 of tmc.py). When that call occurs, the OS at the client machine will assign that client an ephemeral port, which is a port number for the server to use when sending information to the client. The OS on the client machine sends a connection request to the server machine, informing the latter as to (a) the Internet address of the client machine and (b) the ephemeral port of the client. At that point, the connection has been established. The OS on the server machine sets up a new socket, termed a connected socket, which will be used in the server’s communication with the remote client. You might wonder why there are separate listening and connected sockets. Typically a server will simultaneously be connected to many clients. So it needs a separate socket for communication with each client. All this releases accept() from its blocking status, and it returns a two-element tuple. The first element of that tuple, assigned here to conn, is the connected socket. Again, this is what will be used to communicate with the client (e.g. on Line 35). The second item returned by accept() tells us who the client is, i.e. the Internet address of the client, in case we need to know that.6 Line 27: The recv() method reads data from the given socket. The argument states the maximum number of bytes we are willing to receive. This depends on how much memory we are w