Description
Overview
You will design, develop and test Nutella, a light-weight, peer-to-peer streaming movie system. Nutella uses multicast for efficient decentralized queries and text-based movies for lightweight videos that can be easily created, transferred and decoded, as well as constructed by hand.
Search
Nutella uses a non-centralized system for movie searching. Users input a text-based query, which the local Nutella client multicasts to other Nutella peers. Any Nutella peer with a match on the content responds. At that time, the local Nutella client contacts a remote peer that responded and requests the movie. The remote peer then streams the movie to the local Nutella client.
From the viewpoint of an active Nutella client looking for a movie, the steps of the search process are as follows:
- Get query (i.e., movie name) from user.
- Multicast query to well-known address and port.
- Listen at a different well-known multicast address and port for peers with match to query to respond. Any responses provide IP address and port for contact to initiate streaming.
- Contact (unicast) the remote peer at IP and port, requesting streaming of the movie.
- If there is no response to an initial multicast query, assume the movie is not available on any Nutella peer. The local client then reports an appropriate message to the user.
- Once the movie is done or a query has failed, the client prompts the user for an additional movie query (i.e., return to step 1).
From the viewpoint of a passive Nutella client (the Nutella “server” process), the steps are:
- Listen on well-known multicast address and port for query.
- If an incoming query matches a movie in the local movie list, multicast a response back, providing IP address and port for streaming.
- If a streaming request comes to a IP address and port for a movie in the local movie list, (unicast) stream the movie.
Stream
When contacted, a Nutella peer streams a requested movie. Whereas downloading and playing a movie means the entire movie file is transferred to a client before it is played out, streaming has a client play out the movie while it receives the frames over the network. So, when streaming, the server sends the movie frames at the frame playout rate.
Nutella streams at no more than 10 frames per second. Streaming at 1-3 frames per second is fine, too (and it means the movie needs fewer frames for the same amount of viewing time :-)). In Linux C/C++, fine-grained delay between sending each frame can be achieved by using nanosleep() (see the man pages for details). While the actual frame rate is not particularly important for this assignment, the idea is to get an “animated” effect.
You can assume a Nutella peer only sends one movie at a time. Thus, while sending a stream, the Nutella peer can either ignore any subsequent multicast searches for movies, or ignore any additional requests for streaming movies. Adding the capability of simultaneously streaming more than one movie (to different Nutella peers) is optional.
A user should be able to play a movie (i.e., request a stream from a different Nutella peer) while sending a movie to someone else. Note, this requirement can be handled by actually having a multi-process/threaded Nutella client or by having the Nutella player being a separate program/process from the Nutella streamer.
Movies
Each Nutella peer has some movies they are willing to share by streaming to other Nutella peers (you can assume some tit-for-tat reasoning for this, not specified in this assignment). For convenience, the list of movies a peer has should be specified outside of the compiled code (i.e., not hard coded). Options include: 1) kept in a configuration file (say, “.nutella”) which contains information on the file location of each movie, or refers to a directory with the movies; 2) an environment variable that refers to the movie directory (e.g., NUTELLA=/home/claypool/nutella); or 3) specifying the movie directory via a command line argument, where every file in that directory is a movie.
The movie files themselves are human-readable text containing the animation frames and a frame separator. The format of a movie file is:
picture 1, arbitrarily long end picture 2, arbitrarily long end ...
For example, the start of the walk.txt file is:
o
|
|
end
o
|
/ \
end
o
|
|
end
o
|
/ \
end
o
|
|
end
...
Earlier creative creations include:
- Star Wars, by Christopher Tashjian
- Pong, by Dennis “spOOny” Mattoon and Jeff “killer” Cilley
- Clerks Trailer, by Glenn Barnett, Keith Berard, and Darren Ranalli
- Tiger Cub
- Walk (the full walk movie, above)
- The Matrix (good for a stress test – see the README for details and this clip for how it might look in your player).
For inspiration, there have been some amazing, hand-made creations of ASCII movies, such as Star Wars ASCIIMation. There is also a community that has worked on converting and playing standard image video as ASCII video, such as at ASCIImeo.
Feel free to be creative with the movies you make. If you stick to the movie format given above, others in the class can share your creations. Have fun!
Multicast
Nutella uses IP multicast sockets for query communication. IP multicast allows a single message sent to potentially reach multiple receivers. An end-point of a multicast socket is the same as that of a unicast socket, specified by an IP address and a port. The valid range of multicast addresses are: 239.0.0.1 to 239.255.255.255. The valid range for ports are integers above the ephemeral port range (so, 5000+).
To simplify the use of multicast sockets, you may use some code wrappers (think of them as middleware) written especially for projects like this. The source file containing these wrappers is msock.c and a header file containing prototype definitions is msock.h. The routines provided in msock.h are:
/* msockcreate -- Create socket from which to read. return socket descriptor if ok, -1 if not ok. */ int msockcreate(int type, char *address, int port); /* msockdestroy -- Destroy socket by closing. return socket descriptor if ok, -1 if not ok. */ int msockdestroy(int sock); /* msend -- Send multicast message to given address. return number of bytes sent, -1 if error. */ int msend(int sock, char *message, int len); /* mrecv -- Receive message on given mcast address. Will block. return bytes received, -1 if error. */ int mrecv(int sock, char *message, int max_len);
To use the msock wrappers, you must compile the file msock.c yourself. Do this by downloading both the msock.c and msock.h files to your directory and using the command gcc -c msock.c. Better still, use a Makefile.
See mcast.tar for an example of how these wrappers might be used (extract with tar xvf and then type “make“).
You may only need one multicast address for your Nutella system, since both queries and responses can be to the same address. However, conceptually it may be cleaner to have a multicast address for queries and a multicast address for responses. For example:
USE IP PORT Query 239.xxx.xxx.1 7000 Response 239.xxx.xxx.2 7001
Warning! Be careful of errant multicast code. Below is an example of what might happen due to multicast program that sends too much data (this one due to my own errant code :-().
From: "Charles R. Anderson" <cra@WPI.EDU> To: system@cs.wpi.edu, mvoorhis@WPI.EDU, jbanning@WPI.EDU Cc: netops@WPI.EDU Subject: cs.wpi.edu flooding network cs.wpi.edu is flooding the network with multicast traffic, causing network outages in Fuller Labs. I have disabled the port. -- Charles R. Anderson cra@wpi.edu Network Engineer (508) 831-6110 Computing and Communications Center X2220 on-campus Worcester Polytechnic Institute Fax (508) 831-5483
In general, you should be ok if you only send one multicast packet request for each query the user types in but be sure to debug carefully before you run multicast code and monitor your code performance as your do so.
Hints
Important! The network port for the streaming server can have default values, but must be also configurable via a command line option (or something similar).
Below is an example of a possible peer-to-peer interaction, with one node acting as a server and the second a client:
SERVER PEER CLIENT PEER
Server started. Client started.
Listening... Enter movie name: Thor
Sending search request
Waiting for response ...
Received search request for "Thor"
Movie not here, so no response
Listening...
Timeout, movie not found
Enter movie name: Walk
Sending search request
Received search request "Walk"
Movie here
Sending IP 130.215.36.67, port 7832 to client
Listening...
Received IP and port from server
Sending request to stream...
Connection request received.
Streaming movie "Walk" to client... O
Movie done. Sending done message. |
Listening... / \ ...
Received done message.
Enter movie name:
If you want to make a player that reads from a movie file, your code would:
Open file
While file not empty
Read everything until "end", writing to stdout
Discard "end"
Pause appropriate amount
Clear screen
end while
Close file
Making such a player can be good for trying out movies before offering them for streaming or as part of implementing a multicast client player.
For your streaming socket code, use the information on sockets from your project 2, and the class slides on sockets.
Remember, if a process reads from a socket that has no data, it blocks until there is data. You can poll a non-blocking socket for data (use MSG_DONTWAIT with recvfrom() or use fcntl() on the socket). Or, you can use select() to see if any socket has information. See select.c for a C/C++ sample and do a man 2 select for more information. Or, the ioctl() call can be used to see the number of bytes available in a socket before reading using the FIONREAD parameter as the second argument (e.g., ioctl(sock, FIONREAD, &bytes)). Again, see the man pages for more information.
To clear the screen between frames, check out this snippet of code (works on most terminals).
Experiments
After designing, implementing and testing your Nutella streaming system, you conduct some experiments to determine the network data rates required by a working system and predict the number of users supported.
First, determine the data rate required for a movie query-response protocol and streaming of a typical text-based movie. This can be done by measuring network load using a tool such as Wireshark. But better, is to determine network load based on analysis of your code and the movie content. In other words, you know the sizes of your queries and responses, as well as the sizes of videos and their frame rates. You can use this information to determine network data requirements. Make sure to consider lower-layer header sizes!
Assuming users continually watch movies (query-watch-repeat), compute how many users Nutella could support on a typical LAN. Be sure to explicitly state your assumptions about movies (length, frame rate, frame sizes) as well as assumptions about the LAN. Provide references, where appropriate.
Assuming Nutella did multicast streaming in addition to multicast queries, compute how many users Nutella could support on a typical LAN.
Repeat the same analysis for actual (non-text-based) movies (e.g., NetFlix) on a typical U.S. residential broadband network. Be sure to provide data on movie data rates, with references, as appropriate.
When complete, prepare a brief (1-2 page) write-up with the following sections:
- Design – describe how you determined network data rates, both for Nutella and for your non-text-based movies.
- Results and Analysis – show your computations to determine the maximum user capacities for the indicated networks. Presenting the results as graphs or in tables with any final conclusions.
Hand In
Important! Before you are ready to submit, make sure your nutella system works on the CCC machines (e.g., one peer on cccwork2.wpi.edu and another on cccwork3.wpi.edu and a third on cccwork4.wpi.edu). Regardless of where you develop your code, you are responsible for making sure your projects run on these machines!
You must hand in the following:
- A source code package:
- All your source code.
- Any other support files, including
.hfiles. - Any appropriate Makefile for building your program(s).
- Movie files that work with your Nutella system.
- A
README.txtthat contains information required to run your system.
Also, be sure to have your experiment writeup in a clearly labeled file. The document must be in pdf format.
Before submitting, “clean” your code (i.e., do a “make clean”) removing the binaries (executables and .o files).
Use zip to archive your files. For example:
mkdir lastname-proj3 cp * lastname-proj3 /* copy all the files you want to submit */ zip -r proj3-lastname.zip lastname-proj3 /* package and compress */
To submit your assignment (proj3-lastname.zip), log into the Instruct Assist website:
Use your WPI username and password for access. Visit:
Tools → File Submission
Select “Project 3” from the dropdown and then “Browse” and select your assignment (proj3-lastname.zip).
Make sure to hit “Upload File” after selecting it!
If successful, you should see a line similar to:
Creator Upload Time File Name Size Status Removal Claypool 2016-02-10 22:40:03 proj3-claypool.zip 1508 KB On Time Delete
Grading
A grading guide provides a point breakdown for the individual project components. A more general rubric follows:
90-100 The streaming system meets all specified requirements. Nutella p2p streaming fully implemented, with clients able to query movie, server processes responding, clients playing and the system being robust overall. Proper use of multicast and socket communication. All documentation included. Experiment report is clear, complete and includes all needed information.
80-89 The streaming system meets most specified requirements. Nutella p2p streaming implemented, with queries, response and movie playing, but occasional bugs and/or documentation is missing. Experiment report is complete and includes all needed information.
70-79 The streaming system meets some specified requirements. Nutella p2p streaming framework setup, but query or response not fully functioning and playout of movie may not commence properly. Occasional bugs, but system still compiles and runs. Documentation may be missing. Experiment report is incomplete and/or is missing some needed information.
60-69 The streaming system fails to meet many specified requirements. Nutella p2p streaming not functional. Server and client pieces compile and run but are not fully integrated. Socket communication works for some aspects of the system but not completely. May be some bugs. Documentation may be missing. Experiment report is incomplete and is missing some needed information.
0-59 The streaming system does not meet most specified requirements. Nutella p2p system not implemented in that most of the system cannot be started, significant problems or crashes arise during even short runs. Documentation missing and/or problems in compilation. Experiment report is mostly incomplete or is missing.
