Internet Technologies and Applications (ITS 413)

Assignment 2

Updates: 11 Feb - deadline extended | 4 Feb - iperf server and network.

In this assignment you will perform real network performance tests. The objective is that you see how Internet technologies/protocols may impact on the performance of end-user applications.

This is a group assignment - use the same groups as assignment 1. There are two sets of tasks.

Task 1 - TCP/UDP Performance

Aim: Measure and observe the performance of Internet transport layer protocols (TCP, UDP) under different conditions.

You should use iperf to measure the performance of TCP and UDP across several different networks. You may also use the Java-based GUI to iperf, called jperf, if you desire (note, jperf is available from the iperf website and also includes a Windows binary executable). iperf/jperf is open-source software that runs on Linux, Windows and Mac.

What network should you use?: You should repeat your measurements over at least three networks. By networks, I mean using source/destination locations and different access technologies. The first scenario, which every group must use, is directly between two computers, with as little background traffic as possible. For example, connect two laptops together using WLAN (direct in ad-hoc mode, or if that is not possible, via an AP) or connect two PC's together using a cross-over cable. This first scenario will act as a baseline test. The second and third scenario should use different locations and access technologies. Some locations you have available include: Home/Dorm, SIIT Rangsit, SIIT Bangkadi, Thammasat Uni, Work, Internet cafe, ... . Some access technologies you may have available include: Ethernet, ADSL, WLAN, ... . For example, one set of tests may be between user A's PC at home (via DSL) to user B's laptop at SIIT Rangsit.
Update 4 Feb: To access the iperf server, the computer must have a public IP address. For your home/dorm ADSL, you may be able to configure your ADSL modem to give access to the iperf server (see Portforward.com). However, if you cannot, you should test AT LEAST on two internal SIIT networks, e.g. laptop to laptop at SIIT Bangkadi via wireless LAN; Lab Pc to Lab Pc in SIIT Lab; laptop to laptop connected via cross-over cable.
How many different parameter values should be considered?: For each scenario you will perform a set of tests. Each test will be characterised by a set of parameter values. Some of the most important/interesting parameters are: transport protocol; buffer size; packet size; window size; sending rate. Of course, not all parameters are relevant for both transport protocols (TCP/UDP). You must consider at least two parameters for each transport protocol.
What values should be used for parameters?: Start with the default values. And then consider a range of values that are likely to give results that will demonstrate the impact of that parameter on performance. For example, the default value packet size was 1000 Bytes, then you could try tests for 2000 Bytes and 3000 Bytes. The results from these three tests should indicate what other values to try: if the throughput doesn't change for each value, then maybe try a much higher (10000 B) or lower value (100 B). You may need to apply some trial-and-error. Finally, you should arrive at some measurements that if plotted in a graph would show the trend in performance as the parameter value changes. In most cases, you will need between 3 and 10 values.
What should you measure?: For each test, you should measure at least the metrics provided by iperf (e.g. throughput, delay, jitter, packet loss).
How long should each test be?: By default, an iperf TCP test is for 10 seconds. This time should be sufficient, unless you notice significant changes in the performance over time.
How many tests are needed?: You should consider repeating each test multiple times. For example, run a 10 second TCP test and then repeat the same test (possible several times). Compare the results. If the performance results are about the same, then you may record just one value or the average of all values. However, if the results change significantly you may repeat more times and/or record the average results and discuss the difference. For example, if 3 tests gave you the throughput results of 10.1, 10.7 and 9.6MB/s, then they are quite similar. But if the results were 2.3, 10.7 and 25.6MB/s, then may need to repeat more times until you see a trend. If you do not notice any trend, then discuss this in your report.

Task 2 - Streaming Performance

Aim: Measure and observe the performance of audio/video streaming over a network.

You should use VideoLAN (vlc) as the streaming server and client. Many of you will be familiar with VideoLAN as a client, but it can also be used as a streaming server, either via the GUI or in command-line mode. VideoLAN is open-source software that runs on Linux, Windows and Mac.

What source files should be used?: Choose your own. However you should have at least one audio only file and one video (with audio) file. See the questions below for further details about source rates.
What network should you use?: Choose just one network. You may be limited by the network topology, especially the use of firewalls. If so, then use a very simple network (e.g. 2 laptops via the WLAN or Ethernet). Perform all tests in the same network.
How many different parameter values should be considered?: You should try source files which have at least two different source rates (that is, the rate at which the source data is encoded). For example, try a high quality and a low quality video. You should try at least two different protocols for the streaming data transfer (you will see the possible options in VideoLAN). You may try varying some of the other parameters available in VideoLAN.
What values should be used for parameters?: Choose reasonable/realistic/default parameter values.
What should you measure?: For each test, you should try to measure common metrics, e.g. throughput, delay, jitter, packet loss (although it may be very difficult to measure these in some cases). Other metrics that help in explaining how streaming works would also be useful (e.g. distribution of packet sizes, packet sending/arrival rates, overheads introduced by each protocol). You should also give a quality measurement based on your (the viewers) perception. This quality measurement may be a description, rather than a number.
How do I measure?: You should use Wireshark (or similar) to capture a set of packets, and then use Wireshark statistics/analysis. You do not have to capture all packets. For example, if you have a 1 hour video, you do not have to play the entire video - a 30 second period should be sufficient.

Report and Deliverables

You should record all your test results in a file (or files). A spreadsheet is a good choice for most of you (e.g. .xls or .ods), as you can then easily create plots of your results. You need to submit this results file, however I will NOT look at it in detail. Therefore it doesn't have to be presented professionally - it is just your own record of the results. I am not interested in the exact data values - I am interested in how you present and interpret the data values you record.

You will be assessed on a report from your group. Read on for what is expected in the report...

How do I describe the tests?: For each test (or set of tests) you should try to give enough information so that someone else could repeat your test (and get the same results). You do not need detailed descriptions, instead you should use tables, lists and diagrams. Some things to consider include: host computer specification (hardware and software); network technologies; background traffic; parameter values; test software; test duration; test methodology (the steps you took to complete the test); number of tests. If you give all the details for the first test, then when describing the later tests you can simply list the differences.
How do I report results?: Most performance results should be presented as plots (e.g. throughput versus packet size). Some results may also be presented in tables. You do not have to report every results - concentrate on the results that show interesting trends. For example, if the throughput remainined the same for all packet sizes, then you could write that rather than giving a plot.

What do I need to discuss?: For each set of tests, state any conclusions. Explain why the results are as you present. For example, if your results show the throughput increasing as the packet size increases, then say that and then explain why!
How many pages should the report be?: The report should include descriptions of the tests, selected results and discussion of the results. There is no minimum or maximum page limit. It should be long enough to clearly demonstrate that you have completed the tests and understood what is happening; and it should be short enough so that I don't get bored reading it. My guess is 10-15 pages is reasonable, but a report less than 10 pages or longer than 15 pages still may get full marks.
How do I submit?: Submit the test file(s) and a copy of the report by email. The report should be a PDF. A hardcopy of the report is not needed.
What is the marking scheme?: 45% for Task 1; 40% for Task 2; 15% for presentation. For the tasks I will be looking for your report to demonstrate that: you made an effort to complete a reasonable number and variety of tests; you selected test parameters based on your understanding of their potential impact; and you understood what the measured results indicate.
What is the deadline?: 9am, Wednesday 25 February 2009.

Return to: ITS413 Home | Course List | Steven Gordon's Home | SIIT