Any changes to the assignment instructions or hints will be added in this section.
Some questions asked by students, and a summary of my answer is below. I've also included some answers to questions from past years about different assignments (and even different course), but may be relevant to this assignment
Do I have to perform the tests with ping and iperf at the same time?
No. Perform them separately, but try to perform them under that same conditions. A simple way: run an iperf test, and then run a ping test. Then change parameters and repeat.
How many parameters (values) do I need to try?
There is no one correct answer. With Ethernet you probably should try at least 2 or 3 parameters, whereas with Wifi there may be some more.
It is important that the parameters you try impact on performance and that reasonable values are used. For example, if you decide the 'colour of your laptop' is a parameter, then you will not get full marks. Similar, if you decide 'distance' is a parameter and try values of 1m, 1.1m and 1.2m, then you will not get full marks.
When marking I will look for sensible parameters with reasonable values, accurate plots and good explanations. These are more important than the number of parameters.
What does iperf do?
iperf client generates random application data (payload), and sends it to the transport layer protocol (TCP or UDP) to be sent across the network to the iperf server. The iperf server receives the data and collects statistics on the receive rate (iperf calls it "bandwidth", we call it "throughput") and other things.
Which version of iperf should I use?
I recommend iperf2, as iperf3 reports strange (I haven't yet been able to make sense of them) results for some UDP tests, especially with WiFi.
It seems there are some substantial differences between the versions. In summary, I discovered:
What should you do?
Here are my test results that show the "wrong" results of iperf3 when using WiFi/UDP.
You may use iperf3 or iperf2 (or jperf, a Java-based iperf2). They all should produce accurate results. However note most of my instructions refer to iperf2, so you have to adapt to iperf2. (I have used iperf2 for many years, and since iperf3 is quite new have not tried it yet).
What is TCP?
Transmission Control Protocol is a transport layer protocol used between client and server applications. Its main roles are: provide reliability using a Go-Back-N style ARQ; provide flow control using Sliding-Window style; provide congestion control so that the source doesn't overflow the intermediate devices in the Internet. It is quite complex and we don't have time to cover the details in this course. We already know about ARQ and flow control, but not congestion control. For this assignment, it is important that you know the following: an application sends data to TCP via a buffer, then TCP follows its flow/error/congestion control algorithms to determine how much to send. The result is the application does not control how fast TCP sends; TCP controls itself.
What is UDP?
User Datagram Protocol is a transport layer protocol used between client and server applications. Compared to TCP it is very simple: application sends data to UDP via a buffer, then UDP sends that data in a UDP packet. UDP has no error/flow/congestion control. So the application DOES control how fast UDP sends, hence you need to use the -b option in iperf to create different sending rates.
Which options for iperf should I use?
Understand all the options in iperf requires good knowledge of TCP and UDP. We will not cover them in detail in this course. Therefore I suggest you don't spend too much time on trying all the iperf options. In fact you could complete the assignment using mostly the default values (as long as you analysed other parameters). Below I have some notes on the options (from an old course, so some may be out of context for this course; also they refer to iperf2, but should be similar options in iperf3).
iperf -m prints the Maximum Segment Size (MSS) for TCP. (In jperf there is an option "Print MSS"). The MSS is normally determined by the operating system. Recall each Data Link layer has a maximum frame size or Maximum Transmission Unit (MTU). For example, Ethernet MTU is 1500 bytes. The operating system will try to find the minimum of all MTU's along a path from source to destination. For example, if the path from source to destination has three links (Ethernet, MTU=1500; Wireless LAN, MTU=1500; Dial-up, MTU=576), then the Path MTU is 576. Sometimes the operating system can determine the entire path MTU and may only use the MTU of the local link. The MSS is calculated from the MTU minus the TCP and IP headers which is usually 40 bytes. So if Path MTU = 1500, then MSS may be 1460 bytes.
iperf -M sets the MSS. You can change the MSS, however some values will give you a error/warning. You can investigate what happens when you use a smaller MSS than the default value.
iperf -w sets the TCP buffer size, i.e. the "window" parameter in jperf. Recall a TCP receiver indicates how much space is available in its buffer using the Advertised Window (for flow control). You can use the -w ("window") option to set the size of this buffer.
iperf -l sets the length of data written by the application (at client) or read by the application (at the server). If recall your sockets programming, the application uses write() to send data and read() to receive. The "length of the buffer" is the amount of data that is sent with each write() or the maximum amount of data that can received with each read(). With TCP, it is hard to tell what impact the length will have on performance (since it depends on the details of each flow/congestion/error control algorithm). With UDP the length can be used to set the payload/packet size.
If you want to try iperf parameters I suggest, in order of 'easiness':
I have a detailed explanation of TCP window size experiments here.
What is the difference between sending rate, receiving rate, throughput, bandwidth and data rate?
Consider a link from A to B, where A will send data to B. The data rate is typically a characteristic of the link technology and is often fixed (although different rates possible). For example, with Ethernet, the data rate may be 10 Mb/s, 100 Mb/s or 1000 Mb/s. With WiFi, it may take one of many values like 2 Mb/s, 11 Mb/s, 54 Mb/s, 72 Mb/s (see Wikipedia for a table listing most of them). This is the rate at which the Physical layer can send bits across the link, and is sometimes referred to as capacity.
Now consider an application on A sending data to B. The rate at which A generates the data to send is the sending rate. The sending rate will depend on the user and application.
Unfortunately, the rate at which A sends data may not be the same as the rate at which B receives data. Firstly, the data rate is the upper limit: if A sends at 200 Mb/s, but the data rate is 100 Mb/s, then B cannot receive at a rate higher than 100 Mb/s. Secondly, and an important part of this assignment, the protocols used to deliver the data introduce overheads, such as: headers, ACKs and other non-data packets, time spent not sending, retransmissions. So in practice, the receiving rate will be less than the data rate. Of course, the receiving rate must also be less than the sending rate (A cannot send at 50 Mb/s and B receive at 60 Mb/s - that's crazy!).
We call the receiving rate the throughput. iperf calls the receiving rate bandwidth (I prefer throughput, so we don't confuse the receiving rate bandwidth in b/s with the bandwidth of a signal in Hz).
In this assignment, for the throughput (and efficiency) we are trying to explore which parameter values we should use to get maximum throughput. So you should:
Do you have an example spreadsheet or plots?
I found a spreadsheet from previous years with fake (made-up) data and some plots. Don't use the spreadsheet as a guide for what parameters to use; just use it as an example of one way to collect the data and create plots. Of course you can use other ways, and yours may be better than the example. It was created with LibreOffice and has two sheets; it should open in Excel as well. Here it is.
Two popular technologies for conducting devices across a small area, such as in homes and offices, are:
The standards for these technologies are created by IEEE, and have been updated/improved over the past 20+ years. For example, WiFi has had data rate (and other) upgrades, from 2 Mb/s, to 11Mb/s to 54 Mb/s to more than 300 Mb/s. The updated standards are often referred to by a letter, e.g. IEEE 802.11b, IEEE 802.11g, 11n, 11ac. In this assignment we will refer to them by their common names: Ethernet and WiFi.
Ethernet is commonly used in a topology where each computer in the LAN (laptop, PC, server) connects via a LAN cable to a central Ethernet switch. The Ethernet switch also often has a cable to a router that connects to other LANs and the Internet.
WiFi is commonly used in a topology where each computer in the WLAN (laptop, phone) connect wireless to a central WiFi Access Point (AP). The WiFi AP also often has a LAN cable to a router that connects to other LANs and the Internet.
Below is a picture showing an example of a network that combines WiFi and Ethernet. There are three APs. The mobile devices (M1 to M8) would associate with one of the three APs (how they choose which AP depends on a variety of factors not discussed here). E.g. M1 and M2 may associate with and use AP1, while M3, M4 and M5 may associate with and use AP2. The APs in turn have wired connections to a central switch. There are also PCs and servers connected to that switch. Finally the switch has a connection to a router which then connects to other LANs/WANs for Internet access.
Both Ethernet and WiFi also support direct communications between two computers. For example, PC1 connects directly to PC2 via a LAN cable (not using a switch) or M7 connects directly to M8 via a WiFi direct or adhoc connection (not using an AP).
Both Ethernet and WiFi support different physical layer technologies which define: data rate, signal encoding scheme, media requirements (bandwidth, calbe type), transmit and receive powers, etc. Often devices support more than more physical layer standard. For example, a Ethernet card can support data rates of 10, 100 and 1000 Mb/s. A WiFi adapter can support data rates of 11Mb/s and 54 Mb/s. Usually the best available will be automatically chosen, but the user can also manually select.
Across the different physical layers, a common data link layer is used. With respect to data transfer:
The focus of this assignment is to measure the performance of both Ethernet and WiFi. There are three performance metrics of interest:
Delay and throughput need to be measured; efficiency can be calculated once you have measured the throughput (and know the data rate).
There are many factors that may impact on performance of both Ethernet and WiFi. You should learn about some of the factors in this assignment. Some factors, or parameters, that may impact on performance are:
In this assignment you should consider the first 6. You do not need to compare different hardware or drivers (you can use the hardware/drivers that you have available, and different groups may have different hardware/drivers).
You need to perform experiments to measure the network performance (delay and throughput). In general, for a single experiment you need to:
Then the experiment may be repeated under the same conditions and the mean/stdev/min/max values of the set recorded. Then repeat the experiments with different parameter values.
Network Topology
The network topology is the arrangement of devices for the experiment. Some topologies you may choose (where "computer" may be PC, laptop or phone):
For example, those topologies correspond to:
You do not need to experiment with all topologies. You must have one wired only topology (1 or 3) and one with a wireless link (2, 4, 5 or 6). Choose whatever is easiest for you.
Parameter Values
For each experiment, choose and set the relevant parameter values, e.g. frame size, data rate, distance, protocol, ... . The values should be constant throughput the duration of the experiment. This may be a challenge for data rate, since many WiFi devices automatically adjust the data rate depending on signal strength. You can however fix the data rate in some APs, or at least monitor it during the experiment and discard the experiment if the data rate changes.
Generate Frames
You need an application on your computer to generate frames to send. You can use:
I have written a quick start guide for using iperf, as well as a 10-minute video explanation of using iperf. Some additional resources on using iperf: 1, 2.
Recording Performance Metrics
ping and iperf (and similar applications) report the delay and throughput, respectively (although sometimes using different names, like RTT and bandwidth).
It is recommend that for each experiment you record the topology, parameter values and performance metrics in a spreadsheet.
Repeating Experiments and Experiment Duration
When you run an experiment it will be for some duration. By default, ping runs forever, while iperf runs for 10 seconds. Recommend durations are:
When you run an experiment there may be factors outside of your control that impact on performance. For example, with WiFi, other devices nearby may be transmitting. Therefore you should repeat each experiment multiple times, and report the mean of those runs. Recommend number of repeat runs are:
As an example, if you are measuring the delay across Ethernet link, you may run ping (for 20 pings), record the delay, then run it again, record the delay, then run it a third time and record the delay. Then calculate the mean of those three runs and report the mean in your results.
For WiFi throughput tests, run iperf (for 20 seconds) five times, and record the mean of the throughput across those five runs.
Accuracy of Performance Metrics
Delay should be measured to the nearest tenth of a millisecond (0.1 ms). Ping reports to the nearest microsecond. For example, if you measure the delay to be 1.356 ms, then report as 1.4 ms.
Throughhput should be measured to the nearest megabit per second (1 Mb/s). For example, if you measure the throughput to be 27.836 Mb/s, then report as 28 Mb/s.
The tasks for this assignment are:
How many different parameter values?
There is no one correct answer. Firstly, you should think of the possible parameters that may impact on performance, and perform experiments for values that you think will have the most impact. If you don't know which parameters will impact on performance, then try them and see. For example, if you don't know if the distance will impact on performance of Ethernet, try with a short cable and long cable and see if there is any significant difference. If you plot the performance (y-axis) versus parameter value (x-axis) and see a flat line, then it suggests that parameter does not impact on performance.
Two files must be submitted on Moodle:
The report structure is explained below. The spreadsheet may be structured as you wish. I may not check the spreadsheet of all groups - it will only be used if I need to confirm your results or if I see strange results in the report.
Structure of Report
Include the report title, your group number and names of the students at the start. Then include at least the following sections:
Sections 1 and 2 should include:
Section 3 should explain what is the difference between the efficiency of Ethernet and Wifi: how much they differ and why?
Reporting Results
Lets assume you run some ping tests for Ethernet for different payload sizes: 100 B, 200 B and 300 B. Then you would give a plot of delay vs payload size with payload on the x-axis (100, 200 and 300) and the delay (in ms) on the y-axis. Then under the plot, explain how many runs you performed (e.g. "The above plot was obtained from the mean values across 3 runs"), explain the results ("The above plot shows the delay increasing linearly with the payload size") and explain why ("The delay increases with the payload size because ... ").
Some further advice on plots:
Submitting the Assignment on Moodle
Submit your assignment here. Only 1 person per group submits the report.
Only two files are allowed to be submitted, one must be a PDF and the other should be a spreadsheet (e.g. XLS, XLSX, ODS, CSV, or other). Maximum size of each file is 5MB.
You may change your submission (e.g. re-upload a new version) anytime before the deadline. The deadline is strict - no submissions will be accepted after the deadline. It is recommend you upload your submission at least 24 hours before the deadline, to ensure you have no problems with submitting. You may then re-upload newer versions up until the deadline.
Submissions via other means (e.g. email) are not allowed, and will be ignored. If you have problems with using Moodle to submit, then you must contact me BEFORE the deadline.