Experimental Evaluation of End-to-End Available Bandwidth Measurement Tools
Young-Tae Han1, Eun-Mi Lee1, Hong-Shik Park1, Ji-Yun Ryu2, Chin-Chol Kim2, and Yeong-Ro Lee2
1 Dept. of Information and Communications, Korea Advanced Institute of Science and Technology, Daejeon, S. Korea
{organon,boricha,parkhs}@kaist.ac.kr
2 National Information Society Agency, Seoul, S. Korea {rjy,cckim,lyr}@nia.or.kr
Abstract. Accurate bandwidth measurement is essential for network management, monitoring, and planning. Various active probing-based strategies and tools are have been developed for estimating available bandwidth. However, the test result of each tool could be different ac- cording to the strategies and tools even in the same network environment.
The purpose of this paper is to give comprehensive information to users with interpretive comparison and performance test. The comparison and test with regard to measurement efficiency, and traffic load of probing packets. including IGI/PTR, pathload, pathChirp, spruce, and Iperf, are performed in dynamic network environments.
Keywords: Active measurement, available bandwidth, performance measurement, network monitoring.
1 Introduction
Accurate bandwidth measurement is essential for network management, moni- toring, and planning. Various active probing-based strategies and tools are have been developed for estimating available bandwidth. However, the test result of each tool could be different according to the strategies and tools even in the same network environment not only because the dynamic network environment causes variations of test results but also because these tools are not standard- ized. Therefore, users should be preacquainted with differences of performance and characteristics of them. The purpose of this paper is to give comprehen- sive information to users with interpretive comparison and performance test.
To provide comprehensive information about performance of existing tools for end-to-end available bandwidth measurement, the comparison and test of these tools, including IGI/PTR [1], pathload [2], pathChirp [3], spruce [4], and Iperf [5], is performed in terms of measurement efficiency, and traffic load.
The remainder of this paper is organized as follows. Section 2 summarizes bandwidth measurement tools and describes test environments and section 3 shows the results. Finally, section 4 concludes this paper.
C.S. Hong et al. (Eds.): APNOMS 2009, LNCS 5787, pp. 498–501, 2009.
c Springer-Verlag Berlin Heidelberg 2009
End-to-End Available Bandwidth Measurement Tools 499
2 Tools and Test Scenarios
Several open source tools for estimating available bandwidth are selected as following table 1.
Table 1.Available Measurement Tools
Tool Author Version Method Protocol
pathload Jain 1.3.2 SLoPS UDP
pathChirp Ribeiro 2.4.1 SLoPS Chirp UDP
IGI/PTR Hu 2.1 SLoPS UDP
Spruce Strauss 0.2 PGM UDP
Iperf NLANR 2.0.4 Achievable TCP throughput TCP/UDP
Due to the page limit , our test environment is not presented but our test environments is similar to [6]. To generate cross traffic, AX4000 is exploited.
The tests are performed in the following scenarios:
(1) All measurement tools are tested without any cross traffic and all probing packets are traced and analyzed using tcpdump in terms of the number, traffic volume, and packet length distributions of probing packets. (2) All measurement tools are tested with cross traffic; 50, 100,150, 200, 250, 300, and 350Mbps.
CBR cross traffic is injected along the path and generated packets are 1500- byte length of TCP. (3) All measurement tools are tested while increasing delay.
The delay is increased by 50ms up to 400ms. (4) All measurement tools are tested while increasing packet loss. The packet loss rates of path are 1, 3, 5, and 10%.(5) Performance evaluation of the single and parallel mode of Iperf in lossy environment is performed. The network delay and packet loss are imposed with TC [7] at the linux-based network emulator.
3 Test Results
Table 2 shows workload characteristics of each tools. Time is the duration be- tween the first probing packet and the last probing packet passed on the path.
Packets are the total number of observed probing packets.Vpis the total volume of probing packets. To show efficiency and accuracy, we defined two metrics; the average probing rate,Rp(Vp/T) and measurement error,ε(|Ae−Aee|/Ae) where Aee is actual end-to-end available bandwidth (400Mbps) without cross traffic.
Fig. 1 shows the test results when network delay gradually increased. Only Iperf is affected by delay because TCP throughput is significantly affected by round trip time (RTT). According to [8], TCP throughput is being decreased when RTT is being increased.
Fig. 2 and 3 show test results of single and parallel modes of Iperf without any network congestion such as delay and loss.In the lossy network, we observed that
500 Y.-T. Han et al.
50 100 200 300 400 500 0
100 200 300 400
Bandwidth (Mbps)
Delay (ms) (a) IGI
50 100 200 300 400 500 0
100 200 300 400
Bandwidth (Mbps)
Delay (ms) (b) PTR
50 100 200 300 400 500 0
100 200 300 400
Bandwidth (Mbps)
Delay (ms) (c) pathChirp
50 100 200 300 400 500 0
100 200 300 400
Bandwidth (Mbps)
Delay (ms) (d) pathload
50 100 200 300 400 500 0
100 200 300 400
Bandwidth (Mbps)
Delay (ms) (e) Spruce
50 100 200 300 400 500 0
100 200 300 400
Bandwidth (Mbps)
Delay (ms) (f) Iperf
Fig. 1.Test results with delay increment
Table 2.Summary of probing packets
Tool Time Packets Vp Result (Ae) Rp ε
(second) (bytes) (Mbps) (Mbps)
IGI/PTR 1 540 3,983,120 IGI-345 30.389 0.139
PTR-351 0.122
pathload 1 946 564,512 315 4.307 0.213
pathChirp 120 39,618 40,489,332 348 2.574 0.130
Spruce 9 200 300,000 405 0.254 0.011
Iperf 10 510,518 60,963,944 407 46.512 0.020
0 20 40 60 80 100
250 300 350 400
Number of test
Bandwidth (Mbps)
(a) Single Mode
0 20 40 60 80 100
250 300 350 400
Number of flows
Bandwidth (Mbps)
(b) Parallel Mode
Client Server Client Server
Fig. 2.Iperf single vs. parallel mode with- out packet loss
0 20 40 60 80 100
250 300 350 400
Number of test
Bandwidth (Mbps)
(a) Single Mode
0 20 40 60 80 100
250 300 350 400
Number of flows
Bandwidth (Mbps)
(b) Parallel Mode
Client Server Client Server
Fig. 3.Iperf single vs. parallel mode with packet loss
End-to-End Available Bandwidth Measurement Tools 501
the performance of the parallel mode is unstable. According to our observation and [9], the loss rate ofpis the dominant factor of the throughput of aggregate TCP flows. Thus, the parallel mode upon TCP is not appropriate for estimating bandwidth in the lossy network.
4 Conclusion
Several available bandwidth measurement tools have been evaluated . To verify the influence of delay and loss, tests were performed in dynamic network en- vironments. Intrusiveness with respect to Iperf is the highest probing rate and this high probing rate may cause loss of cross traffic. According our observa- tion, the TCP-based tool is more sensitive to delay, and packet loss than the UDP based tools. The delay is more critical than packet loss within TCP based tools. In UPD based tools, some tools such as pathChirp use sampling based estimation methods. In this case, the measurement performance is deteriorated by the packet loss. Thus, to develop more accurate and non-intrusive tools, more complex and dynamic environments should be considered.
Acknowledgments. This work was supported in part by the MKE, Korea, under the ITRC support program supervised by the IITA (IITA-2009-(C1090- 0902-0036)).
References
1. Ningning, H., Steenkiste, P.: Evaluation and characterization of available band- width probing techniques. IEEE Journal on Selected Areas in Communica- tions 21(6), 879–894 (2003)
2. Jain, M., Dovrolis, C.: End-to-end available bandwidth: measurement methodol- ogy, dynamics, and relation with tcp throughput. IEEE/ACM Trans. Netw. 11(4), 537–549 (2003)
3. Ribeiro, V., Riedi, R., Baraniuk, R., Navratil, J., Cottrell, L.: Pathchirp: Efficient available bandwidth estimation for network paths. In: Passive and Active Measure- ment Workshop, vol. 4 (2003)
4. Strauss, J., Katabi, D., Kaashoek, F.: A measurement study of available bandwidth estimation tools. In: Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement, Miami Beach, FL, USA, pp. 39–44. ACM Press, New York (2003) 5. Iperf,http://sourceforge.net/projects/iperf/
6. Montesino-Pouzols, F.: Comparative analysis of active bandwidth estimation tools.
In: Passive and Active Network Measurement, pp. 175–184 (2004) 7. TC,http://tldp.org/HOWTO/Traffic-Control-HOWTO/index.html
8. Padhye, J., Firoiu, V., Towsley, D., Kurose, J.: Modeling tcp throughput: a simple model and its empirical validation. In: Proceedings of the ACM SIGCOMM 1998 conference on Applications, technologies, architectures, and protocols for computer communication, Vancouver, British Columbia, Canada, pp. 303–314. ACM Press, New York (1998)
9. Hacker, T.J., Athey, B.D., Noble, B.: The end-to-end performance effects of parallel tcp sockets on a lossy wide-area network. In: Parallel and Distributed Processing Symposium., Proceedings International, IPDPS 2002, Abstracts and CD-ROM, pp. 434–443 (2002)
