Contributed Paper
Manuscript received March 16, 2004 0098 3063/04/$20.00 © 2004 IEEE
An Ethernet Based Data Storage Protocol for Home Network
Wilson Yong Hong Wang and Tow Chong Chong, Member, IEEE
Abstract — In this paper, we present an open source network storage protocol, HyperSCSI, which can be applied to build a home storage network. HyperSCSI is designed for transmission of SCSI (Small Computer System Interface) protocol data across Ethernet based network. With HyperSCSI, a home user can access SCSI or non-SCSI storage device (like IDE, USB) natively through the network as if the storage device is attached locally. We describe HyperSCSI protocol architecture, operations, as well as some key protocol features that make it distinct from conventional solutions. We also demonstrate the experiment and applications that integrated HyperSCSI protocol with Fast Ethernet and IEEE 802.11b/g WLAN (Wireless Local Area Network) environment. The results show that it is promising to build efficient and cost-effective HyperSCSI based home storage network that utilizes common-off-the-shelf storage hardware and well deployed network infrastructure1.
Index Terms — HyperSCSI, Home Network, Network Storage, Ethernet, WLAN.
I. INTRODUCTION
With the tremendous advancements in computer technologies, networks, and consumer electronics, users at home face growing challenges as where and how to store and manage digital information generated for home applications and how to efficiently move data from device to device within the home environment. Traditionally, people think of digital information storageas a hard disk drive (HDD) inside a desktop computer, or perhaps a CDROM or DVD disc. Each of these items is usually connected to, or used directly by, a computer or device. However, recent technological advances have created a new technical field by making these digital data storage devices network-enabled. There will be a growing demand for home users to manage their digital information and moving data efficiently within such home environment.
For a home network, a variety of protocols have been proposed. For example, Jini and UPnP have been proposed for home appliances connectivity without special configuration before use [4]. IEEE 1394, USB [5] and Bluetooth [6] are candidates developed to connect the home appliances with the bandwidth and availability benefits. The network technology like Ethernet, with the advantage of pervasiveness and maturity track record from corporate environment, moves into home domain aggressively. As a network protocol, Ethernet
1 This work is supported by Data Storage Institute, which is funded by the Agency for Science, Technology and Research Singapore.
Wilson Y.H. Wang is with the Network Storage Technology Division, Data Storage Institute, DSI Building, 5 Engineering Drive 1, 117608 Singapore. (e-mail: [email protected])
Tow Chong Chong is with the Data Storage Institute, DSI Building, 5 Engineering Drive 1, 117608 Singapore.
can be applied to both wired and wireless situations, which include Fast Ethernet, HomePNA, powerline networking as wired solution and IEEE 802.11 a/b/g as Wireless Local Area Network (WLAN) solutions [3],[13],[14]. However, all solutions mentioned above do not focus their efforts on data storage networking characteristics. Whereas with the experience of network storage research, we believe there is a need to connect home storage device to the network in the very native manner in order to provide efficient and cost- effective solution for managing and moving large amount of data storage over the home network.
As in network storage industry today, a technology called Fibre Channel (FC) [8] is used originally to meet the requirements of building a storage area network (SAN) for large servers in high performance network environments.
However, this technology is expensive and complex, and is not practicable to be deployed at home. With the ubiquity of Ethernet technology, the effort of providing networked storage for home becomes attractive. With the capability of storage networking, large amount of digital information can be easily managed and accessed, and the data scalability and availability can also be insured. Among the efforts, there is one protocol called iSCSI (Internet Small Computer System Interface) [9][10], which can transport storage data over TCP/IP network platform.
Although TCP works well in the environment that it provides both flow and congestion control mechanism, the complexity of TCP protocol will be the burden for low-end home appliances, while in most situation, a local network connectivity is sufficient for home applications. Furthermore, TCP encounters problem due to its inability to handle packet loss efficiently in wireless network [16]. With such consideration, based on the characteristics of network storage and home storage application, we design and deploy a new network storage protocol called HyperSCSI without using TCP/IP protocol to further address the simplicity and.
efficiency. HyperSCSI is designed to transport large amount of data on basic Ethernet by encapsulating SCSI standard storage interface protocol so that a user can access and control SCSI and even non-SCSI storage devices (like IDE, USB, or Firewire) natively over the network [1],[2]. It provides necessary reliability and flow control mechanism. It is also desirable to work on WLAN directly with simplicity in configuration, efficiency in operation, and does not require special hardware and, moreover, provides good performance.
HyperSCSI contains built-in security feature as an option, thus it can be turned on or off based on user requirement separately.
The paper is organized as follows. In Section II, we present the main HyperSCSI protocol architecture and operations to emphasize its uniqueness and how it can be used to fit into
DA SA EtherType HyperSCSI Packet CRC
Fig. 1. Ethernet frame and HyperSCSI packet
Ver M OpCode HPB Length P. Param
HS Serial No HS Digest
HS Protocol Parameters (if any) Data Block
Fig. 3. HyperSCSI Protocol Data Unit (PDU) Header Block Template HyperSCSI PDU HyperSCSI Header
Fragment No.
LF Tag
Reserved
Fig. 2. HyperSCSI Packet and Header existing home network infrastructures. Section III presents key
features of the proposed protocol. Section IV describes a benchmark experiment, where HyperSCSI is used over Fast Ethernet and WLAN, and analyses the performance results. in In Section V, we also discuss and demonstrate applications with audio, video playback and data transfer in a home wireless environment. Finally, Section VI concludes the paper.
II. HYPERSCSI PROTOCOL DESCRIPTION
HyperSCSI is designed to work on an Ethernet platform.
Our first effort was to standardize work over the SCSI, which is the predominant mechanism for various storage and even non-storage devices. The question then turned quickly to how we could make SCSI “network-enabled”. This gave rise to our idea of “HyperSCSI”. Its packets can be encapsulated within a standard Ethernet frame and has its own Ethernet protocol type, packet format. Furthermore, HyperSCSI Protocol Data Unit (PDU) and all kinds of operations can be defined.
A. HyperSCSI Packet and EtherType
HyperSCSI packet can be encapsulated within a standard Ethernet frame. There are five information fields in one Ethernet Frame according to IEEE Standard 802.3. The first two six-bytes fields are for Destination Address (DA) and Source Address (SA) respectively. The two-byte EtherType field (with the value of > 1536 or 0x0600) provides the protocol identification of the data unit inside the Ethernet frame. HyperSCSI packets are transported over the Ethernet with the EtherType of 0x889a2 for the protocol identification.
There are also the Data field, which contains HyperSCSI packet with a maximum length of 1500 bytes, and the
Checksum field (CRC) field. Fig. 1 shows the Ethernet frame containing HyperSCSI packet.
B. HyperSCSI Packet Header
At the network layer, the HyperSCSI PDU is fragmented to fit the network packet size. Each HyperSCSI packet is encapsulated with a 3-byte header that consists of the Packet Identification information.
The HyperSCSI packet header consists of a 4-bit reserved field, a 9-bit Tag, a single bit Last Fragment (LF) field and a 10-bit Fragment Number field. The reserved field is filled with zeros. With this 3-byte header, HyperSCSI receiver can recognize and re-assemble the data block correctly. The packet header is shown in Fig. 2. The tag number is used to support multiple HyperSCSI data streams. A counter tags the HyperSCSI block each time a SCSI command block is encapsulated. The counter increments the tag number as each
2 IEEE granted Ethertype number of #889a to HyperSCSI in December 2001
HyperSCSI block is being tagged.
If fragmentation occurs, the fragment number orders the fragments of a HyperSCSI packet. But if there is no occurrence of fragmentation as the HyperSCSI command block and data are sufficiently short to be encapsulated into a single Ethernet frame, then the HyperSCSI fragment number is set to zero. Otherwise, the SCSI command block and data is fragmented into smaller fragments with the numbering ordered from one to the maximum number. When the last fragment is transmitted in the case of fragmentation, the last fragment flag (LF) is set to value 1.
When multiple SCSI command blocks are encapsulated and transmitted in parallel by the HyperSCSI protocol, then each HyperSCSI data block is assigned a unique tag number, which is then attached to every fragment of the HyperSCSI packet.
The receiver is then able to sort out and re-assemble the fragments of each data block; based on the knowledge of the tag number and fragment number that it receives.
C. HyperSCSI PDU Header Block Template
The HyperSCSI PDU includes the protocol header block (HPB) and if necessary a data block. The HyperSCSI protocol header block has the template format shown in Fig. 3. This template contains the parameters that define the specific HyperSCSI operation. All integer fields in the HyperSCSI HPB are in the network byte order format.
The first byte is used for the HyperSCSI protocol version.
The highest 4 bits are allocated to contain the major version number and lowest 4 bits are allocated to contain the minor version number. The current version is 1.0, with 1 as the major version number and 0 as the minor version number. The second byte is used for the HyperSCSI operation mode (M) and operation code. The operation mode, which is represented by a single bit, is used to show the mode of the HyperSCSI module (or node) that sends this PDU. If the PDU is sent from a HyperSCSI server module, the M bit is set to the value of 1, otherwise 0. The operation code is used to indicate the
Ver M OpCode HPB Length CDB_LEN HS Serial No
HS Digest HyperSCSI Device ID
SCSI Command Descriptor Block (CDB) Buffer
SCSI Command Block Serial Number SCSI Command Block Data Buffer Length
SCSI Command Result
SCSI Sense Buffer
LSB Sense Offset Data Dir
HyperSCSI Reply Result Option (if any)
Fig. 4. HyperSCSI Command Block (HCBE) structure.
different operations for the protocol. The operations codes are described in Table 1. The third byte is used for the HPB length. The HPB length is defined as a count of 32-bit words, implying that the end of the HPB is padded with zeros when necessary. The HyperSCSI serial number, which is a 32-bit integer, is a sequence number for HyperSCSI PDU. Every HyperSCSI PDU is stamped with the HyperSCSI serial number, starting from 1 for the first PDU and incrementing to 232 – 2 for the 232 – 2 PDU. The HyperSCSI serial number cycles back to 1 after it has cycled past 232 – 2. The HyperSCSI digest is a HMAC value of the HPB. This allows the detection of any illegal tampering of the HPB. The HyperSCSI digest is set to zero if no HMAC is required for the HPB.
D. HyperSCSI Command Block
Inside a regular computer system, the SCSI command block or SCSI request block is used to support the SCSI command and data transfer between SCSI middle layer and SCSI lower layer drivers. With HyperSCSI protocol, the SCSI command and data are transmitted over the network. Therefore, a new category of data structures is defined, which is called HyperSCSI command block (HCB). It has been designed with the concerns of compatibility between different systems and between different versions of HyperSCSI protocol.
The operations used to transport HCB are called HCB Encapsulation (HCBE), which the operation codes are defined in Table 1. Within the HyperSCSI command block, the basic data fields of SCSI command packet are included, such as SCSI command descriptor block, SCSI command data buffer length and SCSI sense data buffer. The protocol data fields related to HyperSCSI protocol are also included, such as HyperSCSI version, HyperSCSI serial number, HyperSCSI device ID and HyperSCSI command block length.
In Fig. 4, in addition the common fields of HyperSCSI header block, more fields are defined to cater for the storage data transportation. The HyperSCSI device ID is a 32-bit integer indicating the device number of the device that has been allocated by the HyperSCSI Server. The SCSI command descriptor block (CDB) buffer is a 16-byte long buffer, which holds the SCSI command descriptor block follows the SCSI international standard. The length of the SCSI command descriptor block is contained in CDB length field.
Every SCSI command block has its unique serial number, which is preserved to indicate the uniqueness of every HyperSCSI command block. The SCSI command execution result is kept in SCSI command result field. The SCSI sense buffer is also used to contain the SCSI sense data, which may be queried from HyperSCSI Client.
The format of sense data follows the SCSI international standard. Its basic length is 18 bytes, where in HyperSCSI command block, the sense data buffer is defined of 18 bytes. If additional sense data is required, this portion will be included
in the Option field with a Sense Offset containing the value of the starting position.
The SCSI data buffer contains the whole data of the SCSI command block to be transmitted. The HPB length indicates the HyperSCSI command block length, which excludes the SCSI data buffer. With this field, additional information can be included in the Option field within the HyperSCSI command block. The HyperSCSI protocol specifies that it be filled with the value of zero, if the data field is not supported by the system.
E. HyperSCSI Operations
HyperSCSI protocol comprises of various packet structures.
These structures are categorized by classes and types subsequently. Packets of a specific class and type may also have more than one function depending on the context of the communication. These packets are responsible for transporting SCSI data and command blocks as well as managing the connection and communication channel. Table 1 presents operation codes in various HyperSCSI packets.
1) Typical HyperSCSI Connection Flow Sequence
Fig. 5 illustrates a typical sequence of the communication stages between a client and server using the HyperSCSI protocol. The various stages of the connection flow sequence are described below.
Server Client
HCC_ADN_REPLY FC_ACK_SNR FC_ACK_REPLY
HCC DEVICE DISCOVER HCC_ADN_REQUEST
Fig. 5. HyperSCSI Connection Setup The HyperSCSI connection set-up is a three-step
handshaking procedure between a HyperSCSI client and server pair. Typically, in a storage network, the host machine (HyperSCSI client) is responsible for locating and initiating connections to storage devices (HyperSCSI servers). During this process, the HyperSCSI client issues a HCC_DEVICE_DISCOVERY via Ethernet broadcast packet, to locate devices on the network. Once the HyperSCSI server receives this packet, it checks the client address for authentication purposes and transmits the HCC_ADN_REQUEST packet back to the HyperSCSI client.
In order for the HyperSCSI client to establish a connection with the HyperSCSI server, it must then send the correct response through a HCC_ADN_REPLY command and add the ID numbers of the devices that it has access to into its own registry. If the server successfully authenticates the HCC_ADN_REPLY, the connection is accepted and the HyperSCSI client can now send commands to the server.
Within the HCC_ADN request and reply method, authentication challenges, encryption key exchanges, device specific option negotiations and other information supporting N-channel communications such as server/client IDs and network addresses are also provided and exchanged.
2) Flow Control and ACK Window Size Setup
An acknowledgement (ACK) mechanism has been adopted to support flow control of data between a HyperSCSI client and server pair. The ACK window size refers to the number of packets that the transmitter may continuously send before waiting for an acknowledgement. This window size is negotiated and agreed upon before data flow can take place and is set by the requestor through an FC_ACK_SNR command. This packet is issued as a separate message and typically, the server will be the one to issue this command so that the server has the ability to balance loads or priorities
across multiple clients, although the client may also issue the same command to adjust the network traffic load. Once the FC_ACK_SNR has been received, the new status will be acknowledged to the requestor with an FC_ACK_REPLY. If the requestor receives the acknowledgement, it assumed that the window size is accepted and packet transmission using the new window size can begin. The ACK window size can be set based on traffic loads, or buffer capacities and can be set at start-up or changed dynamically during run time. This allows for different window sizes to be dynamically set by clients and servers to fit changing performance, reliability or QoS requirements.
3) HyperSCSI Data Transmission
When there is a SCSI request from the local OS SCSI upper layer of the host machine, the HyperSCSI client software is responsible for converting the OS-specific SCSI command block together with any relevant data (as in a write command) into a platform independent HyperSCSI Command Block (HCB). The client then encapsulates and fragments the HCB into one or more HCBE_REQUEST packets that it sends to the HyperSCSI server. SCSI command blocks and user data TABLEI
HYPERSCSI OPERATION CODES DESCRIPTIONS
Class Type Hex Name Description
0x0 0x00 HCBE_REQUEST HyperSCSI Command Block Encapsulation Request
0x0 0x1 0x01 HCBE_REPLY HyperSCSI Command Block Encapsulation Reply
0x0 0x10 HCC_DEVICE_DISCOVERY HyperSCSI Client issues this packet to discover the storage devices on the network
0x1 0x11 HCC_ADN_REQUEST Authentication / Device options Negotiation Request 0x2 0x12 HCC_ADN_REPLY Authentication / Device options Negotiation Reply 0x1
0x3 0x13 HCC_DISCONNECT HyperSCSI Connection Termination
0x0 0x20 FC_ACK_SNR Flow Control Set up and Acknowledge Request
0x2 0x1 0x21 FC_ACK_REPLY Acknowledge Reply
0x0 0x30 HMC_ADDR_REPORT HyperSCSI node report its multiple address to remote node 0x1 0x31 HMC_ADDR_REPLY Reply about multiple address report to remote node
0x2 0x32 HMC_LOCAL_REQUEST HyperSCSI node checks local address for local error detection 0x3 0x33 HMC_LOCAL_REPLY Reply about the multiple address local detection
0x4 0x34 HMC_REMOTE_REQUEST HyperSCSI node checks remote address for remote error detection 0x3
0x5 0x35 HMC_REMOTE_REPLY Reply about the multiple address remote detection Other classes and types are reserved
FC ACK REPLY HCBE REQUEST HCBE REQUEST
HCBE REQUEST HCBE_REPLY HCBE_REPLY FC_ACK_REPLY
Server Client
HCBE_REPLY
•
•
•
• SCSI
Data Reply
SCSI Data Request
Fig. 6. HyperSCSI Data Transmission
Server A Server B
Disk 1 Disk 2
Client X Client Y
Disk 1
Group ABC
Group DEF
Fig. 7. HyperSCSI Device Discovery and Storage Groups Example will therefore be transmitted together in the same packet. The
HyperSCSI server receives the data stream, re-assembles the HyperSCSI command block and relevant user data, converts it back to an OS-specific SCSI command block and passes it to the relevant hardware for execution. When the result of this SCSI request is ready, the HyperSCSI server will send the result together with the requested data back to HyperSCSI client by issuing the HCBE_REPLY packet stream in a similar manner as the request. The HyperSCSI client reassembles the HyperSCSI command block and converts it back to an OS- specific SCSI command block before passing it on to the local OS SCSI upper layer. During this transmission, flow control mechanisms take effect through the use of FC_ACK_REPLY commands. Fig.6 shows the flow of HyperSCSI data transmission.
4) Connection Termination
The HyperSCSI client can close a connection by sending an HCC_DISCONNECT command to the HyperSCSI server. The server will then remove this client from its connection list and close the connection. Servers do not need to acknowledge disconnect requests from clients because SCSI connections are host-target based. Unlike TCP/IP connections, which are full- duplex and can be closed by both clients and servers, SCSI connections can only be terminated (gracefully) by clients. If a server were to terminate a connection, it implies that service has been lost (or a hard disk has crashed). Servers do not keep connection information forever, and will drop relevant connections if its “keep-alive” to a particular client should fail for some reason. Through the use of hashing, encryption and security methods, connections are protected against attacks from hackers by arbitrarily using such command.
III. SELECTED KEY FEATURES OF HYPERSCSI HyperSCSI is a new open source network storage protocol designed for the transmission of SCSI commands and data across a network. It can allow one to connect to and use SCSI
and SCSI-based devices (include IDE, USB, Fibre Channel) natively over a network as if it was directly attached locally.
This section focuses on a few key features of the HyperSCSI protocol, and how they differ from existing solutions.
A. Device Discovery Mechanisms – Using the HyperSCSI Group Name
To identify and locate storage devices, Fibre Channel uses the World Wide Name (WWN) mechanism while iSCSI uses iSNS [11]. Such mechanisms are complex and add another hindrance to achieving ease of use and even plug-and-play networking. For this purpose, especially in home environment, HyperSCSI uses a standard Ethernet broadcast mechanism for device discovery but adds a Group Name to segregate servers and clients. This allows clients to dynamically locate targets on the network.
If a server is configured to be in the same group with a client, it will respond appropriately, otherwise the device discovery request is ignored. Thus the only configuration users have to be concerned about is granting permissions, rather than setting up complex name servers of some type. This is particularly useful in a plug-and-play style environment.
HyperSCSI clients will then attempt to connect to the servers in the groups that are assigned to them.
For instance, as shown in Fig. 7, Server A has 2 disks and Server B has 1 disk that can be exported. Server A exports Disk 1 to “Group ABC” and Disk 2 to “Group DEF”. Server B also exports its one disk to “Group DEF”. Client X can access Disk 1 of Server A only, since it has access to “Group ABC”.
However, Client Y being configured to “Group DEF” can see both Disk 2 on Server A as well as Disk 1 on Server B.
Groups are secured through HyperSCSI’s authentication mechanisms. This example shows that HyperSCSI Group Names are flexible and easy to use.
B. Data Security
In order to provide a secure data transfer, conventional TCP/IP based storage network protocols, like iSCSI leverage IPsec to protect connection as an independent function. IEEE 802.11 WLAN adopts a Wired Equivalent Privacy (WEP)
SCSI
iSCSI + TCP/IP
Ethernet
Reliability Functions Reliability Functions ReliabilityFunctions
SCSI
HyperSCSI
Ethernet
Reliability Functions
Fig. 8. Reliability Comparison
Fig. 9. HyperSCSI 802.11b Test System Configuration protocol to protect wireless link from eavesdropping and other
attacks. However, use these methods, adds to the complexity and implies securing the entire connection.
HyperSCSI, on the other hand, has integrated security options to be specified by individual devices instead of at the connection level. HyperSCSI supports the generally accepted encryption algorithm, Rijndael, which has a long security key (128bit).[2] It also adopts the HMAC method to protect the data integrity. A feature of HyperSCSI is that these security features can be configured by user based on the application purposes. Thus it can help to achieve better balance level between security and channel efficiency for home multiple data storage applications environment.
During the connection initialization stage, the HyperSCSI server will authenticate the client, and decide whether or not to export the storage resource by using the HyperSCSI Group Name and connection password. Once the authentication succeeds, a security key is exchanged between the server and client pair. HyperSCSI allows for security to be modularized into different levels of requirements such as data hashing, encryption or none at all, thereby giving even more options to secure (or not) the device and/or the connection. By integrating the different security functions, HyperSCSI can provide more efficient, flexible and secure methods for network storage.
C. Reliability
As shown in Fig. 8, each individual layer has its own functions to ensure the reliability of data transfer. The SCSI layer has the functions to check and ensure that SCSI data and commands are transferred properly. TCP/IP guarantees that the network connection is reliable, while the Ethernet layer also has its own features to check data correctness.
IP storage protocols, like iSCSI, rely solely on the TCP/IP layer for reliable data transfer. However, HyperSCSI makes use of the SCSI layer to ensure that SCSI data and commands are transferred properly and the Ethernet layer to check packet correctness. In addition, HyperSCSI provides its own reliable control mechanisms, such as flow control and packet
retransmission. Hence, by combining the features of different layers, the entire system with HyperSCSI is as reliable as one using TCP/IP based encapsulation. However, HyperSCSI is likely to be more efficient by working with functions that already exist in other layers.
IV. EXPERIMENTAL RESULTS AND PERFORMANCE ANALYSIS
HyperSCSI, as a storage network protocol, transfers data in a simple and efficient manner. Based on design and protocol specification [1],[2], we implemented HyperSCSI protocol over Fast Ethernet and IEEE 802.11b WLAN to verify whether HyperSCSI performs as intended in wireless environment. As such, we ran a battery of tests and benchmarks. The results so far prove to be most encouraging.
A. Description of the test environment
For the test environment, we set up a HyperSCSI client and a server (or often called initiator and target in the storage world) connected either through Fast Ethernet switch, or by a wireless access point. In wireless test environment, the client is connected to the wired network (Fast Ethernet) while the server is connected to the wireless network via a Cisco wireless client adapter (Aironet 350 Series). The access point is placed in an enclosed area at approximately 10 meters away from the server. Both the client and server run RedHat Linux 7.3 using the standard Linux kernel version 2.4.18-5 with the Linux Extended File system (ext3). The HyperSCSI server contains an Adaptec 39160 Ultra160 SCSI controller attached to one IBM Ultrastar DDYS-T09170 9GB 10000 RPM SCSI drive. The version of HyperSCSI code we ran is 20020808.
We used the hdparm, dd3, cp (file copy) and iperf programs as benchmark tools. The iperf is an application program used to benchmark TCP network data transfer performance. The version we used is 1.6.2. For the tests involving copy/read data from disk, the destination (copy to) used was /dev/null. Fig. 9 shows the test system configuration of 802.11b (WLAN) test systems. All the results were obtained by averaging the output from running the same test five times.
B. HyperSCSI over Fast Ethernet
Our first test is to investigate if HyperSCSI can fully utilize a simple channel, like Fast Ethernet efficiently. The results of this test shows that even Fast Ethernet can provide network storage efficiently.
3 “hdparm” and “dd” are disk test utilities provided in Linux environment
.
12.11 12.12
0.00 2.00 4.00 6.00 8.00 10.00 12.00
MB/s
dd cp
HS FE-UNI-SINGLE
Fig. 10. HyperSCSI Fast Ethernet Performance
5.19 5.17
0 1 2 3 4 5 6
cp dd
Mbps
Fig. 11. HyperSCSI on 802.11b Performance
In this test, both the HyperSCSI server and client use an Intel Pentium III 1GHz CPU, 32bit 33MHz PCI bus, 256MB 133MHz SDRAM. The Network Interface Card (NIC) used is a 3Com 3C905B-TXNM card and the switch is Cisco Catalyst 3500 XL with 24 ports for 10/100 Fast Ethernet.
There are two kinds of test results shown in Fig. 8, dd and cp. For the dd test, the data set chosen is 1GB of raw data while for cp test, the file used is a large MPEG file of size 616137020 Bytes. By using large data sets, we hope to minimize or eliminate the effects of local cache on the test results. The maximum Ethernet frame size is 1500 Bytes.
Using the HyperSCSI protocol, the user data transfer rate reaches up to 12.11 MB/sec.
In order to understand the channel utilization, we can conduct a theoretical calculation. From the Ethernet standard [12], one frame contains an 8-byte preamble, a 14-byte header, a 4-byte CRC, a 12-byte inter-frame gap and 1500 bytes of data. With the HyperSCSI protocol, every packet has a 3-byte HyperSCSI header. Thus the ratio of the bandwidth available for HyperSCSI user data is 1497 out of 1538. Considering the channel bandwidth is 100Mbit/sec, the theoretical boundary for HyperSCSI is 12.16 MB/sec.
However, in actual implementations, various overheads need to be factored in. This includes the protocol overhead, SCSI block data that may not always be fragmented into a full Ethernet frame size and so on. But as our measured results are close to the theoretical limits, we conclude that HyperSCSI is indeed efficient, from both theoretical calculations and actual measurement.
C. Performance of HyperSCSI over Wireless LAN (IEEE 802.11b)
In this test, we investigated whether HyperSCSI can fully utilize a simple wireless channel, such as the IEEE 802.11b
(WLAN) efficiently. Due to the bandwidth limitation of the IEEE 802.11b, actual user data transfer rate can reach only about 5.2 Mbps [15]. The purpose of this test is therefore to run HyperSCSI protocol over wireless channel to find what data transfer rate can be achieved.
In the test, both HyperSCSI server and client used a computer with an Intel Pentium III 1GHz CPU, 32bit 33MHz PCI bus and 256MB 133MHz SDRAM. The Network Interface Card (NIC) used was a 3Com 3C905B-TXNM card and the Wireless Client Adapter used was a Cisco Aironet 350 series.
There were two kinds of test results for dd and cp as shown in Figure 8. Both data set of raw data for dd test and file size for cp test were 51.2MB. The maximum Ethernet frame size was 1500 Bytes. By using HyperSCSI protocol, the average user data transfer rate for dd and cp can reach up to 5.17 MB/sec and 5.19 MB/sec, respectively.
These results are close to that obtained from the theoretical analysis model [15], which shows that little overhead is applied after running HyperSCSI protocol on wireless channel. With HyperSCSI, mobile devices can access the networked storage more efficiently, and many data intensive applications can be exploited to meet user requirements.
D. Performance of TCP over Wireless LAN (802.11b)
TCP is a protocol widely used in the existing network infrastructure developed for conventional network applications, especially in the wired environment. We conduct the test for TCP over WLAN environment by using TCP benchmark tool, iperf. Both server and client used the same setup configuration that was used for HyperSCSI on Wireless LAN benchmark test. The Ethernet frame size was 1500 bytes.
The average data transfer rate of TCP was 4.9 MB/sec. The value is shown in Fig. 12, together with HyperSCSI benchmark results. Note there is an important difference with the previous HyperSCSI test. The TCP test was conducted purely in the RAM level between two computers, while the HyperSCSI test was done as reading data from real SCSI storage, which generates more data input/output processing requirements. Such kind of processing must be provided in
Fig. 13. HyperSCSI Demonstration for Home Wireless Environment
5.17 5.19 4.90 5.15
0 1 2 3 4 5 6
iperf hdparm dd cp
Mbps
Fig. 12. HyperSCSI and TCP Performance Comparison
order to deliver real storage data to user. However, the purpose of this test is to find the wireless channel performance with TCP, there was no data input/output processing in our test situation.
We conclude from this experiment that HyperSCSI is capable of achieving faster data accessing rate from storage disk than TCP data accessing from computer RAM. This proves that by using a simplified data transport protocol, the storage system can allocate resource and processing power to do more jobs, such as processing data from real storage disk, without reducing the performance. Furthermore, desktop PCs were chosen to do the benchmark test in our lab, which were relatively powerful compared to the mobile devices currently used in wireless environment. We believe that if HyperSCSI protocol is applied on a low-end mobile device, where the computer processing power is a limiting factor for system performance, the difference between HyperSCSI and TCP will be even more significant.
V. APPLICATIONS AND DISCUSSIONS
The development of an Ethernet based home network grows at stunning speed in recent years. Together with IEEE 802.11 WLAN technology, capability of home device access and control can be extended to a broad range [3],[13],[14].
However, applying the concept of storage network to such environment is not straightforward. In order to support an home user to access the storage devices natively as if those devices are attached locally, there is a demand to apply a simple and efficient network enabled storage protocol on top of existing network infrastructure. By doing this, home storage devices will release great power after they can access and manage large amounts of information (or storage) at home, especially via wireless network.
Here we introduced some applications, which apply HyperSCSI into IEEE 802.11b/g based home wireless environment. One application is the set-up of a media player (shown in Fig. 13), which is called NetDVR. It is a separate box with a hard disk drive and a CD/DVD/RW recorder inside, which is connected to a TV and a remote PC. In this setup, the prototype is connected to the network via wireless adapter cards. This box works as HyperSCSI storage server that keeps audio and video data into its disk. It can directly
playback the audio and video to the TV attached to it or over the WLAN to a remote PC or Wireless Webpad, which function as HyperSCSI clients.
There is another application that was demonstrated with the prototype system. One CD/DVD/RW recorder is attached to NetDVR. With HyperSCSI protocol, the remote computer will detect this recorder over the home network, therefore, the remote user can record a CD/DVD disc without locally attach such a disc recorder.
These applications are useful to home users for entertainment purposes. For example, a user might want to watch a movie or listen to music while he is in the garden and the player is in the living room. He may also share the drive or disk in the basic storage access level, such as record a CD disc over the network simultaneously. In the lab, we also demonstrate the real time DVD video playback over 802.11g connections. Our next effort is to provide quantitative results for the 802.11g platform.
There is one concern for HyperSCSI, which can be argued as either advantage or disadvantage. HyperSCSI eliminates TCP protocol for the network data transfer, therefore, it becomes a local area storage access protocol, while iSCSI is claimed to support wide area storage access. As was shown by the measurement results both in this paper and in a previous study [4], if the storage access is maintained within the local area environment, HyperSCSI will deliver better performance.
This will be interoperated as the requirement of system with lower processing power, especially, for a mobile device, the cost will be reduced.
As home networking is the area which is gaining popularity in recent years, more and more people are beginning to have small networks at home. We will foresee the majority of mobile storage access requirement would be within the local environment. Therefore, HyperSCSI will be an efficient solution to meet the home storage access requirement.
VI. CONCLUSIONS
In this paper we have presented HyperSCSI as a new open source network storage protocol, which can be applied to the home environment. It is developed upon the standard SCSI storage interface and can be used to access generic devices, using IDE or USB, over the Ethernet platform. It is targeted to
utilize common-off-the-shelf storage hardware and well- deployed network infrastructure yet provide users with performance, flexibility and security. Based on the benchmark experiment and application exploitation, we believe that HyperSCSI, as a light-weight protocol, is efficient and cost- effective to be used for home storage based applications, thus makes it a viable solution for home networks.
ACKNOWLEDGMENT
This work is supported by Data Storage Institute as part of a research project. The authors would like to thank various people who have contributed tremendously in conducting this work, including Yeo Heng Ngi, Ng Tiong King, Patrick Khoo and Daniel Khoo. The authors also like to thank Prof. Kees A.
Schouhamer Immink for the valuable comments, who is a visiting professior in Data Storage Institute.
REFERENCES
[1] Patrick B. T. Khoo, Wilson Y. H. Wang, “Introducing A Flexible Data Transport Protocol for Network Storage Applications”, 10th Goddard Conference on Mass Storage Systems and Technologies and 19th IEEE Symposium on Mass storage System. pp. 241-257, Apr. 2002.
[2] “HyperSCSI Protocol Specification”, Network Storage Technology Division, Data Storage Institute, Singapore, http://www.dsi.a- star.edu.sg/research/hyper.html.
[3] Yasser Rasheed, John Ritchie, “High-Quality Media Distribution in the Digital Home,” Intel Technology Journal, Vol. 6, Issue 4, pp.17-29, 2002.
[4] Tatsuo Nakajima, Daiki Ueno, Eiji Tokunaga, Hiro Ishikawa, Ichiro Satoh, “A Virtual Overlay Network for Integrating Home Appliances,”
Applications and the Internet, 2002. (SAINT 2002). Proceedings. 2002 Symposium on , 28 Jan.-1 Feb. 2002.
[5] Amit Dhir, Saeid Mousavi, “Home Networking Using ‘New Wires’ — IEEE 1394, USB, and Fast Ethernet Technologies,” White paper WP134 (v1.0), http://www.xilinx.com March 21, 2001.
[6] Tajika, Y., Saito, T., Teramoto, K., Oosaka, N., Isshiki, M., “Networked home appliance system using bluetooth technology integrating appliance control/monitoring with internet service,” Consumer Electronics, IEEE Transactions on , Volume: 49 , Issue: 4 , pp. 1043 - 1048, Nov. 2003.
[7] Julie Jacobson, “Networking Consortium Conspicuously Quiet on 1394,” CE Pro Editorial, http://www.homenetnews.com/
pages/Main/HNNEditorials.htm, Aug. 2003.
[8] A. Benner, “Fibre Channel: Gigabit Communications and I/O for Computer Netowrks,” McGraw-Hill, 1996.
[9] Prasenjit Sarkar, Kaladhar Voruganti, “IP Storage: The Challenge Ahead”, 10th Goddard Conference on Mass Storage Systems and Technologies and 19th IEEE Symposium on Mass storage System. pp.
35-42, Apr. 2002.
[10] Kalman Z. Meth, Julian Satran, “Features of the iSCSI Protocol,”
Communications Magazine, IEEE , Volume: 41 , Issue: 8, pp. 72- 75, Aug. 2003.
[11] IPS Working Group Internet Engineering Task Force, “Internet Storage Name Service Internet Draft”, August 2002.
[12] Mitchell L. Loeb, Andrew J. Rindos, William G. Holland, and Steven P. Woolet, “Gigabit Ethernet PCI Adapter Performance”, IEEE Network, Volume: 15 Issue: 2, pp. 42-47, March 2001.
[13] Brian P. Crow, Indra Widjaja, Jeong Geun Kim, Prescott T. Sakai,
“IEEE 802.11 Wireless Local Area Networks”, IEEE Communications Magazine, pp. 116-126, September 1997.
[14] Chris Heegard, John T. Coffey, Srikanth Gummadi, Peter A. Murphy,
“High-Performance Wireless Ethernet”, IEEE Communications Magazine, pp. 64-73, November 2001.
[15] Jim Lansford, Wade Gillham, “Alphabet Soup: Tradeoffs Between 802.11b, 802.11a, and 802.11g,” Microsoft Windows Hardware Engineering Conference, WinHEC 2002.
[16] George Xylomenos, George C. Polyzos, “TCP Performance Issues over Wireless Links, ” IEEE communications Magazine. April 2001.
Wilson Yong Hong Wang (M’99) received the B.Eng degree in information engineering and M.Eng degree in computer communications all from XIDIAN University, China. He is currently a senior research engineer in Data Storage Institute, and is also working towards his Ph.D degree in the Department of Electrical and Computer Engineering, the National University of Singapore (NUS). His research interest is in the field of network storage, optical data storage and signal processing.
Tow Chong Chong (M’91) received the B.Eng. degree from the Tokyo Institute of Technology, Tokyo, Japan, the M.Eng. degree from the National University of Singapore, and the Sc.D. degree from the Massachusetts Institute of Technology, Cambridge, all in electrical engineering. He is currently the Director of the Data Storage Institute under the Agency for Science, Technology & Research and hosted by NUS. His research interest is in the field of magnetic and optical data storage as well as network storage. He is also a Professor with the Department of Electrical and Computer Engineering, NUS. He has authored and co-authored more than 120 publications in international refereed journals, presented 20 invited talks, and held 16 patents. He served as co-chairman of APMRC2002 and as a member of the Technical Program Committee for ODS (USA), ISOM (Japan), MORIS (Japan), CLEO Pacific (USA), and OECC (Japan).
