1 3
2
Fundamentals and
Design Principles
This c hapter presents the fundamental axio ms and princ iples o f high-speed netwo rk design that will set the stage fo r the rest o f the bo o k. Sec tio n 2.1 pre-sents a brief histo ry o f netwo rking and c o nsiders the defining c harac teristic s o f fo ur generatio ns o f netwo rking. We emphasize the impo rtanc e o f kno wing the past in preparing fo r the future. Sec tio n 2.2 disc usses high-perfo rmanc e distributed applic atio ns as the mo tivatio n fo r high-speed netwo rks, presents an ideal netwo rk mo del that expresses the desired c harac teristic s, and dis-c usses the dis-c o nstraints that dis-c hallenge this ideal.
All o ur effo rts to design high-perfo rmanc e netwo rks are attempts to get as c lo se to the ideal high-bandwidth, lo w-latenc y netwo rk mo del as po ssible. Given the c o nstraints o f the real wo rld, we c an appro ac h but never reac h this ideal.
The fundamental princ iples intro duc ed in Sec tio n 2.3 express the c o ndi-tio ns under whic h prac tic al netwo rk arc hitec ture and design c an mo st c lo sely appro ac h this high-perfo rmanc e netwo rk ideal. These princ iples will be used as the basis fo r c o ntext-spec ific versio ns in the c hapters that fo llo w.
path, reso urc e tradeo ffs, the end-to -end argument, pro to c o l layering, and the management o f state.
Sec tio n 2.4 intro duc es the mo st impo rtant design tec hniques that will be used thro ugho ut the bo o k to apply these princ iples to ward o ur ideal net-wo rk go al. These inc lude sc aling time and spac e, masking and c heating the speed o f light, spec ialized hardware implementatio n, parallelism and pipe-lining, data struc ture o ptimizatio n, and c ut-thro ugh and remapping. Sec tio n 2.5 c o nc ludes with a list o f the axio ms and princ iples intro duc ed in this c hapter.
A no te o n termino lo gy: We use the terms end systemand hostinterc hange-ably fo r a c o mputing platfo rm attac hed to the netwo rk fo r the purpo se o f run-ning applic atio ns. We use the term nodefo r a system within the netwo rk that perfo rms data ro uting, switc hing, o r fo rwarding func tio ns. Thus a no de c an be a switc h, ro uter, bridge, o r link repeater; these terms will be used when the distinc tio n matters. We use the term linkto designate the c o nnec tio ns be-tween no des. We use the term pathto generic ally indic ate a sequenc e o f no des and links thro ugh the netwo rk o n whic h data o r c o ntro l info rmatio n flo ws. A path c an be a virtual c irc uit, c o nnec tio n, flo w o f datagrams, o r data-gram transac tio n; these individual terms will be used when the distinc tio n matters.
2.1
A Brief History of Networking
The ability to make pro gress in sc ienc e and engineering depends o n under-standing the past. Thus, we will begin with the first part o f o ur first axio m:
When an o ld idea surfac es in a new c o ntext, the impo rtant thing is to rec o g-nize what makes this c o ntext different fro m the c o ntext in whic h the idea’s past resides. This suggests the fo llo wing:
Know the Present ideas look different in the present because the context in which they have reappeared is different. Understanding the difference tells us which lessons to learn from the past, and which to ignore.
Ø2
Know the Past Genuinely new ideas are extremely rare. Almost every “ new” idea has a past full of lessons that can either be learned or ignored.
A final c o ro llary is required to c o unter the hubris engendered by all o f this “kno wing”:
The histo ry o f netwo rking and c o mmunic atio ns c an be divided into gen-eratio ns [Sterbenz 1995], as summarized in Table 2.1. Eac h gengen-eratio n has
Know the Future The future hasn’t happened yet, and is guaranteed to contain at least one completely unexpected discovery that changes everything.
[image:3.612.99.460.220.601.2]Ø3
Table 2.1 Com puter Netw ork Generations
TI M E PERVASI
VE-GENERATI ON FRAM E APPLI CATI ONS TECH NOLOGY NESS
1. Em ergence Through Voice Circuit-sw itched PSTN Universal 1970s over copper w ire
RF
Entertainm ent Broadcast RF Universal Data Serial link over copper Very lim ited
Modem
2. Internet 1980s Voice Digital-sw itched PSTN Universal Cellular m obile
telephony
Entertainm ent CATV over copper coax High Data Store-and-forw ard packet Corporate
enterprise Gatew ayed subnetw orks Research and
education Transoceanic fiber optic
cables
Bent-pipe satellite links
3. Convergence 1990s Web Fast electronic packet Very high sw itching over fiber
Multim edia IP-based global Internet (First w orld) Sw itched satellite
netw orks
4. Scale 2000⫹ Mobile Wireless access netw ork Global Ubiquity Ubiquitous All optical netw orks
(sw itches and fiber)
Mobility Widely deployed
distinc tly different c harac teristic s in terms o f the sc o pe o f users and applic a-tio ns, the integraa-tio n o f vario us applic aa-tio ns and media, and the enabling tec h-no lo gy and tradeo ffs.
2.1.1
First Generation: Emergence
The first generatio n lasted thro ugh ro ughly the 1970s and is c harac terized by three distinc t c atego ries: vo ic e c o mmunic atio n, bro adc ast entertainment, and data netwo rking, eac h o f whic h was c arried by a different infrastruc ture. Vo ic e c o mmunic atio n was either analo g c irc uit switc hed o ver c o pper wire in the public switc hed telepho ne netwo rk (PSTN), o r analo g radio transmissio n between transc eivers. Entertainment bro adc asts to radio rec eivers and televi-sio ns were c arried by free spac e bro adc ast o f radio frequenc y (RF) transmis-sio ns. Data c o mmunic atio ns was the latest entrant, and pro vided o nly a means to c o nnec t terminals to a ho st. This was ac c o mplished either by serial link lo c al c o mmunic atio ns (fo r example, RS-232 o r Binary Sync hro no us Co mmuni-c atio ns used o n mainframes), o r by mo dem mmuni-c o nnemmuni-c tio ns o ver telepho ne lines fo r remo te ac c ess; in bo th c ases c o pper wire was the physic al medium. Pac ket netwo rking began to emerge in the wired netwo rk in ARPANET1and pac ket radio , primarily fo r military applic atio ns.
2.1.2
Second Generation:
The Internet
In the 1980s a dramatic jump in the types and sc o pe o f netwo rking o c c urred, but the three c atego ries o f c o mmunic atio n (vo ic e, entertainment, and data) re-mained relatively distinc t. This perio d to o k us fro m the experimental
ARPANET to the ubiquito us Internet.
While the end user o f the vo ic e netwo rk generally c o ntinued to use analo g telepho ne sets, the internal netwo rk switc hes and trunks bec ame largely digi-tal. Transmissio n remained mo stly o ver c o pper wire. Additio nally, there was widespread deplo yment o f digital private branc h exc hange telepho ne switc hes (PBXs) o n large c usto mer premises. Mo bile c o mmunic atio ns emerged in the fo rm o f c ellular telepho ny. The signific ant additio n to the entertainment c ate-go ry o f netwo rking was the wide-sc ale deplo yment o f c able televisio n (CATV) netwo rks fo r entertainment video o ver c o pper c o axial c able.
In data netwo rking, we first saw the emergenc e o f c o nsumer ac c ess, but in the primitive fo rm o f bulletin bo ard systems (BBSs) and c o nsumer o nline ser-vic es (suc h as Americ a Online, Co mpuServe, and Pro digy). These were essen-tially first-generatio n netwo rks made available to c o nsumers, with mo dems
c o nnec ting to a c entral server farm. The c o llec tio n o f researc h and educ atio n netwo rks suc h as BITNET, CSNET, and UUNET, were c o llec tively referred to as the Matrix [Quarterman 1989] befo re they began to jo in the Internet, unified by IP addresses, with the do main name system (DNS) symbo lic addressing re-plac ing bang paths.2
The gro wth in data netwo rking fo r universities and the research co mmunity was significant during this perio d, fo r purpo ses o f file transfer, remo te lo gin, electro nic mail, and Usenet news. The techno lo gy emplo yed was the packet-switched Internet utilizing the TCP/IP pro to co l suite. In the wide area, the backbo ne netwo rk co nsisted o f sto re-and-fo rward ro uters co nnected by leased 56-Kb/s telepho ne lines. The NSFNET3upgraded the infrastructure to 1.5-Mb/s T1 lines, and ultimately 45-Mb/s T3 lines at the end o f the seco nd generatio n. In the lo cal area, shared medium Ethernet and to ken ring netwo rks allo wed clus-ters o f wo rkstatio ns and PCs to netwo rk with file and co mpute servers.
At the same time, but relatively distinc t fro m this, c o nnec tio n-o riented c o r-po rate enterprise netwo rks using pro to c o ls suc h as BNA, DECNET, and SNA4 were widely deplo yed, alo ng with the deplo yment o f public X.25 netwo rks (used primarily as c o rpo rate virtual private netwo rks). Thus, even within data netwo rking there were multiple inc o mpatible arc hitec tures and po o rly inter-c o nneinter-c ted netwo rks. Mo st o f these netwo rks used inter-c o pper wire as the predo m-inant physic al medium.
It is impo rtant to realize that pio neering wo rk was do ne in the c o mputer ar-c hitear-c ture and o perating systems in the 1960s and 1970s, and signifiar-c ant net-wo rking researc h and develo pment were do ne in the c o ntext o f the enterprise netwo rk arc hitec tures and pro to c o ls. The fo undatio ns o f switc hing date far-ther bac k, to the emergenc e o f relay-c o ntro lled railway interlo c kings and tele-pho ne switc hes. Thus, we state the first c o ro llary to the Kno w the Past axio m:
2.1.3
Third Generation:
Convergence and the Web
The third generatio n saw the emergenc e o f integrated servic es: the merging o f data, vo ic e, and entertainment video o n a single netwo rk infrastruc ture. With
Not Invented Here Corollary [1980s version] Operating systems didn’t begin with Unix, and networking didn’t begin with TCP/ IP.
Ø-A
2Bang path addresses required explic it mentio n o f all ho st systems in the path, fo r example,
decvax!ihpn4!wucs!axon!jpgsrather than the c urrent fo rm jpgs@acm.org. 3Named after the Natio nal Sc ienc e Fo undatio n, the funding spo nso r.
the advent o f IP-telepho ny gateways, the PSTN started to bec o me a subnet o f the Internet. Netwo rk servic e pro viders sc rambled to keep c apac ity ahead o f demand in o verpro visio ned netwo rks, sinc e the quality o f servic e (QOS) mec hanisms to suppo rt real-time and interac tive applic atio ns were just begin-ning to emerge.
The sec o nd generatio n was c harac terized by the pac ket-switc hed Internet, X.25, and enterprise netwo rks. The third generatio n is c harac terized by an IP-based glo bal info rmatio n infrastruc ture (GII) IP-based o n fast pac ket-switc hing tec hno lo gy interc o nnec ted by fiber o ptic c able, and IP as the single netwo rk layer unifying previo usly disjo inted netwo rks.
The sec o nd signific ant c harac teristic is in the sc o pe o f ac c ess, with c o n-sumers universally5ac c essing the Internet with perso nal c o mputers via Inter-net servic e pro viders (ISPs). Disc o nnec ted BBSs are a thing o f the past, and o nline c o nsumer servic es have bec o me merely value-added versio ns o f ISPs. The Internet went fro m being a kilo bit kilo no de netwo rk, thro ugh megabit megano de, appro ac hing gigabit gigano des.
The final distinguishing c harac teristic o f the third generatio n is the Wo rld Wide Web, whic h pro vided a c o mmo n pro to c o l infrastruc ture (HTTP), display language (HTML), and interfac e (Web bro wsers) to enable users to easily pro -vide and ac c ess c o ntent. Web bro wsers bec ame the way to ac c ess no t o nly data in web pages but also images and streaming multimedia c o ntent. The rate o f ado ptio n o f Internet c o nnec tio ns vastly exc eeded the rate o f new telepho ne c o nnec tio ns. In spite o f the fac t that the Internet and Web bec ame the primary reaso n fo r users to have PCs, these devic es were still no t designed with net-wo rking as a signific ant arc hitec tural c o nsideratio n.
Just as we needed to lo o k beyo nd Unix in the 1980s, we need to lo o k be-yo nd what bec ame the mo st c o mmo n c o mputing platfo rm in the 1990s: the Intel x86 arc hitec ture PC running Mic ro so ft Windo ws.
Ho wever, in this c ase, the marketing mo del has limited the impac t. Re-searc hers require an o pen platfo rm with freely available so urc e c o de fo r the o perating system and pro to c o l stac k, and thus Unix variants are still the plat-fo rm o f c ho ic e plat-fo r the vast majo rity o f netwo rking researc h. The lac k o f o pen interfac es and so urc e c o de o n c o mmerc ial ro uters and switc hes is a
signifi-Not Invented Here Corollary [1990s version] Operating systems didn’t begin with Windows, and host architecture didn’t begin with the PC and the x86 architecture.
Ø-B
c ant hindranc e to the researc h, develo pment, and trial deplo yment o f new pro to c o ls and algo rithms.
2.1.4
Fourth Generation: Scale,
Ubiquity, and M obility
The first dec ade o f the new millennium inaugurates a new netwo rk genera-tio n, whic h will apparently be c harac terized by o rders o f magnitude inc rease in netwo rk sc ale, and by the ubiquity o f mo bile wireless c o mputing devic es. The third generatio n was largely a wired netwo rk; the fo urth generatio n will be largely wireless at the edges, with ac c ess to a high-speed o ptic al bac kbo ne infrastruc ture, inc luding o ptic al switc hes.
Advanc es in the interfac e between bio lo gic al and mic ro elec tro nic inter-fac es, and the benefits o f applying mic ro - and mac ro bio tic behavio r and o rga-nizatio n tec hniques to netwo rks may begin to emerge, o r may be a
c harac teristic o f the fifth generatio n.
Ubiquito us c o mputing, smart spac es, and senso r netwo rk researc h suggest that individual users will c arry tens to hundreds o f netwo rked devic es. These will perhaps inc lude a perso nal no de fo r c o mputing and c o mmunic atio ns to o ther individuals, and a wireless netwo rk o f wearable I/O devic es and senso rs (enviro nmental and bio lo gic al). Ro o ms and vehic les will c o nsist o f perhaps tho usands o f embedded senso rs and netwo rked c o mputing platfo rms. Thus we need to c o nsider terano de and petano de netwo rks. There are pro fo und im-plic atio ns to high-speed netwo rking. The aggregate wireless bandwidth de-manded will vastly exc eed the c apac ity o f the shared medium using
third-generatio n pro to c o ls and tec hniques, and the highly dynamic nature will stress ro uting and c o ntro l pro to c o ls. The ability to manage po wer amo ng the massive number o f auto no mo us wireless devic es and to do high-speed net-wo rking where po wer is a c o nstraint will be a majo r c hallenge.
We will see no t o nly end systems, the so le purpo se o f whic h is netwo rking, but also a blurring o f func tio nality between end systems and netwo rk no des. In mo bile netwo rking, many devic es will serve bo th as applic atio n platfo rms and as switc hes o r ro uters.
While the c apac ity in pro c essing, memo ry, and bandwidth will dramatic ally inc rease, the tradeo ffs will c o ntinue to shift. This is the subjec t o f Chapter 8. The relative dec rease in the c o st o f pro c essing enabled the field o f ac tive net-wo rking, whic h may play a signific ant ro le in the fo urth generatio n. We no te that speed-o f-light latenc y will c o ntinue to be a signific ant c hallenge, and in-c reasingly so as we begin to build the Interplanetary Internet, initially fo r the Mars missio ns, but with an eye to ward the Jupiter system.6
So we end with this fo urth part o f o ur first axio m, whic h amplifies the third part:
While we c an’t predic t exac tly what will be the next killer app, o r what fun-damental disc o veries will take plac e, we c an do o ur best to prepare fo r what is likely to c o me, and be prepared to adapt to the things we c an’t predic t.
2.2
Drivers and Constraints
This sec tio n desc ribes the fundamental drivers and c o nstraints fo r high-perfo rmanc e netwo rk infrastruc ture. High-high-perfo rmanc e c o mmunic ating appli-c atio ns are the driving reaso n to pro duappli-c e high-speed netwo rks. The go al we wo uld like to ac hieve is an ideal netwo rk with unlimited bandwidth and no la-tenc y, to allo w delay-free transfer o f data amo ng applic atio ns. If it were po ssi-ble to easily ac hieve the ideal netwo rk, there wo uld be no need fo r this bo o k. Unfo rtunately, there are a number o f c o nstraints that make it diffic ult to ap-pro ac h an ideal netwo rk.
2.2.1
Applications
Applic atio ns are the who le po int o f do ing netwo rking:
This do esn’t mean that we c an predic t the next killer app to determine the direc tio n o f high-speed netwo rking. In fac t, being to o narro w in applic atio n target c an just as easily lead us do wn the wro ng path when subsequent killer apps do appear. Thus, it is impo rtant to researc h, design, and deplo y high-speed netwo rk infrastruc ture, and see what applic atio ns will ac tually emerge.
Application Primacy The sole and entire point of building a high-performance net-work infrastructure is to support the distributed applications that need it.
I Prepare for the Future Simply knowing the past does not prepare us to understand the future. We must constantly reevaluate tradeoffs in the face of emerging technol-ogy and question the basic assumptions of the present.
Thus, we have a “field-o f-dreams” versus “killer-app”7pro blem. In so me c ases go vernment agenc ies are willing to build the infrastruc ture o n the antic i-patio n that “If yo u build it, [they] will c o me” [Ro binso n 1989]. An example o f this was the NSFnet and high-perfo rmanc e c o mputing and c o mmunic atio ns (HPCC) initiatives, funded by the U.S. go vernment.
The mo st impo rtant distributed applic atio n metric o f interest to us is delay.
If c o mmunic ating applic atio ns are able to transfer data with essentially zero delay, then they behave like applic atio ns running o n a single c o mputing sys-tem, and we do no t have a high-speed netwo rking pro blem to so lve. In reality, we must be c o nc erned with applic atio ns that c o mmunic ate with o ne ano ther ac ro ss signific ant distanc e thro ugh a netwo rk. The interapplic atio n delay in-tro duc ed by suc h a netwo rk c o nsists o f two c o mpo nents:
1. The delay invo lved in transmitting the bits at a partic ular rate. Fo r a c hunk o f data bbits in size alo ng a path o f bandwidth rbits/s, the delay8 is td⫽b/r. By inc reasing the bandwidth o f the path, we c an reduc e this
c o mpo nent o f latenc y. This delay o c c urs at least o nc e, when the c hunk o f data is transmitted by the sending applic atio n. It also o c c urs every ad-ditio nal time that it is transmitted o ut o f a sto re-and-fo rward ho p h, o r a per byte o peratio n takes plac e c(suc h as c o pies between buffers in memo ry o r manipulating in plac e), fo r a to tal o f 1 ⫹h⫹ctimes.
2. The delay dpalo ng a path c o nsists o f the latenc y inc urred as the bits flo w
alo ng a distanc e due to the speed o f light, and delays thro ugh transmit-ting end system, all o f the intermediate no des in the netwo rk, and the re-c eiving end system.
Interapplication Delay The performance metric of primary interest to communicat-ing applications is the total delay in transferrcommunicat-ing data The metric of interest to users includes the delay through the application.
I.2 Field of Dreams versus Killer App Dilemma The emergence of the next “ killer ap-plication” is difficult without sufficient network infrastructure. The incentive to build network infrastructure is viewed as a “ field of dreams” without concrete projections of application and user demand.
I.1
7Field o f Dreams: Build it and they will c o me, fro m the 1989 mo vie Field of Dreams; this variant o f c hic ken-and-egg is quo ted fro m Marjo ry Blumenthal.
Thus, the to tal interapplic atio n delay c an be expressed as D⫽(1 ⫹h⫹c)b/r⫹dp
To reduc e the delay between applic atio ns we need to be c o nc erned abo ut the bandwidth and latenc y o f a path, as well as the number o f ho ps o r c o pies, whic h affec t latenc y. We will revisit these fo rmulas and their c o mpo nents in mo re detail in subsequent c hapters.
Applic atio ns vary in their sensitivity to delay. So me applic atio ns have tight real-time delay requirements, suc h as remo te pro c ess c o ntro l and vo ic e. Inter-ac tive applic atio ns suc h as Web bro wsing have so mewhat lo o ser c o nstraints, but require delay bo unds o f subsec o nd with preferred bo unds o n the o rder o f 100 ms to be truly interac tive.
Other applic atio ns do no t have tight latenc y bo unds in the c o nventio nal sense, but the size o f the data is so signific ant that to be useful the bandwidth is the usual way o f thinking abo ut the requirement. Examples inc lude remo te bac kup o f data and netwo rk distributio n o f mo vies and televisio n c o ntent. In these c ases, to meet a prac tic al delivery c o nstraint o f minutes to ho urs, band-width must be high.
Fo r c ertain paths, espec ially tho se that inc lude erro r pro ne (fo r example, lo w signal-to -no ise ratio s) wireless links, link retransmissio ns c o ntribute to the path delay. This c o uld be viewed as an additio nal latenc y pro blem, o r as a lo wer effec tive bandwidth pro blem. The latter perspec tive is generally better, bec ause the pro blem respo nds better to bandwidth-intensive measures [fo r example, fo rward erro r c o rrec tio n (FEC) o ver greater bandwidth] than to la-tenc y reduc tio n attempts measures (fo r example, finding sho rter paths).
Finally, we need to c o nsider ho w impo rtant high-speed c o mmunic atio n is to the applic atio ns o n a partic ular system, and ho w this feeds into the design and c o st tradeo ffs.
While this is o bvio us fo r netwo rk c o mpo nents who se so le purpo se is to perfo rm c o mmunic atio ns pro c essing, it is frequently o verlo o ked in the design o f general-purpo se end systems suc h as PCs. Altho ugh a signific ant, if no t
Networking Importance in System Design Communication is the defining charac-teristic of networked applications, and thus support for communication must be an in-tegral part of systems supporting distributed applications.
I.4 Network Bandwidth and Latency Bandwidth and latency are the primary perfor-mance network metrics important to interapplication delay.
primary, applic atio n o f many c o nsumer PCs is to bro wse the Web, netwo rking perfo rmanc e is sec o ndary to pro c esso r c lo c k rate, memo ry size, and disk c a-pac ity in marketing-driven arc hitec ture. The rule o f thumb has been that a maximum o f 10 perc ent o f the c o st o f a PC c an be c o nsumed by the netwo rk interfac e, but this sho uld no t be a fac to r that c o nstrains the impo rtanc e o f net-wo rking in the mo dern net-wo rld.
2.2.2
The Ideal Network
We c an no w c o nsider what we want fro m the netwo rk (whic h fo r this disc us-sio n inc ludes the end systems running the c o mmunic ating applic atio ns). The ideal netwo rk is o ne that is c o mpletely transparent to c o mmunic ating applic a-tio ns. In partic ular, the netwo rk direc tly c o nnec ts applic aa-tio ns with unlimited bandwidth (B⫽앝) and no latenc y (D⫽0), as sho wn in Figure 2.1. Thus, the
applic atio ns c anno t tell the differenc e between exec uting o n the same ho st fro m exec uting o n different ho sts c o nnec ted by the ideal netwo rk.
This ideal, o f c o urse, is unattainable, and the po int o f this bo o k is to under-stand ho w to design and engineer netwo rks that appro ac h this ideal, and what the barriers are. We need to dec o mpo se this high-level ideal mo del to the next level o f granularity befo re we c an begin to understand ho w c lo se we c an c o me to the ideal c ase.
Figure 2.2 sho ws the ideal netwo rk arc hitec ture.
Applic atio ns run o n their respec tive CPUs o perate witho ut interfering with the interapplic atio n c o mmunic atio ns. The applic atio ns are c o nnec ted to o ne ano ther by the ideal netwo rk (B⫽앝and D⫽0), whic h is dec o mpo sed into a
set o f switc hes that fully interc o nnec t the end systems with no blo c king o f c o mmunic atio ns traffic . Clearly, the switc hes themselves must also have the c harac teristic s B⫽앝and D⫽0.
[image:11.612.99.463.509.598.2]In lo o king at the dec o mpo sed piec es (netwo rk, no des, end systems, endto -end pro to c o ls, and applic atio ns), we must remain ever mindful that the go al is to ac hieve an end-to -end high-perfo rmanc e path between applic atio ns. While these (and further) dec o mpo sitio ns are nec essary to get a handle o n the pro b-lem, they c anno t be do ne in iso latio n; this will be the subjec t o f Sec tio n 2.3.
Thus, we c an begin to see what we want the netwo rk to suppo rt: end-to -end high-perfo rmanc e paths between c o mmunic ating applic atio ns, in suppo rt o f lo w Interapplic atio n Delay (I.2).
Indeed, this is the fo c us o f the rest o f this bo o k, and we will examine ho w to ac hieve this in the vario us piec es: netwo rk arc hitec ture and o rganizatio n in Chapter 3, netwo rk no des (switc hes and ro uters) in Chapter 4, end systems in Chapter 5, end-to -end pro to c o ls in Chapter 6, and the applic atio n interfac e in Chapter 7.
We will intro duc e so me c o ro llaries to the High-perfo rmanc e Paths Go al that will help guide us in ac hieving the best arc hitec ture and design po ssible:
We need a mec hanism to disc o ver and establish high-perfo rmanc e paths, and to do it quic kly eno ugh fo r the applic atio n. In the c ase o f a single transac -tio n, this establishment time is part o f the delay in transferring data. In the c ase o f lo nger applic atio n asso c iatio ns suc h as c o llabo rative sessio ns, the
Path Establishment Corollary Signaling and routing mechanisms must exist to dis-cover, establish, and forward data along the high-performance paths.
II.1 High-Performance Paths Goal The network and end systems must provide a low-latency, high-bandwidth path between applications to support low interapplication delay.
II.
network
end system CPU
M app
end system CPU
app M
B = ∞
D = 0
[image:12.612.80.444.58.237.2]nodes
establishment time may have distinc t and mo re relaxed time c o nstraints, o n the o rder o f several sec o nds.
Onc e we have disc o vered a high-perfo rmanc e path, we may have to guaran-tee that it will remain that way fo r the life o f the applic atio n asso c iatio n. There are two ways to ac c o mplish this. We c an vastly o verengineer the net-wo rk, so that the pro bability o f traffic fro m different applic atio ns interfering and c o ngesting the netwo rk is insignific ant. While bandwidth is c heap, it is no t free, and sho ws no sign o f being free in the fo reseeable future. Histo ry has sho wn that applic atio n demand fills the available pipes, and this will be partic -ularly the c ase fo r RF wireless ac c ess links, whic h are limited in available radio spec trum. Thus, the sec o nd appro ac h is to have quality o f servic e (QOS) mec hanisms that reserve reso urc es when a path is established, ensure that ap-plic atio ns do no t exc eed the agreed traffic rate, and blo c k the admissio n o f applic atio ns that wo uld exc eed the c apac ity o f netwo rk links and no des. These mec hanisms vary fro m fine-grained deterministic guarantees, thro ugh tight statistic al bo unds, thro ugh c o urse-grained weak guarantees, to best-ef-fo rt, c arrying traffic as available in exc ess o f reserved reso urc es. We will dis-c uss the tradeo ff between o verengineering and QOS dis-c o mplexity in Chapter 3.
We must be c o nc erned with things that direc tly affec t latenc y. Rec all o ur delay equatio n fro m the previo us sec tio n: D⫽(1 ⫹h⫹c)b/r⫹dp. The delay
o f transmitting the o bjec t o f size bis multiplied fo r eac h ho p at whic h it must be sto re-and-fo rwarded, o r c o pied between buffers in memo ry. These delays are a c ritic al c o mpo nent in the end-to -end latenc y, and we will strive to elimi-nate them.
The best we c an po ssibly do is the single transmissio n delay at the sending system, resulting in a zero -c o py system [Sterbenz 1990a], but applic atio n se-mantic s may c o nstrain this to an additio nal c o py, as desc ribed in Chapter 6. The o ther serio us impac t o n latenc y is when there is blo c king in what sho uld be a lo w-latenc y, high-bandwidth path. This o c c urs when data must be buffered in queues bec ause paths interfere with o ne ano ther to ward the same
Store-and-Forward Avoidance Store-and-forward and copy operations on data have such a significant latency penalty that they should be avoided whenever possible.
II.3 Path Protection Corollary In a resource-constrained environment, mechanisms must exist to reserve the resources needed to provide the high-performance path and prevent other applications from interfering by congesting the network.
destinatio n, o r there is bac k-pressure limiting the ability to transmit data o n a link. These delays are a c ritic al c o mpo nent that we must try to eliminate.
Ano ther fac to r that limits the ability to pro vide a high-perfo rmanc e path is c o ntentio n fo r a shared c hannel. This c an o c c ur in shared medium netwo rks, suc h as traditio nal Ethernet, to ken ring, and wireless. It c an also o c c ur when a bus is used as interc o nnec t bac kplane fo r a netwo rk no de o r end system. While shared medium c hannels have useful pro perties, suc h as the inherent ability to bro adc ast, they limit sc alability. Thus, we need to ensure that c han-nel c o ntentio n do es no t limit sc alability.
There are a number o f c o ntro l mec hanisms that are needed in netwo rking, fo r suc h things as the establishment and fo rwarding o f data, management o f traffic , ro uting o f paths, c o ntro l o f c o ngestio n, and the exec utio n o f parts o f the pro to c o ls. It is essential that mec hanisms o n whic h the c ritic al path trans-fer o f data is dependent be effic ient, and that the interac tio ns and transtrans-fer o f c o ntro l between do mains o f c o ntro l be effic ient.
Finally, rec alling the Applic atio n Primac y (I), there are levels o f servic e that are required fo r the applic atio ns to be able to use the netwo rk.
So me o f these translate direc tly into the Netwo rk Bandwidth and Latenc y (I.3) metric s that have already been intro duc ed. Fo r example, an unreliable
Path Information Assurance Tradeoff Paths have application-driven reliability and security requirements which may have to be traded against performance.
II.7 Efficient Transfer of Control Control mechanisms on which the critical path de-pends should be efficient. High overhead transfer of control between protocol-processing modules should be avoided.
II.6 Contention Avoidance Channel contention due to a shared medium should be avoided.
II.5 Blocking Avoidance Blocking along paths should be avoided, whether due to the overlap of interfering paths, or due to the building of queues.
link may have an erro r rate high eno ugh that retransmissio ns o r an added fo r-ward erro r c o rrec tio n (FEC) c o de reduc es the effec tive bandwidth and in-c reases latenin-c y. Blo in-c k in-c o ding fo r vo iin-c e in the Internet intro duin-c es signifiin-c ant delay. Other aspec ts o f info rmatio n assuranc e c anno t be direc tly translated in this manner. Fo r example, if the reliability o f high-speed netwo rk no des is so po o r that paths c anno t even be established muc h o f the time, we c anno t de-liver high-speed netwo rking to the applic atio ns. Similarly, there may be re-quirements fo r sec urity and privac y that signific antly reduc e perfo rmanc e.
2.2.3
Limiting Constraints
We have no w disc ussed the High-Perfo rmanc e Paths Go al (II) driven by the Applic atio n Primac y (I). Unfo rtunately, we must no w c o nsider the c o nstraints that make this go al diffic ult to ac hieve, whic h will give us plenty to c o nsider in the subsequent c hapters.
The mo st fundamental c o nstraint is the latenc y c o ntributio n due to the speed-o f-light pro pagatio n alo ng a path. We c an reduc e the interapplic atio n delay by inc reasing the bandwidth and dec reasing the latenc y o f individual c o mpo nents suc h as the end system and intermediate no des. Unfo rtunately, we c anno t dec rease the speed-o f-light latenc y between two po ints, barring fundamental new disc o veries in physic s.
While we c an’t beat the speed o f light, we will do o ur best to live with it by engineering direc t lo ng-haul netwo rk paths, and avo iding unnec essary ro und trips. These issues are disc ussed in Chapters 4 and 6. Furthermo re, by c ac hing, prefetc hing, and predic ting, we c an mask the effec ts o f the speed o f light, as will be disc ussed in Chapter 8.
Channel Capacity The capacity of communication channels is limited by physics. Clever multiplexing and spatial reuse can reduce the impact, but not eliminate the constraint.
III.2 Speed of Light The latency suffered by propagating signals due to the speed of light is a fundamental law of physics, and is not susceptible to direct optimization.
III.1 Limiting Constraints Real-world constraints make it difficult to provide high-performance paths to applications.
Co mmunic atio n c hannels, suc h as wires, fiber o ptic c ables, and free spac e have physic al limits o n c apac ity. Clever c o ding and multiplexing tec hniques c an inc rease the effic ienc y o f c hannel utilizatio n but no t exc eed physic al lim-its. Spatial reuse allo ws inc reased aggregate c apac ity in the presenc e o f spa-tial lo c ality. In so me c ases, spaspa-tial reuse is easy to implement, fo r example, by inc reasing the number o f fiber o ptic c ables in parallel. In so me c ases, c hannel c apac ity is a diffic ult c o nstraint, partic ularly fo r wireless c o mmunic atio n where free spac e is a shared medium c hannel with limited radio spec trum.
The rate at whic h lo gic , memo ry, and transmissio n c o mpo nents c an switc h limits the bandwidth o f c o mmunic atio n systems. There are fundamental physi-c al limits o n the switphysi-c hing speed o f elephysi-c tro niphysi-c , o ptiphysi-c al, physi-c hemiphysi-c al, and bio lo gi-c al systems. The pragi-c tigi-c al limit is generally related to the partigi-c ular
tec hno lo gy (fo r example, CMOS and GaAs semic o nduc to rs) arc hitec ture (fo r example, SRAM versus DRAM), and fabric atio n pro c ess. Generally, a c o st tradeo ff (disc ussed next) determines ho w the limits sho uld be applied to a partic ular system. In so me c ases, switc hing speed is o ptimized against c han-nel c apac ity, fo r example, in determining the o ptimal number o f wavelengths and per wavelength data rate in wavelength divisio n multiplexing (WDM) transmissio n.
There are o ther c o nstraints that affec t the design and deplo yment, and while o f less direc t c o nc ern to this bo o k, are impo rtant to understand and keep in the bac kgro und thro ugho ut. When partic ularly relevant, we will ex-plic itly refer to the fo llo wing c o nstraints:
In the end, the dec isio n to deplo y a partic ular netwo rk tec hno lo gy is driven by a c o st versus benefit tradeo ff. When c o nsumers, c o rpo rate entities, o r go v-ernments are willing to pay fo r a partic ular servic e, it will be deplo yed at the c o rrespo nding level. This affec ts whena partic ular deplo yment takes plac e as the c o st o f tec hno lo gy dec reases, and we will generally no t be c o nc erned with this issue.
We are, ho wever, c o nc erned with pic king so lutio ns whic h have relatively lo w ec o no mic c o st, and whic h sc ale well, rather than mo re expensive alterna-tives. Fo r example, while it might be po ssible to design a high-speed wo
rksta-Cost and Feasibility The relative cost and scaling complexity of competing architec-tures and designs must be considered in choosing alternatives to deploy.
III.4 Switching Speed There are limits on switching frequency of components, con-strained by process technology at a given time, and ultimately limited by physics.
tio n ho st–netwo rk interfac e as an array o f expensive mic ro pro c esso rs, there are o ther so lutio ns whic h are c o mpetitive in perfo rmanc e, but an o rder o f magnitude lo wer in c o st.
We are also c o nc erned with designs that sc ale well, and thus we will explic itly disc uss the c o mplexity o f netwo rk no des and o verall netwo rk arc hitec -ture. Fo r example, we c an design a c ro ssbar no nblo c king switc h that c o ntains n2switc h elements fo r npo rts. As ngets large, we sho uld c o nsider multistage interc o nnec tio n netwo rks that have a c o mplexity o f o nly n䡠lo g2(n) switc h el-ements. (We will disc uss this in Chapter 4).
Hetero geneity is a fac t o f life, allo wing diversity and tec hno lo gic al substitu-tio n. The applic asubstitu-tio ns and end systems whic h netwo rks tie to gether are het-ero geneo us due to user preferenc e and spec ializatio n in func tio nality. The no de and link infrastruc ture o f whic h netwo rks are built is also hetero geneo us due to netwo rk pro vider preferenc e and emerging tec hno lo gy. While a c ertain degree o f hetero geneity is nec essary and desirable, it c hallenges perfo rmanc e. The netwo rk arc hitec ture and pro to c o ls sho uld mediate in an effic ient manner to deliver high-perfo rmanc e to applic atio ns.
In a perfec t wo rld, netwo rks are deplo yed in a way that glo bally o ptimizes infrastruc ture. This c eased to be the c ase fo r the U.S. telepho ne netwo rk in the 1970s with the AT&T divestiture, and the U.S. Internet in the 1990s with the dec o mmissio ning o f the NSFNET. While c o mpetitio n has c lear benefits in driving c o sts do wn, the result is neither universal ac c ess no r glo bal o ptimality, exc ept to the degree required by regulatio n. In Chapter 4 we will c o nsider the impac t o f netwo rk to po lo gies that are no t o ptimal. Furthermo re, po lic ies fre-quently dic tate where traffic between c o mmunic ating applic atio ns may o r may no t go , partic ularly based o n sec urity c o nc erns, suc h as avo iding a c o mpeti-to r’s netwo rk o r ho stile natio ns. Ro uting, signaling, and QOS mec hanisms need to c o nsider po lic y c o nstraints, and deliver high-perfo rmanc e servic e to applic atio ns in spite o f these c o nstraints.
Policy and Administration Policies and administrative concerns frustrate the de-ployment of optimal high-speed network topologies, constrain the paths through which applications can communicate, and may dictate how application functionality is distributed.
III.6 Heterogeneity The network is a heterogeneous world, which contains the applica-tions and end systems that networks tie together, and the node and link infrastructure from which networks are built.
Backward compatibility has always been a concern to computer systems ven-dors; it is difficult to make changes that require a wholesale replacement of in-stalled software. In the consumer PC market, users are lured to get new versions of software with new features, but this happens over a period of time. Network protocol software is less visible to the end user, more pervasive in end systems and network nodes, and must interoperate to allow nodes to communicate.
Thus, change in network architecture is pushed to be incremental and for-ward compatible. The location of the protocol directly affects how easy it is to replace. Replacing TCP with an incompatible new transport protocol would be nearly impossible to do, since virtually all systems on the Internet would be af-fected. It is somewhat easier to replace components and protocols within the network, particularly when the effect can be localized to part of the net, such as a single service provider or small set of servers. This is how routing protocols such as border gateway protocol (BGP) are currently upgraded in the Internet.
The failure o f ATM to replac e IP, and the delay in deplo ying IPv6 as a re-plac ement fo r IPv4, are examples o f this c o nstraint. The ability to layer hac k upo n hac k, and fo r them to bec o me institutio nalized, is so bering. Just when we tho ught that IPv6 wo uld be required to so lve the limited address spac e in IPv4, netwo rk address translatio n (NAT) [NATref] hac ks have further po st-po ned the nec essity o f IPv6.
Pro gress c an be made in spite o f bac kward c o mpatibility issues; the c ano ni-c al example is Ethernet, whini-c h has ni-c hanged ni-c o nsiderably fro m its o riginal 10-Mb/s inc arnatio n, thro ugh 100BaseT, Gigabit, and 10Gigabit. The o riginal c arrier sense, multiple ac c ess, c o llisio n detec t, and expo nential bac k-o ff are no w irrelevant; QOS has been added; and 8B/10B c o ding has replac ed Man-c hester. Example 5.2 disMan-c usses this in details. Even tho ugh ATM o ver SONET were sc alable standards fro m the beginning, the ubiquito us deplo yment o f Ethernet helped c o ntribute to its do minanc e to the deskto p.
Standards Both Facilitate and Impede Dilemma Standards are critical to facilitate interoperability, but standards which are specified too early or are overly specific can impede progress. Standards that are specified too late or are not specific enough are useless.
III.8 Backward Compatibility Inhibits Radical Change The difficulty of completely re-placing widely deployed network protocols means that improvements must be back-ward compatible and incremental. Hacks are used and institutionalized to extend the life of network protocols.9
III.7
Standards are an impo rtant way to c o dify design princ iples and pro duc e in-tero perable systems with predic table behavio r. Sinc e netwo rking is all abo ut the intero peratio n o f systems and pro to c o ls, netwo rking standards are partic -ularly impo rtant. After researc h indic ates that a new appro ac h sho uld be taken, there is signific ant inc entive to pro duc e early standards so that multiple vendo rs c an begin pro duc ing intero perable netwo rk c o mpo nents and so ware. It is also impo rtant to pro duc e o pen standards so that individual so ft-ware and c o mpo nent vendo rs do no t push their o wn agendas and c reate multiple inc o mpatible de fac to standards.
Unfo rtunately, standards are frequently an impedimentto pro gress. Stan-dards are o ften defined prematurely, befo re researc h and pro to typing have disc o vered all o f the implic atio ns o f an arc hitec ture that sho uld affec t the standards. The standards pro c ess o f the Internatio nal Telec o mmunic atio ns Unio n (ITU), fo r example, values agreement amo ng its primarily go vernment-o wned pvernment-o stal, telephvernment-o ne, and telegraph (PTT) agenc ies abvernment-o ve all else, inc lud-ing tec hnic al superio rity. One o f the mo st appalllud-ing c o nsequenc es o f this is the ATM c ell paylo ad size o f 48 bytes, whic h was c ho sen as the average o f the final c o mpeting pro po sals o f 32 and 64 bytes.
Standards that o verspec ify also impede pro gress, partic ularly when they spec ify the internal implementatio n o f systems, rather than their externally visible behavio r. Finally, standards that are c o dified to o late are useless and ir-relevant. Fo rmal standards that are released after de fac to standards take o ver are irrelevant.
2.3
Design Principles and Tradeoffs
So far, we have presented the axio ms that we must Kno w the Past and Pre-pare fo r the Future, that we have an Applic atio n Mo tivatio n fo r the High-Perfo rmanc e Paths Go al in the netwo rk, and we have disc ussed Limiting Co n-straints in ac hieving this. We are no w ready to present the final axio m and two high-level c o ro llaries, whic h will serve to c o ver the design princ iples that we will use thro ugho ut the bo o k:
The fundamental princ iple that will guide o ur analysis and design o f net-wo rks is that they are c o mplex systems that must be c o nsidered as a who le. It makes no sense to o ptimize part o f the netwo rk o r a netwo rk c o mpo nent witho ut c o nsidering the o verall impac t o n netwo rk perfo rmanc e, and witho ut
Systemic Optimization Principle Networks are system s of system swith complex compositions and interactions at multiple levels of hardware and software. These pieces must be analyzed and optimized in concert with one another.
ensuring that individual o ptimizatio ns wo rk to gether to enhanc e perfo rmanc e in c o nc ert.
An impo rtant fac to r in c o nsidering o ptimizatio ns is the granularity o ver whic h the o ptimizatio n applies. The range o f time intervals is vast, c o vering many o rders o f magnitude (at least 1018), fro m years fo r pro to c o l deplo yment, thro ugh days o r mo nths fo r netwo rk c o nfiguratio n, to small frac tio ns o f a sec -o nd f-o r bit transfer time:
■■ Pro to c o l deplo yment ■■ Netwo rk c o nfiguratio n
■■ Netwo rk management interval ■■ Sessio n establishment and lifetime
■■ Co nnec tio n/flo w establishment and lifetime ■■ Pac ket/frame transfer
■■ Cell transfer ■■ Byte transfer ■■ Bit transfer
Fo rtunately, the mo dular, layered, and hierarc hic al princ iples we will intro -duc e in the rest o f this sec tio n help iso late these c o nc erns fro m o ne ano ther. We will highlight the granularities o f c o nc ern in the subsequent c hapters, as appro priate fo r partic ular aspec ts o f high-speed netwo rking.
Optimizatio ns frequently have unintended side effec ts that impac t perfo r-manc e in o ther ways. Optimizing a partic ular c o mpo nent o r aspec t o f a system virtually always c o mes at so me c o st elsewhere. If the balanc ing c o sts are no t c o nsidered, the overallperfo rmanc e may ac tually dec rease. While it is impo r-tant to analyze systemic effec ts o f o ptimizatio ns in advanc e, this is very hard to do in c o mplex systems o f systems, and no t all o f the c o nsequenc es o f o pti-mizatio ns are likely to be disc o vered until deplo yment. Testbeds and phased deplo yments c an help c atc h side effec ts befo re wide-sc ale deplo yment; this further c o mplic ates the Standards Bo th Fac ilitate and Impede Dilemma (III.8).
Keep it Simple and Open It is difficult to understand and optimize complex systems, and virtually impossible to understand closed systems, which do not have open pub-lished interfaces.
IV2
Consider Side Effects Optimizations frequently have unintended side effects to the detriment of overall performance. It is important to consider, analyze, and understand the consequences of optimization.
Simplic ity in arc hitec ture and design o f netwo rks, pro to c o ls, algo rithms, and c o mpo nents c an pay o ff in two ways. First, c o mplex systems are diffi-c ult to understand, o ptimize, maintain, and enhandiffi-c e. Lesso ns in diffi-c o mputer arc hitec ture and so ftware engineering have taught us that simplic ity and reg-ularity make it easier to pro duc e high-perfo rmanc e systems. This is partic u-larly true as systems evo lve, where eac h inc remental enhanc ement pro vides diminishing return, to the po int where o nly a c o mplete redesign will
straighten things o ut. If this is an implementatio n o f a pro to c o l, it is an in-c o nvenienin-c e to the develo per. If it is the pro to in-c o l design itself, it may no t be prac tic al to do the rewo rk at all, due to Bac kward Co mpatibility Inhibits Radic al Change ( III.7) .
Furthermo re, c lo sed mo no lithic systems are the antithesis o f ho w we have learned to struc ture and design systems, and are partic ularly egre-gio us in the netwo rk enviro nment. Operating systems need to be mo dular with do c umented o pen interfac es so that netwo rk pro to c o l func tio nality c an be understo o d, o ptimized, and enhanc ed. Netwo rk no des, suc h as ro uters and switc hes, need o pen published interfac es fo r signaling and c o n-tro l fo r similar reaso ns. Unfo rtunately, this is no t the direc tio n that o perat-ing system and ro uter vendo rs are takperat-ing. Their go al o ften appears to be to maximize pro fit and market do minanc e. The o pen so ftware mo vement in general, and the o pen signaling and arc hitec ture researc h initiatives in par-tic ular, attempt to mo ve the field bac k to ward ro bust and flexible netwo rks that c an evo lve.
The sec o nd way in whic h simplic ity must be c o nsidered is in the tradeo ff between o verengineered simplic ity and o ptimal c o mplexity, whic h will be ad-dressed in detail later (Design Princ iple 2B).
We have to be c areful no t to impo se netwo rking o verhead by impro perly distributing an applic atio n. We have already po inted o ut that netwo rking is driven by applic atio n and user needs, and in many c ases the lo c atio n and par-titio ning o f func tio nality is dic tated by the physic al lo c atio n o f users. Ho w-ever, given this c o nstraint, there is a signific ant quantity o f underlying applic atio n func tio nality and interac tio n with the netwo rk that may be arbi-trarily partitio ned to reside o n user end systems, at servers, o r within the net-wo rk. We must make sure that this partitio ning is no t do ne in a way that c hallenges the ability to pro vide a high bandwidth path o r makes an applic a-tio n appear to be high-speed when it do esn’t need to be.
System Partitioning Corollary Carefully determine how functionality is distributed across a network. Improper partitioning of a function can dramatically reduce overall performance.
Technology improves, tradeoffs change, and new applications emerge that drive changes and new functionality in network architecture and protocols. What is optimal at a particular point in time is likely not to be in the future, but Backward Compatibility Inhibits Radical Change (III.7) prevents us from mak-ing progress if radical replacement is required. Thus, protocols should have data structures and fields to allow changes, negotiation of new options, and addi-tions in a backward-compatible manner. Similarly, the specification of control mechanisms should be such that these can evolve over time in a graceful way. The rest o f this sec tio n will present the next level o f refinements o f the Sys-temic Optimizatio n Princ iple (IV), that is, the detailed design princ iples. To simplify the numbering sc heme, these princ iples o mit the leading ro man nu-meral IV, but any arabic numbered princ iple c an be assumed to be a subprinc i-ple o f IV, fo r exami-ple, Design Princ ii-ple 2 rather than IV-2.
2.3.1
Critical Path
In a systemic view o f o ptimizatio n, the mo st impo rtant thing is to understand which o ptimizatio ns have the mo st significant impact o n the system as a who le.
Network components (end systems, network nodes, and links) are not free and unlimited, nor are the pieces of which they are constructed, such as processors, memory, disk, and interface electronics. Trying to optimize everything to a partic-ular degree comes at the cost of further optimizing the pieces that really matter, both in terms of time and effort in architecture, design, and implementation, as well as in the cost of deploying systems. Thus, we need to carefully consider what matters, how much it matters, and apply optimization effort accordingly.
Second-Order Effect Corollary The impact of spatially local or piecewise optimiza-tions on the overall performance must be understood; components with only a second-order effect on performance should not be the target of optimization.
1A Selective Optimization Principle It is neither practical nor feasible to optimize everything. Spend implementation time and system cost on the most important con-tributors to performance.
1 Flexibility and Workaround Corollary Provide protocol fields, control mechanisms, and software hooks to allow graceful enhancements and workarounds when funda-mental tradeoffs change.
In partic ular, any c o mpo nent that has o nly a sec o nd-o rder effec t (⬍10 per-c ent) o n perfo rmanper-c e is no t a per-c andidate fo r o ptimizatio n. This per-c an be viewed either in terms o f o ther c o mpo nents o r the to tal perfo rmanc e. Fo r example, c o nsider a netwo rk path c o nsisting o f fo ur segments, with relative latenc ies o f 50 ⫹2 ⫹25 ⫹10 ⫽77. The o ne link that c learly doesn’tneed attentio n is the sec o nd, whic h c o ntributes o nly 2/77 ⫽0.03 o f the to tal latenc y. Co mpletely eliminating this delay do esn’t signific antly affec t the o verall latenc y. Similarly, upgrading a single link fro m 1 Mb/s to 10 Mb/s is a waste o f reso urc es if there are already 100Mb/s links (with similar latenc y, erro r, jitter, and c o st c harac -teristic s) that fo rm parallel paths. Finally, this c o ro llary indic ates that we need to be far mo re c o nc erned with o ptimizing the latenc y thro ugh wide area netwo rks than LANs; the speed o f light latenc y o f the fo rmer is a signific ant frac -tio n o f a 100-ms interac tive delay budget.
On the o ther hand, the latenc y thro ugh LANs c anno t be igno red. Distributed pro c essing and feedbac k c o ntro l applic atio ns c an have extremely tight latenc y bo unds. This c o nstraint has been felt even with relatively primitive third-generatio n netwo rks o f wo rkstatio ns (NOWs).
A related c o ro llary is to take a data flo w view, and examine the o peratio ns that must take plac e to transfer the data between the c o mmunic ating applic a-tio ns. Operaa-tio ns that happen frequently have pro po ra-tio nally mo re impac t o n the o verall perfo rmanc e, and are c andidates fo r o ptimizatio n. This is related to the c o nc ept o f a so ftware fast path, but is bro ader in implic atio ns; a fast path might be a tec hnique to implement part o f a c ritic al path.
It is useful to c o nsider c o ntro l and data flo w separately. Clearly, data trans-fer functionsthat are part o f the no rmal data flo w are part o f the c ritic al path, that is, we c an view the end-to -end path between applic atio ns as a sequenc e o f o peratio ns o n the c ritic al path. Optimizatio n o f o peratio ns no t o n the c ritic al path, suc h as the pro c essing o f exc eptio ns and erro rs is misdirec ted. There are three impo rtant o bservatio ns to make:
1. Transfer control functionssuc h as flo w c o ntro l and framing must be c o nsidered part o f the c ritic al path, even tho ugh they may o c c ur at a larger granularity (fo r example, applic atio n frame versus pac ket versus byte).
2. The c ritic al path may no t be stric tly serial; it may be fo rmed by parallel branc hes, whic h rejo in befo re the data has reac hed the destinatio n applic atio n.
Critical Path Principle Optimize implementations for the critical path, in both con-trol and data flow.
3. Dependenc ies are impo rtant; even if an o peratio n happens relatively in-frequently, if subsequent data flo w wo uld be held up, this infrequent o p-eratio n must be c o nsidered part o f the c ritic al path. Fo r example, if o nly 1 perc ent o f the pac kets need a partic ular data transfo rmatio n applied, we might generally no t c o nsider this func tio n to be part o f the c ritic al path and wo rthy o f o ptimizatio n. On the o ther hand, if pac ket sequenc e must be maintained, then subsequent pac kets will be delayed whenever this o c c asio nal o peratio n o c c urs. We must either c o nsider it part o f the c ritic al path o r find a way to redesign the c ritic al path to remo ve the slo w o peratio ns.
It is impo rtant to no te the relatio nship o f the Critic al Path Princ iple and the Sec o nd-Order Effec t Co ro llary (1A). The Critic al Path Princ iple tells us o n whic h o peratio ns the c ritic al end-to -end data flo w depends, based o n the fre-quenc y o f these o peratio ns and dependenc ies. The Sec o nd-Order Effec t Co ro l-lary tells us if—and ho w muc h—these func tio ns need to be o ptimized based o n their relative c o ntributio n to perfo rmanc e.
The result o f applying Design Princ iples 1A and 1B is a determinatio n o f what needs o ptimizatio n and by ho w muc h. This c an guide us o n ho w to im-plement within a partic ular tec hno lo gy, suc h as what func tio nality to put in a so ftware fast path o r hardware pipeline.
Typic ally, we have a variety o f implementatio n tec hno lo gies to c ho o se fro m, and the faster tec hno lo gy is generally mo re expensive. Thus we have a func tio nal partitio ning and assignment pro blem, fo r example, so ftware versus hardware, memo ry, versus c ac he versus registers, semic usto m versus fully c usto m VLSI, o r c o mplementary metal o xide semic o nduc to r (CMOS) versus gallium arsenide (GaAS) semic o nduc to r tec hno lo gy. Thus we use the Sec o nd-Order Effec t Co ro llary and Critic al Path Princ iple (1B) to determine what needs o ptimizatio n, and the Func tio nal Partitio ning and Assignment Princ iple to determine the implementatio n tec hno lo gy based o n the relative c o sts and benefits.
This c an be c ruc ial fro m a pac kaging standpo int, as there may be a finite amo unt o f c hip o r c irc uit bo ard area, o r fast memo ry available, amo ng whic h func tio nality must be partitio ned. This is similar to the pro blem o f trading o ff CPU versus c ac he memo ry area in high-perfo rmanc e VLSI pro c esso r design.
Functional Partitioning and Assignment Principle Carefully determine what func-tionality is implemented in scarce or expensive technology.
2.3.2
Resource Tradeoffs
Our next design princ iple rec o gnizes that we must balanc e the utilizatio n o f reso urc es to ac hieve o ptimal c o st and perfo rmanc e.
The mo st o bvio us reso urc es to c o nsider in netwo rk deplo yment are the triple o f bandwidth, pro c essing, and memo ry 具B, P, M典. A simple example, de-sc ribed in mo re detail in Chapter 3, is the ability to trade bandwidth fo r mem-o ry in the lmem-o c atimem-o n mem-o f c ac hed mmem-o vie c mem-o ntent. If bandwidth is free and
unlimited, but memo ry is very expensive, we minimize the memo ry require-ments and have a single c o py o f eac h mo vie at c entral servers and blast to users eac h time it is viewed. If bandwidth is expensive and memo ry free and unlimited, then we minimize the bandwidth by sending every mo vie that a user will ever watc h fo r indefinite sto rage.
It is impo rtant to no te that the func tio ns that go vern these tradeo ffs are no t nec essarily linear, no r are they always c o ntinuo us. Fo r example, inc reases in pro c essing po wer and memo ry c apac ity are expo nential in nature. The in-c rease in pro in-c esso r frequenin-c y fo llo ws a relatively smo o th in-c urve with small and frequent inc reases in c apac ity. On the o ther hand, memo ry c apac ity is a step func tio n who se po ints matc h an expo nential c urve o nly at the interval at whic h c apac ity is multiplied by fo ur.
Other perfo rmanc e metric s limit o ur freedo m to balanc e these reso urc es, and are better viewed as c o nstraints. The mo st impo rtant o f these is latenc y, but o thers inc lude jitter, erro r rate, po wer c o nsumptio n, and heat dissipatio n. Latenc y is suc h an impo rtant c o nstraint due to the speed-o f-light pro pagatio n delay. It is impo rtant to no te, ho wever, that we c an trade c o nstraint values against reso urc e utilizatio n. Fo r example, by inc reasing the bandwidth we c an lo wer the lo ss rate by fo rward erro r c o rrec tio n, and lo wer the effec tive interapplic atio n latenc y by presending all data likely to be needed within a ro und-trip time. We will thus be mo st interested in reso urc e/c o nstraint tuple
具B, P, M D典
Resource Tradeoffs Change The relative cost of resources and the impact of con-straints change over time, due to nonuniform advances in different aspects of technology.
2A Resource Tradeoff Principle Networks are collections of resources. The relative composition of these resources must be balanced to optimize cost and performance.
It is extremely impo rtant to be aware that the relative c o st o f reso urc es c hanges o ver time, and this dramatic ally affec ts o ptimal netwo rk arc hitec ture and pro to c o l design. Fo r example, first-generatio n netwo rks were designed as-suming that bandwidth was relatively expensive. With the advent o f o ptic al fibers and fast pac ket switc hing, bandwidth bec ame relatively c heap during the third generatio n. In the fo urth generatio n, pro c essing and memo ry have bec o me signific antly c heaper, reversing the tradeo ff. While it may be hard to mo dify pro to c o ls quic kly to the tradeo ffs, we need to be aware o f the po tential fo r c hange. Netwo rk to po lo gy and engineering sho uld be as flexible as po ssi-ble to allo w the o ptimal plac ement o f reso urc es as tradeo ffs c hange.
Multic ast is a servic e that depends o n reso urc e tradeo ffs, whic h will be ex-amined in Example 3.4. It is impo rtant to no te here that multic ast c o nserves bandwidth o n links and thro ugh no des, and reduc es pro c essing requirements in end systems by allo wing them to transmit o nly a single c o py o f data. This c o mes at the c o st o f the c o mplexity o f implementing suppo rt fo r multic ast in netwo rk no des and pro to c o ls.
Furthermo re, a number o f netwo rk c o ntro l mec hanisms and pro to c o ls relay o n multic ast fo r effic ient o peratio n, suc h as netwo rk to link layer address translatio n. Thus, multic ast sho uld be suppo rted by lo c al area link pro to c o ls and netwo rk pro to c o ls and no des.
If reso urc es are very expensive, then c o mplexity in arc hitec ture and imple-mentatio n may be required, but as no ted by Keep It Simple and Open (IV2), this c o mes at o ther c o sts. As an example, in an enviro nment where bandwidth is expensive and thus a sc arc e reso urc e, c o mplex, fine-grained quality o f ser-vic e mec hanisms to ensure o ptimal bandwidth utilizatio n o f the netwo rk are in o rder. This was the appro ac h taken with ATM traffic management. This also requires the memo ry and pro c essing required to sto re detailed link state metric s and pro c ess the admissio n c o ntro l and QOS-based c o nnec tio n ro uting algo rithms. On the o ther hand, when bandwidth is relatively c heap, it makes mo re sense to have simpler, c o arser-grained QOS mec hanisms with so me o verpro visio ning o f bandwidth to ac c o unt fo r the inac c urac y o f the algo
-Support for M ulticast Multicast is an important mechanism for conserving band-width and supporting network control protocols, and should be supported by the net-work.
2C Optimal Resources Utilization versus OverEngineering Tradeoff Balance the ben-efit of optimal resource utilization against the costs of the algorithms that attempt to achieve optimal solutions.
rithms. Finally, if bandwidth is free and unlimited, QOS mec hanisms to o pti-mize bandwidth and prevent c o ngestio n are no t needed at all.
No te that c o st inc ludes the pro jec ted c o st o f upgrade, and benefit inc ludes the fac to r o f useful life. Parts that are very diffic ult to upgrade must be o v-erengineered fo r the beginning o f their life to delay the time at whic h they be-c o me the bo ttlenebe-c k. Examples o f this are transo be-c eanibe-c be-c ables and last-mile residential wiring. Bec ause the fixed c o st is so high, it makes sense to greatly o verengineer the bandwidth based o n c urrent requirements to allo w fo r future gro wth in traffic .
2.3.3
End-to-End versus
Hop-by-Hop
There are two ways in whic h partic ular pro to c o l func tio nality c an be imple-mented: end-to -end (E2E) o r ho p-by-ho p (HBH), as sho wn in Figure 2.3.
This is a partic ularly c ritic al issue in determining what func tio nality go es in the end-to -end transpo rt pro to c o l, versus the ho p-by-ho p data transfer link pro to c o l, and will be extensively disc ussed in Chapter 7. We will thus intro -duc e the End-to -End Arguments here, rephrased so mewhat fro m its o riginal fo rm [Saltzer 1981, 1984].
This argument states that end-to -end servic es c anno t be pro vided so lely by the c o mpo sitio n o f ho p-by-ho p servic es. We might first think that f⫽f1⬚f2⬚f3 as sho wn in Figure 2.3.10This is no t really the c ase, sinc e we c anno t c o ntro l
End-to-End Arguments Functions required by communicating applications can be correctly and completely implemented only with the knowledge and help of the appli-cations themselves. Providing these functions as features within the network itself is not possible.
3
Hop-by-Hop End-to-End
f -1
f
f1 f1-1 f2 f2-1 f3 f3-1
[image:27.612.101.461.479.574.2]g g′ g′′ g′′′
Figure 2.3 End-to-end versus hop-by-hop.
10Co mpo stitio n o f func tio ns is indic ated by the
what happens between the bo undaries o f the ho ps, In reality we have f⫽ g⬚f1⬚g⬘⬚f2⬚g⬙⬚f3⬚g. A simple example o f this is the use o f enc ryptio n fo r data c o nfidentiality. This func tio n c anno t be perfo rmed “in the netwo rk,” sinc e if the data is in the c lear at any po int between their end-systems so urc e and destinatio n, there is no c o nfidentiality.
Sinc e the end-to -end func tio nality c anno t be pro vided as a c o mpo sitio n o f ho p-by-ho p func tio nality, it is generally a duplic atio n o f func tio nality, whic h c o mes at inc reased c o st and perhaps reduc ed perfo rmanc e. This is no t always the c ase, ho wever, and func tio nality that increasesthe end-to -end perfo r-manc e o f mec hanisms shouldbe do ne ho p-by-ho p. An example is that while ho p-by-ho p erro r c o ntro l c an’t replac e end-to -end erro r c o ntro l, it may lo wer the pro bability o f end-to -end erro rs signific antly with a reduc tio n in the fre-quenc y and latenc y o f end-to -end retransmissio ns. This situatio n is the o ne that is fac ed, fo r example, by netwo rks with erro r-pro ne wireless links.
Hop-by-Hop Performance Enhancement Corollary It may be beneficial to dupli-cate an end-to-end function hop-by-hop, if doing so results in an overall (end-to-end) improvement in performance.
3A
end system subnetwork
link E2E HBH
HBH HBH
subnet E2E end
system
HBH HBH
[image:28.612.82.441.356.584.2]transport E2E
Generally, end-to -end is used in the c o ntext o f asso c iatio ns between appli-c atio ns o n end systems, and it is the transpo rt layer pro to appli-c o l that implements end-to -end func tio nality. These issues will be c o nsidered in detail in Chapter 7. The relative no tio n o f end-to -end (E2E) with respec t to ho p-by-ho p (HBH) do es no t o nly apply between end systems, as sho wn in Figure 2.4.
With respec t to the netwo rk no des, flo ws between switc hes o r ro uters c an be viewed as end-to -end, while links between link repeaters (o r bridges) are viewed as ho p-by-ho p. Similarly, within a netwo rk, link pro to c o l func tio nality is ho p-by-ho p with respec t to the end-to -end netwo rk layer. An internetwo rk c o nsisting o f multiple subnets c an view eac h subnetwo rk layer 3 pro to c o l as ho p-by-ho p with respec t to the entire netwo rk. In eac h o f these c ases, the questio n is whether to plac e func tio nality end-to -end o r ho p-by-ho p and the next lo wer layer o f rec ursio n.
2.3.4
Protocol Layering
Layering has lo ng been a mo del fo r spec ifying and implementing c o mmunic a-tio ns netwo rks; very early pro to c o l suites suc h as SNA used layered mo dels. The evo lutio n o f the layered mo del resulted fro m the way c o mmunic atio ns netwo rks are o rganized:
1. Physical layerfo r wavefo rms, bit c o ding (fo r example, NRZI), and mo d-ulatio n (fo r example, QAM) o ver wire, fiber, o r radio11
2. Link layerfo r framing and ho p-by-ho p data assuranc e ac ro ss a link (fo r example, HDLC and SONET12)
3. Network layerfo r addressing, ro uting, and signaling amo ng switc hes, gateways, and ro uters (fo r example X.25 IP, and ATM)
4. Transport layerfo r data transfer between end systems (fo r example TCP and TP4), inc luding end-to -end flo w and erro r c o ntro l
This layering has the benefit that pro to c o ls fo r a partic ular layer c an be im-plemented o n the c o mpo nents o f that layer, witho ut generally affec ting the de-sign o f pro to c o ls o r c o mpo nents o f o ther layers. No te that higher-layer
systemsmust terminate lo wer-layer protocols, as sho wn in Figure 2.5. The end
Endpoint Recursion Corollary What is hop-by-hop in one context may be end-to-end in another. The End-to-End Arguments can be applied recursively to any se-quence of nodes in the network, or layers in the protocol stack.
3B
system terminates the link and netwo rk layers in additio n to the transpo rt layer; the switc h terminates the link layer in additio n to the netwo rk layer.
Layering pro vides a useful servic e abstrac tio n whic h iso lates func tio nality, allo ws async hro no us o peratio n, and independent replac ement and upgrading o f c o mpo nents, whether hardware (as in Figure 2.5) o r pro to c o l so ftware. Lay-ering pro vides a way o f struc turing systems to avo id mo no lithic arc hitec tures that are diffic ult to understand, debug, and upgrade.
This separatio n o f c o mmunic atio ns pro to c o l func tio nality is extremely use-ful, and led to the desire to standardize layer func tio nality and interfac es. A layered framewo rk fo r c o mmunic atio ns pro to c o ls was develo ped, the ISO
Protocol Layering Principle Layering is a useful abstraction for thinking about net-working system architecture and for organizing protocols based on network structure.
4
end system
link link
network
link network transport
link network transport end system router
[image:30.612.79.445.59.151.2]repeater / bridge
Figure 2.5 Layering based on com ponent type.
abstraction boundary
abstraction boundary peer-peer virtual interaction services provided
services used
layer n +1
layer n –1 layer n
layer n +1
layer n –1 layer n
[image:30.612.80.446.444.597.2]o pen systems interc o nnec tio n (OSI) mo del [ISO7498], whic h refined the func -tio nality o f existing layers and defined addi-tio nal layers:
5. Presentation layerpro vides refo rmatting data fo r hetero geno us applic atio ns
6. Session layerc o ntro l struc ture fo r c o mmunic atio n between applic a-tio ns; sessio n management between c o o perating applic aa-tio ns
7. Application layerfo r applic atio ns using c o mmunic atio ns servic es It is c ritic al to understand layering fro m three different perspec tives:
■■ As an abstractionto think abo ut netwo rk design and arc hitec ture ■■ As an architectural division of protocolsbased o n the func tio nal
re-quirements o f netwo rk c o mpo nents, netwo rk to po lo gy, end systems, and applic atio ns
■■ As a mechanism for implementationo f pro to c o l func tio nality
Unfo rtunately, many implementers o f the OSI spec ific atio n c o nfused these three aspec ts, in partic ular pro to c o l specificationand mechanism. The OSI layered struc ture is sho wn in Figure 2.7.
There are two types o f info rmatio n flo ws:
Peer-to-peer protocols, whic h spec ify the data enc apsulatio n (header
