Scalable Network Design
Ryan J. Determan, CCIE 5276
Scalable Network Design
Ryan J. Determan, CCIE 5276
Copyright 2002 DDLS
Internetwork Design Goals
• Functionality
• Scalability
• Adaptability
• Manageability
• Cost effectiveness
• Basic trade-off in network design is cost versus availability
• Recurring costs tend to predominate
Characterizing Scalable Internetworks
Scalable internetworks need to be:
• Reliable and available
• Responsive
• Efficient
• Adaptable
• Accessible but secure
The Cisco Design Model
The Core Layer
The Core is responsible for optimized transport between remote sites
• No routing
• Load sharing / Efficient use of bandwidth
• Standards based technologies
• Don’t be cheap!
The Distribution Layer
The Distribution Layer is responsible for Layer 3 resiliency
• Company resources and new services
• Routing peers
• Filter and summarize!
• Inbound security and policy
• Addressing: private or public
The Access Layer
The Access Layer is responsible for connecting logical workgroups to
backbones
• User segmentation
• Isolate traffic to/from the workgroup
Designing The Core
Layer 1 predetermines network success
• Worth its weight in copper?
• Not a good place for cost efficiency
• Keep it simple and standard
• Layer 1 affects all other layers
Core Design Choices
ATM
• Good if available
• Can be expensive
• Built in QoS mechanism
Point-to-Point Links (T1’s, OC3’s, etc)
• Best if available and affordable
• Always expensive
• Nothing built in
Core Design Choices (cont.)
Frame Relay
• Useable but not favorable
• Always available
• Can perform some limited traffic shaping
Ethernet/LAN
• Best choice if applicable
• 10mb, 100mb, 1000mb
• GigE, FastE, FDDI
Designing the Distribution & Access Layers
A good Layer 2 design can hide Layer 1 problems from Layer 3
• Design for redundancy
• Do you know who your root bridge is?
• Spanning Tree is your friend & foe
How redundant are you?
A fast converging, redundant Layer 2 network will prevent Layer 3 flaps
• Use multiple trunks utilizing different blades
• Don’t mix and match standards (ie 802.1Q & ISL)
• HSRP
Root Bridge?
Proper Layer 2 design denotes a root bridge
• Defines STP metric for algorithm
• Should be the ‘most’ redundant
• Commonly forgot
Spanning Tree
STP should provide fault tolerance, not loop avoidance
• Designing ‘looped’ layer 2 networks is beneficial
• Should have ‘primary’ path and secondary
• PVST gets around down time
• Tweak the STP protocol (timers, cache, diameter)
Hot Standby Routing Protocol
HSRP provides Access Layer fault tolerance for hosts
• Cisco proprietary solution (VRRP is RFC)
• Allows multiple gateways to respond
• Only need to configure 1 gateway
• No IRDP
HSRP (cont)
HSRP in action:
Router B Priority 100 172.16.10.169 0010.0b79.5800
Core
Virtual Router 172.16.10.110
0000.0c07.ac2f Router A
Priority 200 172.16.10.82 0010.f6b3.d000
I need to get to 172.16.3.127.
Use MAC address 0000.0c07.ac2f.
File Server A 172.16.3.127
Designing Layer 3
Let Layer 3 make the decisions
• Design for layer 3 switching (FS, Netflow, CEF)
• Scalable routing protocols (OSPF, BGP)
• Routers route and firewalls firewall
• Public vs. private addressing and NAT
• Plan for ‘special’ routers
Switching at Layer 3
Different switching technologies allow for faster path choice
• Fast Switching (route cache)
• NetFlow (IP pair based flow with ACL awareness)
• CEF (express forwarding using a FIB)
• Be careful of the recursive lookup
Scalable Routing Protocols
Choose the correct protocol
• Static routes are not evil
• OSPF for small to medium IGP’s
• ISIS for large IGP’s
• BGP for internet routing policy
Static Routes
Multiple static routes can provide:
• Load balancing
• Fault tolerance
• Good method for BGP advertisements
Designing OSPF
OSPF is stable and efficient in a properly designed network
• Watch for limits: 50 areas, 3-4 areas per router
• Summarize at area borders
• Implement OSPF features when possible
OSPF Features
Configurable OSPF features:
• Stub, totally stub and not-so-stubby areas
• LSA pacing
• Multiple default-gateways by changing default-cost
• External type 1
• Demand circuits (doesn’t have to be a DDR link)
Designing BGP
BGP is only desirable when you have multiple internet connections
• Use loopbacks and statics
• Do you need an internet routing table?
• Use IGP to get ‘out’ to the internet
• Use BGP to get traffic ‘in’
• Let the ISP do the work
BGP features
Configurable BGP features:
• Know your attributes (Med, LP, community)
• Default originate
• Many filtering options (as-path, prefix, dist.)
• Large scale: route reflectors and confederations
• Route dampening
Security at Layer 3
Routers connect & firewalls protect
• Standard ACL’s are OK
• Use routers for choke points
• Use firewalls for security and NAT
• Firewall IOS is not a firewall
Addressing Design
At some point you must NAT
• Proper NAT design helps security &
implementation
• Let the Firewalls translate addresses
• Should have distinct ‘line’ of addressing
• Use NAT for services and PAT for users
Special Routers
Router features change quickly, but your design should not
• Design with ‘router only’ routers
• Design for special purpose/IOS routers to support changing services
• Don’t hope for the ‘perfect’ IOS
• Run GD code on ‘router only’ routers
Designing Layer 4+
I thought network design ended at L3?
• What about the services and servers
• Content delivery design
• CSS, Cache Engine, Content Engine
Changing Services
Chances are the services your provide as a business model reside on your
network
• Don’t design yourself out of business
• Design for single points of security
• Have a place to ‘sniff’ traffic
Content Delivery Networks
A new type of design has emerged:
Content Delivery
• How do I get my content to my customer quickly, reliably, and accurately?
• How can I support 20 million hits per day?
• Can I offload any server traffic?
Content Delivery Networks (cont)
Making content more available
• Push the content to the edge
• Load balance mirrored content
• Creative DNS solutions
Content Delivery Networks (cont)
Content delivery hardware and features
• Cache Engine (cache’s local servers static content and offloads server of these requests)
• Content Engine (provides web services)
• Content Smart Switch (glue that connects it all together)
Content Smart Switch
The CSS is a multi-service box
• Can switch/route traffic on any layer 1 - 7
• Can provide DNS server functionality
• Can replicate web updates to all mirrored sites
• Can load balance to local or remote servers based upon user definable criteria
