Target Hardware
4.1 Broadcom
Broadcom is the world’s largest producer of merchant silicon networking chips.
Their chips are used by a variety of networking hardware vendors, including those that also produce their own ASICs, such as Cisco. Broadcom ASICs support a variety of SDN standards, either natively [17, 18, 19, 20], or through operating systems created by other vendors [3, 4, 93].
Ingress Port
VLAN
VLAN 1
Termination MAC
MPLS 0
MPLS 1
MPLS 2
Unicast Routing Multicast Routing
Bridging
Policy ACL Maintenance Point
MPLS L2 Port Color-Based Actions
Figure 4.1:The OF-DPA flow table pipeline. Arrows indicate the potential targets of Goto-Tableinstructions.
4.1.1 OF-DPA
The OF-DPA is an interface to Broadcom ASICs that can be used by vendors to produce an OpenFlow implementation that closely models the underlying ASIC pipelines [18].
The OF-DPA can support multiple different pipelines. Most are non contra- dictory and can be used in parallel, with a few minor scenarios where that is not possible. An OF-DPA pipeline using all possible tables is shown in figure 4.1. Each table has a set of entry types, and each entry type has its own set of matches and actions that can be used. This results in tables that do not make
sense when translated directly to P4—for instance, the unicast routing table contains both IPv4 and IPv6 routes.
The entry types used in OF-DPA are very strictly defined. For instance, the Untagged Packet Port VLAN Assignment entry type in the VLAN table has the following constraints:
• entries must Match Ingress Port,
• entries must match a VLAN VID of 0,
• entries must assign a VLAN VID to matching packets,
• entries must direct packets to the Termination MAC table, and
• entries may, optionally, set VRF metadata.
There are 9 different entry types in the VLAN table, and each has similar constraints to the Untagged Packet Port VLAN Assignment type.
The bridging table uses a vendor extension to perform MAC learning. Every time the bridging or routing tables are applied to a packet, the OF-DPA per- forms a look-up of the packet’s Ethernet Source address against the entries in the bridging table, comparing the ingress port of the packet with the associated port in the bridging table. If the two do not match, or the Ethernet Source address is not in the bridging table, then the datapath will either notify the controller, or automatically update the entry in the bridging table, depending on configuration.
Most tables, by default, forward packets to the next stage in the pipeline. This is advantageous when reordering tables or recirculating packets, as it means packets can easily reach the intended next table without needing to match entries in the intervening tables.
L3 ECMP (select) L3 Multicast (all) L2 Rewrite (indirect) L2 Multicast (all) L2 Flood (all)
L3 Interface (indirect) L3 Unicast (indirect)
L2 Interface (indirect)
Figure 4.2:The group tables used by the OF-DPA for bridging and routing, and their group types. Arrows indicate groups that reference other groups.
The OF-DPA makes extensive use of group tables (§2.2.5). Figure 4.2 shows the groups the OF-DPA uses for bridging and routing. Groups are also used by the Multiprotocol Label Switching (MPLS) pipeline, including fast-failover groups for protected circuits.
The only writeable metadata used in the OF-DPA is a 16-bit non-maskable Virtual Routing and Forwarding (VRF) field, limiting how a mapping algo- rithm can use metadata to control the flow of packets through the pipeline.
4.1.2 OpenNSL
Broadcom ASICs also support other standards, including OpenNSL [19]. OpenNSL provides a very detailed, low-level interface to program Broadcom ASICs, and includes some features not included in OF-DPA. For instance, the OpenNSL field processor has the ability to mirror packets, both at ingress and egress.
This thesis uses such features to infer the capabilities of Broadcom hardware.
4.1.3 Other OpenFlow Implementations on Broadcom ASICs
Broadcom does not create operating systems for its ASICs, it sells ASICs to vendors who produce their own operating systems. As such, an OpenFlow implementation using the OF-DPA is not supported by all hardware using Broadcom ASICs, the operating system has to include support for OpenFlow.
Some vendors that support OpenFlow on Broadcom ASICs do so without using the OF-DPA.
Allied-Telesis AlliedWare Plus provides support OpenFlow version 1.3 on a variety of Enterprise devices that use Broadcom ASICs [3]. Allied-Telesis sup- port OpenFlow by running Open vSwitch on the chips, with a cache of flows in hardware. Packets are processed by a software implementation of Open vSwitch by default, and for every new flow the software processes, it installs a rule in hardware applying the same actions to all subsequent traffic belonging to that flow.
The number of hardware flows is limited, from 117 on the IE210L series switches to 8183 on the x950 series switches. Traffic processed by software is orders of magnitude slower than hardware, so this gives an upper limit on the amount of devices that can be active on the device at a time, depending on how many flows each device creates. Because IPv6 addresses are so long, matching IPv6 addresses requires two entries per flow, effectively halving the number of available hardware flows.
Some features are not able to be supported in hardware, depending on the device. For instance, the x530 series (Enterprise switches with 20–40 1Gb ports and 4 10Gb ports) support the hardware flows using an ACL system, and therefore cannot match MPLS headers, or modify MAC addresses for multicast traffic in hardware. These flows are handled in software instead (suffering a significant reduction in performance).
Arista EOS supports OpenFlow version 1.0, and allows for the use of a routing recirculation-interface to process packets multiple times. This sets a port into MAC loopback mode, meaning all packets output to the port will immediately be redirected to the switch. This does not require a transceiver to be present,
and can recirculate packets at the maximum bandwidth for the port.