Target Controllers
5.2 Production Deployments
5.2.5 Updates
How frequently controllers update table entries affects how the tables can be mapped to hardware. If a table is updated infrequently, then it is less important to ensure the update rate is not affected by the mapping. For instance, entries could be split so that fields are matched across multiple tables, or they could
Datapath
Percentage Updates
0%
25%
50%
75%
100%
Redcables aruba-2930Redcables at-tq4600-1Redcables at-tq4600-2Redcables at-tq4600-3Redcables at-x510-1Redcables at-x510-2Redcables at-x510-3Redcables at-x510-4Redcables at-x930-1Redcables at-x930-2 Redcables ovs
eResearch aruba-1geResearch aruba-poe eResearch at1
Scinet 2018 dnoc1034Scinet 2018 dnoc2316Scinet 2018 dnoc2644Scinet 2018 dnoc4026 Scinet noc
NZNOG 2018 aruba-1gNZNOG 2018 aruba- NZNOG 2018 at1REANNZ Office Other Updates Port Updates Route Updates MAC Updates
Figure 5.1:The percentage of updates to datapaths for each type of entry class.
be mapped to tables that use slower match kinds.
Figure 5.1 shows the percentage of updates relating to each type of entry class. The overwhelming majority of updates occur in entry classes relating to Ethernet switching, routing, or ports and VLANs. Many datapaths only received updates relating to Ethernet switching. There were only a handful of other entry classes that received more than 1% of the total updates. The Scinet deployment included a deployment of Poseidon [45], a network security application that could automatically install ACLs in response to anomalies it detects in the network. ACL updates accounted for up to 4.1% of updates to Scinet datapaths.
Port and VLAN updates accounted for up to 18.25% of updates, with the exception of two outliers: one of the Scinet aggregation datapaths, dnoc1034, had 85.6% of its updates for port state changes, and the REANNZ office net-
work had 90.9% of its updates for port state changes. Scinet dnoc1034 had the fewest connected hosts of any of the Scinet devices, and those hosts were stable servers where the MAC addresses did not need to be relearned often.
Most of the port state changes are from the set up and testing stage rather than the conference itself.
The high rate of messages relating to port state changes in the REANNZ of- fice was caused by inefficient handling of VLAN flooding, to accommodate incorrect OpenFlow hardware implementations. The OpenFlow specification requires that a device should not forward packets to the packet’s ingress port, unless the action specifies virtual port IN_PORT as the egress. Due to hard- ware implementations that did not correctly implement this behaviour, Faucet created separate flooding rules for each VLAN on each port. When a port changed state, Faucet would update every rule associated with every VLAN on that port. The REANNZ office had many VLANs, and provides wired con- nections to staff laptops, which were frequently connected to, and disconnected from the network. This resulted in a peak rate of 295 messages relating to port updates in a second. Faucet has since been modified to handle VLAN flooding more efficiently.
5.2.5.1 Table Size
The size of tables impacts table mappings in a number of ways. First and foremost, physical tables must be large enough to hold as many entries as the virtual table requires. The size also impacts how practical it is for tables to be aggregated with other tables, aggregating large tables may exhaust the number of entries, and small tables may be small enough to be combined with entries from another table, provided the update rate is also low.
Entry Class
Maximum Entries
0 100 200 300 400 500
Port ACLs
VLAN assignment
Filter BogonsMAC Learning Termination MAC
IPv4 RoutingIPv6 Routing
Control Plane Whitelist Ethernet Switching
Flooding
Figure 5.2:The maximum number of entries for each entry class in the Redcables OVS datapath
The entry classes with the most entries, for the most part, were those that were updated the most frequently. The exception was ACL entry classes, which could be moderately large, but received few updates. Figure 5.2 shows the maximum number of entries for each entry class in the Redcables OVS datapath, which is typical of all datapaths.
As the smaller tables often had single-digit numbers of entries and were rarely updated, mapping software could easily aggregate them into other tables, and physical hardware could support them with a flexible table such as an ACL table.
datapath Percentage Entry Classes Performing Action 0.00%
25.00%
50.00%
75.00%
100.00%
Redcables at-tq4600-1Redcables at-tq4600-2Redcables at-tq4600-3Redcables at-x510-1Redcables at-x510-2Redcables at-x510-3Redcables at-x510-4Redcables at-x930-1Redcables at-x930-2 Redcables ovs-redcables
eResearch aruba-1geResearch aruba-poe eResearch at1
Scinet dnoc1034Scinet dnoc2316Scinet dnoc2644Scinet dnoc4026 Scinet noc NZNOG aruba-1gNZNOG aruba-poe
NZNOG at1Cardigan REANNZ office drop actions output and controller actions vlan actions set actions goto actions
Figure 5.3:The percentage of entry classes that perform a particular action. Note that entry classes may have multiple possible action sets and that sets may include multiple types of actions.
5.2.5.2 Actions
Actions can affect how the tables in a pipeline can be re-ordered. Certain actions may need to be applied to packets in a specific order, for instance, if two tables set the same field the second table must supersede the first.
Furthermore, actions can affect what entries are applied in subsequent tables.
Figure 5.3 shows the percentage of entry types that perform each action. The most common action for an entry class to apply to a packet is to drop it. This is beneficial when mapping a virtual pipeline to a physical pipeline as, for the most part, it does not matter where in the pipeline a packet is dropped. The
only actions that are impacted by a drop being applied later in the pipeline is a clone action, a count action, or an action that modifies the packet so that it will no longer match the drop entry class.
Modifying a packet and then subsequently matching on the modified field is a difficult operation for hardware to support. All of the Faucet datapaths set VLAN ID fields and then match those fields in later tables. Only Scinet noc and Redcables ovs do this with another field: both datapaths set the Ethernet destination field when routing packets, then matched that field to determine the correct output port for the next hop.
5.2.5.3 Transactions
We identified three main types of transaction that require multiple table up- dates. The first, and by far the most frequent—making up 100% of all messages to some datapaths—were Ethernet switching updates. In the OpenFlow model, Ethernet switching is typically handled in two tables; the first table is respon- sible for learning the port association for each MAC address, and the second is responsible for forwarding packets based on their Ethernet source address.
The second transaction updated flooding rules, as described in Subsection 5.2.5.
The third transaction, and the least common, is when a controller resolves a next-hop. Typically this results in multiple routes and the Ethernet switching table being updated. It is possible to efficiently update next-hops in OpenFlow with Metadata or with Group tables. Faucet does not do this, however, because of inconsistent hardware support.
Overall, this analysis suggests that the OpenFlow protocol is reasonably effi- cient at updating tables, with the only obvious area where improvement could
be made is with layer 2 switching and flooding.