Toward Hardware Support for a Flexible
Sketch-based Network Traffic Monitoring System
Theophilus Wellem*§, Yu-Kuen Lai¶, Chao-Yuan Huang¶, Wen-Yaw Chung* *Department of Electronic Engineering, ¶Department of Electrical Engineering
Chung Yuan Christian University, Zhongli 32023, Taiwan Email: {g10202604, ylai, g10279008, eldanny}@cycu.edu.tw
§Department of Informatics
Satya Wacana Christian University, Salatiga 50711, Indonesia Email: [email protected]
Abstract— Sketch-based data streaming algorithms are
capable of summarizing high-speed network traffic in real-time without packet sampling. Therefore, they are used in many networking applications to make accurate estimates of traffic flow statistics. However, fixed hardware implementation of the sketch counters limits the flexibility of re-using the same data structure for various monitoring applications. In this paper, we propose a generic scheme for collecting flow-level data using high-speed memories on the NetFPGA-10G platform. The purpose is to enable sketch-based algorithms making statistics measurements in various network monitoring applications in a flexible manner. Experiments using real Internet traces are conducted for entropy estimation, superspreader detection, and change detection applications. The results show that the proposed scheme can support flexible selection of keys and counters for the monitoring applications achieving high measurement accuracy.
Keywords—Network traffic monitoring; Sketch; Hash table; Linear probing; NetFPGA-10G
I. INTRODUCTION
Sketch-based data streaming algorithms are widely used in network traffic monitoring to obtain accurate estimates of traffic flow metrics such as entropy [1], [2], flow size distribution [3], heavy hitters [4], heavy changers [5], and superspreaders [6]. Entropy of source IP and destination port, for example, can be utilized for detecting port scan attack. In contrast to per-flow monitoring tool such as NetFlow [7] that uses packet sampling to reduce number of flows collected, sketch-based algorithms employ sketch data structure to
summarize network traffic into a compact representation without packet sampling, using only a small amount of memory space. Because the sketch data structure is updated for each incoming packet, its update process is the critical bottleneck. Therefore, the sketch is implemented by using fast memory (SRAM) in the network device line card.
Sketch-based data streaming algorithms are efficient and have provable memory-accuracy tradeoffs for traffic flow measurements. However, they are too application-specific to be implemented to perform primitive operations on network devices [8], [9]. Sketch-based algorithms require a separate instance of sketch data structure for each flow key, which is defined according to the measurement tasks of interests, as shown in Fig. 1. Current implementation of sketch-based
algorithms on network device is not flexible since once the data structure is implemented in hardware for a measurement task, another monitoring application that requires different flow key and update value cannot re-use the data structure.
Motivated by the previous works of a "minimalist" approach for flow monitoring system [9], [10] and OpenSketch [8], this paper presents a data plane scheme to collect network traffic flows without packet sampling. The goal is to create a flexible sketch-based traffic monitoring system where the data plane can enable flexible selection of flow keys and counters for various monitoring applications. Rather than maintaining sketch data structure for each flow key in data plane, the proposed data plane scheme employs two tables: flow counter
table and flow key table for collecting traffic flows. The main
contributions of this work are:
• We propose a data plane scheme for flow-level data
collection to support more flexible implementation of sketch-based monitoring applications. Various monitoring applications have greater flexibility of choosing which flow keys and counters to monitor and measure.
• We present the evaluation of the scheme using three
monitoring applications, entropy estimation, superspreader detection, and change detection, to show the feasibility of the data plane scheme.
The remainder of this paper is organized as follows. Sketch for network traffic measurements is discussed in Section II. Section III describes the system architecture. Hardware implementation overview is outlined in Section IV. Section V explains three monitoring applications used to evaluate the system. In Section VI, the system's evaluation with trace-driven tests using real traffic traces is presented. Section VII reviews some related works, and Section VIII concludes this paper.
II. BACKGROUND
Sketch is a data structure representing the compact summaries of a data stream. It can be queried to obtain statistics about the data stream. A sketch is an array of counters
[ ][ ]
C i j , 1≤ ≤i d, 1≤ j≤w. It consists of d hash tables of
size w. Each row is associated to a hash function selected
from a universal class of hash functions. When an item ( , )
a= k v arrives, the key k is hashed by d hash functions
and the corresponding counters are updated by v. A query of
the sketch can yield the estimated counts for the key k. For
example, query of a key to Count-Min sketch [11] returns the minimum value among all counters that correspond to that key. Fig. 2 shows the sketch data structure.
To use sketch for traffic measurement, the sketch data structure must be updated based on a predefined flow key, which is a combination of tuples from the packet header. The update value can be the packet size or packet count. For example, heavy hitters detection application uses the source IP address and packet count as its key and updated value, respectively. In change detection application [5], the sketch is updated using source IP address and packet size. In superspreader detection applications [6], [12], [13], the update process sets the bit in bitmap sketch based on source IP address and pair of source and destination IP addresses.
III. SYSTEM ARCHITECTURE
The architecture for a flexible sketch-based traffic monitoring system is shown in Fig. 3. The 104-bit 5-tuple (source IP, destination IP, source port, destination port, and protocol) is extracted from each packet by the header parser module as the flow key. The hash function module then
computes a 32-bit hash value from the 104-bit flow key as the flow ID using a tabulation-based 2-universal hash function (TabChar). This mapping from 104-bit to 32-bit is collision-free with high probability [14]. The 32-bit flow ID is then hashed again using a 5-universal hash function to obtain a 32-bit hash value. The lower 19-32-bit of this hash value is the memory (SRAM) address where the flow's packet and byte counts are stored. This system utilizes two tables. The flow counter table stores the flow's 16-bit ID, packet count, and byte
count, while the 5-tuple flow key is stored in the flow key table,
which is implemented as a FIFO in DRAM.
During an observation interval, when a flow entry is created, its 16-bit ID and counters are stored in the flow counter table. This 16-bit ID is obtained from hashing the 3bit flow ID (i.e., the result from TabChar hash) using a 2-universal hash function. A Bloom filter is used to check whether the 5-tuple flow key already exists in the flow key table. If the flow does not exist, then the 5-tuple is added into the flow key table. After the observation interval, flow data collected in both tables are moved to the host CPU and can be utilized as required by the monitoring applications.
The modules presented in the upper part of Fig. 3 are implemented in software on multi-core CPU for parallel processing of sketches, while the bottom part is implemented in FPGA hardware. The hardware development platform used for implementation is NetFPGA-10G [15], which contains a Virtex-5 XC5VTX240T FPGA, four 10GbE SFP+ interfaces, three banks of 9MB QDRII SRAM with a total capacity of 27MB, and four 72MB RLDRAM II memories. The main hardware modules for flow processing inside the system is depicted in Fig. 4.
IV. HARDWARE OVERVIEW
In the NetFPGA-10G platform, incoming packets from four 10GbE interfaces are collected in four ingress queues, one for each interface. An input arbiter module then takes the packet out from these queues in round robin policy, and sends it to the next module in the data path. The field extractor module extracts the 104-bit 5-tuple and 16-bit packet length from each packet header and sends it to the hash function module.
Hash Function: Three hash modules are utilized to implement
the hash functions required. The first module implements the tabulation-based 2-universal hashing (TabChar) to process the 104-bit 5-tuple flow key to a 32-bit flow ID. The second module hashes the 32-bit flow ID using a 5-universal hash function to obtain the memory address where the flow's packet and byte counts are stored. This module also computes a 16-bit ID from the 32-bit flow ID to be stored in the flow counter table. The computation of 5-universal hash function is based on the four degree polynomial over some prime field as follows. Fig. 2. Sketch data structure and its update operation.
4
0
( ) i mod
i i
h x a x p
=
=
(1)CW-trick [16] is used to accelerate the 5-universal hash
computation without modulo operation. The 5-universal hash function is implemented in a pipeline fashion and utilized seven embedded multipliers on the Virtex-5 FPGA. The third module calculates hash values for the Bloom filter using three 2-universal hash functions, ( ) (h x = ax b+ ) modp. Again, to
avoid slow modulo operation, multiply, shift, and bitwise AND operations, as in CW-trick, are used.
Flow Counter Table: The flow counter table is implemented
using a hash table with linear probing, and a 5-universal hash function is used to handle hash collisions with constant number of probes per update/lookup. Using 5-universal hash function, the average number of probes per update/lookup is a constant less than four [14], allowing fast and deterministic update operation of the flow counters. The flow counter table is stored in the QDRII SRAM and hash collision is handled by comparing the input ID to multiple entries in a bucket in parallel. The linear probing mechanism makes use of the four-word burst access of the QDRII SRAM. A read burst retrieves four IDs from the memory. These flow IDs are then compared in parallel to the input ID. If the input is a new ID, a flow entry is created and its counters are updated. Otherwise, only the counters are modified and then written back to the memory. Updating counters in the table requires a read-modify-write operation for each incoming packet at wire-speed. With a flow entry consisting of a 16-bit flow ID, a 32-bit packet counter and a 60-bit byte counter, the flow counter table can accommodate up to 2 million flow entries in 27MB QDRII SRAM.
Bloom filter: A Bloom filter is used to check whether a 5-tuple
flow key is already exist in the flow key table and to prevent the addition of a duplicated 5-tuple flow key into the flow key table. Recall that Bloom filter allows false positive, the size of Bloom filter and number of hash functions must be chosen properly to reduce the false positive rate. Assume that the Bloom filter size with m bits stores n elements, then the
false-positive rate (FPR) is
1 1 1
(
1)
k kn
k kn m
FPR e
m
−
= − − ≈ −
(2)
Theoretically, for fixed m, the number of hash functions k
that is needed to minimize the false positive rate is
(
)
ln 2k= m n . For the system herein, k is set to 3, while
various m values are utilized to observe the effect of filter size
on the measurement accuracy. The 1.42 MB on-chip Block RAM (BRAM) is used to store the Bloom filter.
Flow Key Table: The flow key table is stored as a FIFO in
RLDRAM II. Four RLDRAM II chips are available on-board with a total size of 288 MB. For each entry in the FIFO, it is capable of holding a 104-bit 5-tuple flow key (13 bytes per entry). The FIFO can contain up to 512K flow entries utilizing only 6.5 MB of the RLDRAM space. Two RLDRAM II controllers are used in the design, each controller connects to two RLDRAMs. Using burst length of two, one controller can write a 72-bit data with latency of 5 clock cycles. The latency is depend on the RLDRAM configuration mode.
Hardware-Software communication: The time that is
required for the monitoring application runs on the host CPU to read the flow data from SRAM and DRAM via the PCIe interface is important for effective operation of the system. Given the table size and number of flows that are processed in the system herein, the required time is less than 10 ms. Furthermore, the processing time on CPU side (for sketch updates and queries) is much less than the commonly used observation interval. Communication between hardware and software utilizes the register infrastructure in the NetFPGA-10G platform. Applications can read data from the hardware or write data to the hardware using these registers.
V. MONITORING APPLICATIONS
A. Entropy Estimation
Entropy is a measure of the amount of randomness of information content in the distribution of a random variable. Let X be a discrete random variables that can take values from
the set of [ ] {1,2,3, , }s = n , mi denotes the frequency of item
[ ]
i∈ s , and
1
n i i
m m
=
=
is the total number of items. Theentropy of X is defined as [17]: H X( )= −
ni=1pilog2 pi , where pi =m mi is the probability of X taking the value i, and the entropy norm of X is defined as2
( ) ( ) log norm
H X =H X m . The entropy norm value is
minimum (zero) if all items in the data stream are the same, and is maximum (log2m) if all the items in the data stream are
distinct [1]. In traffic monitoring context, the distributions of traffic attributes (e.g. distribution of source IP address, destination IP address, and ports) are important for network monitoring system, especially for fine-grained traffic analysis and classification. Entropy is utilized as a metric for anomaly detection. For example, entropy of source IP and destination port can be used to detect port scan attack.
B. Superspreader Detection
Detecting traffic sources that communicate with a large number of distinct destinations during a measurement interval is important for detecting network anomalies. These sources are known as superspreaders [18] or super sources [6]. For example, an abrupt increase in the number of connections from many distinct sources to a particular destination in a short period, may be the evidence of a distributed denial-of-service (DDoS) attack. Conversely, detecting destinations that contacted by a large number of distinct sources (super destinations) can be useful for detecting the victims of DDoS attack. The goal of superspreader detection is to find hosts that have a large number of connections to distinct destinations above a predefined threshold in an observation interval. To identify such host, the superspreader detection module needs to count the number of connections to distinct destination (or fan-out) of that host. Counting the fan-out of each host in the
network requires large memory space to store the host’s source IP and its fan-out counter. Therefore, estimation using bitmap and sketch-based algorithms are commonly used [6], [12], [13].
C. Change detection
The change detection's goal is to identify flows whose contribution to changes in size (packet counts) or changes in volume (byte counts) of total traffic over several consecutive observation intervals are above a predefined threshold. These flows are also known as heavy changers. Let At =a a1, ,2 be the input packet stream in observation interval t. Each packet
i
a is associated with a key k and an update value v .
Following the algorithm in [5], the source IP address and packet size are used as the key and update value, respectively. Let ( )S to denotes the sketch that accumulate the byte counts in interval t and Sf( )t is the forecast sketch computed using a
forecasting model (e.g. moving average). The forecast error sketch is then computed as ( )S te =S to( )−S tf( ). A threshold alarm TA is chosen based on the estimated second moment of the forecast error sketch and a threshold parameter T
determined by application (TA =T× EstF2
(
S to( ))
). Thechange detection task is to find all keys (source IP addresses) whose volume, queried from ( )S te , is above TA.
Although only three applications above are utilized for the system’s evaluation, other monitoring applications may also use the flow data collected by the hardware. For example, the flow size distribution (FSD) estimation, which is useful for network service providers to know the usage pattern of their networks by customers [3]. Given a flow size x, the goal of
FSD estimation is to find the number of flows that have x
packets in an observation interval. In this case, the monitoring application can rely on the flow counter and flow key tables to obtain flow data and estimate the FSD.
VI. EVALUATION
A. Experiment Setup
Trace-driven tests using CAIDA traces [19] are conducted to evaluate the system. The traces were collected over a
10Gbps Ethernet link of a Tier1 ISP. It comprises of 30 million packets and 1.4 million distinct 5-tuple flows, with a duration of 60s. The number of source-destination IP pairs and distinct source IP addresses in the trace are 1 million and 450k, respectively. The observation interval is set as 15s in the experiments. In this interval, the number of distinct 5-tuple flows is ranged from 445k to 510k. Entropy estimation, superspreader detection and change detection applications are run on the host CPU, and utilize the flow keys and counters collected in both tables to do their measurement task. For change detection, a 15-minute trace from MAWI is used [20]. It is a part of 24h-long trace in a trans-Pacific line (18 Mbps CAR on 100 Mbps link) that is collected at MAWI's samplepoint-B. The trace consists of 51,792 distinct source IP addresses and 374,561 distinct source-destination IP flows.
B. Experiment Results
1) Flow Counter Table and Linear Probing: Probe length
per key for different table sizes are shown in Fig. 5. When a 512K-entry table is utilized with observation interval of 15s, 84.1% of the flows have probe length less or equal to four. The maximum probe length of 350 is observed by one flow. After all flows were inserted into the table, the load factor is 85.13%. For 5s and 10s observation intervals, the percentage of flows that have probe length less or equal to four are 98.97% and 92.99%, respectively. If a table size of 1M entries is used in 15s observation interval, up to 98.56% of the flows have probe length less or equal to four, and the maximum probe length decreases to 27. For the 1.44% of flows that have probe length more than four, only one flow has the maximum probe length of 27. The load factor after all flows were inserted into the table is 42.56%. For 5s and 10s intervals, the percentage of flows that have probe length less or equal to four are 99.95% and 99.48%, respectively. These experiment results confirmed that by using 5-universal hash function, most of flows have probe length less or equal to four.
2) Entropy estimation: The entropy norm of 5-tuple flow,
source IP address and destination IP address. Different sizes of Bloom filter are used in the experiments. As presented, the system can yield estimation errors in an acceptable range for anomaly detection. The estimation accuracy increases with the size of Bloom filter. For Bloom filter size greater than 128 KB, the estimation errors of 5-tuple flow, source IP address, and destination IP address are less than 2%.
3) Superspreader detection: For superspreader detection,
source IP-destination IP pairs data from the flow key table in RLDRAM FIFO are utilized. Here, a superspreader is defined as a source whose fan-out exceeds 200. False negative ratio (FNR) and false positive ratio (FPR) are used to measure the superspreader identification accuracy, while the fan-out estimation accuracy is obtained from scatter plots. The FNR is the number of unidentified superspreaders divided by the number of actual superspreaders, while the FPR is number of non-superspreaders that are incorrectly identified as superspreaders divided by the number of actual superspreaders. FPR and FNR of superspreader detection based on two methods are shown in Table I. The first method (2HashTable) maintains two hash tables. One hash table is used to remove duplicated source IP-destination IP pairs, and the other hash table for counting the fan-out. This method is traditionally used for exact fan-out counting, and is very accurate. The FNR is due to the false positive of Bloom filter when the 5-tuple is added into the flow key table. The second method (FM-BF-HT) [12] combines an FM sketch, a Bloom filter, and a hash table. This method has larger FPR and FNR. The FNR is due to the FM sketch filtering, while the FPR is caused by hash collisions since no collision resolution is implemented for the hash table. The accuracy of fan-out estimation is shown in Fig. 7. The closer the points to the diagonal line, the more accurate the estimation is. The mean errors in the fan-out estimation for 2HashTable and FM-BF-HT are 0.11% and 3.6%, respectively.
4) Change detection: The experiments using k-ary
sketch-based change detection algorithm [5] with Moving Average forecast model are based on the parameters listed as follows: Number of hash functions H =5, number of sketch entries
32K
K= , Moving Average window size W =3, and length
of observation interval Δ =t 60s. The threshold parameter T
is varied for all experiments. The detection accuracy is measured using false positive and false negative rates. The false positives are flow keys whose volume do not change but are incorrectly reported, while the false negatives are flow keys that have volume changes but are not reported.
Using a 15-minute trace, there are 15 observation intervals in one experiment. The first forecast sketch Sf( )t is computed
at the beginning of fourth interval since W =3 , while the
forecast error sketch ( )S te is calculated at the end of fourth interval after the observed sketch ( )S to of fourth interval is available. The threshold alarm TA then can be determined. At
the beginning of fifth interval, the forecast error sketch ( )S te is available and queried using the current interval incoming source IP address to find the heavy changers. From this interval onward, the change detection process reports its results in each interval. The experiment results are reported in Table II. As shown, the change detection application achieves high accuracy in detecting heavy changers with very low false positives and false negatives. As an example, in 14th interval with T =0.1, the application reports 10 heavy changers and
only two of them are false negatives. For other intervals, all heavy changers are reported correctly.
VII. RELATED WORK
NetFlow [7] is a tool developed by Cisco for traffic flow monitoring. The flow exporter in NetFlow is responsible for creating flow records from network traffic by capturing packet headers. Sampling and filtering are performed before a flow entry is created or updated in the flow cache. These flow records are stored in the flow cache until they are considered to have expired. When a flow record expires, it is exported to the flow collector. The flow data collected in the flow collector are then used for analysis. To some extent, the system presented in this work is similar to NetFlow. However, the system only collect the flow's 5-tuple and packet/byte counts. It also does not use the expiration mechanism of NetFlow, and simply moves the flow entry from the tables to host CPU at the end of observation interval.
Liu et al. [10] developed a universal monitoring
architecture (UnivMon), where a simple and generic monitoring primitive run on network device. UnivMon data plane utilizes parallel Count Sketch [21] to collect flow-level data. However, its feasibility of hardware implementation is not discussed. OpenSketch [8] uses a hash-based measurement data plane and sketch data structure to collect traffic flow information. The goal of OpenSketch is to provide a generic and efficient means of measuring network traffic by separating the measurement control plane from the data plane functions. It supports customized flow-level data collection such that the flow to collect and measure is selected by using TCAM-based classification. After an observation interval, the sketch counters stored in SRAM are sent to the controller for further analysis. Different to OpenSketch, the system presented herein uses two tables in the data plane to collect flow information.
Fig. 6. Entropy estimation results of source IP address and destination IP address for different Bloom filter size.
Entropy Estimation
0 0.2 0.4
1 2 3 4
Bloom filter size Interval
1 3 5 7
Relative error (%)
Source IP
128KB 256KB 1024KB512KB
0 0.2 0.4
1 2 3 4
Bloom filter size Interval
1 3 5 7
Destination IP
TABLE I. FPR AND FNR FOR SUPERSPREADER IDENTIFICATION
Interval 2HashTable FM-BF-HT
FPR FNR FPR FNR
1 0 0 0 0.01
2 0 0.01 0 0.04
3 0 0.01 0.04 0.08
4 0 0 0.01 0.03
TABLE II. FALSE POSITIVE AND FALSE NEGATIVE FOR CHANGE DETECTION
Threshold (T) 0.01 0.02 0.05 0.1 0.2 0.5
False positive 0.01 0.01 0 0 0 0 False negative 0.02 0.02 0.02 0.02 0.03 0
VIII. CONCLUSION AND FUTURE WORK
This work proposes a system in which different combination of flow keys can be chosen with great flexibility for the sketch-based monitoring applications to perform various measurement tasks. Based on the real network traffic traces, entropy estimation, superspreader detection, and change detection monitoring applications are utilized for system evaluation. The results of entropy estimation and detection of changes in traffic volume are accurate. Moreover, the system is also capable of identifying the superspreaders with low fan-out estimation error. Currently, based on the preliminary hardware simulations, the system can process traffic flows at 50 Gbps with 64-byte minimum-sized Ethernet frame. In future work, we plan to evaluate more applications and further optimize the hardware design to achieve higher performance.
ACKNOWLEDGMENT
This research was funded in part by the National Science Council, Taiwan, under contract number: MOST 104-2221-E-033-007, MOST 103-2221-E-033-030, and NSC 102-2221-E-033-031.
REFERENCES
[1] A. Lall, V. Sekar, M. Ogihara, J. Xu, and H. Zhang, “Data streaming algorithms for estimating entropy of network traffic,” SIGMETRICS Perform. Eval. Rev., vol. 34, no. 1, pp. 145–156, 2006.
[2] Y.-K. Lai, T. Wellem, and H.-P. You, “Hardware-assisted estimation of entropy norm for high-speed network traffic,” Electronics Letters, vol.
50, no. 24, pp. 1845–1847, 2014.
[3] A. Kumar, M. Sung, J. J. Xu, and J. Wang, “Data streaming algorithms for efficient and accurate estimation of flow size distribution,” in
Proceedings of the Joint International Conference on Measurement and Modeling of Computer Systems, SIGMETRICS ’04/Performance ’04.
New York, NY, USA: ACM, 2004, pp. 177–188.
[4] C. Estan and G. Varghese, “New directions in traffic measurement and accounting,” in Proceedings of the 2002 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, SIGCOMM ’02. New York, NY, USA: ACM, 2002,
pp. 323–336.
[5] B. Krishnamurthy, S. Sen, Y. Zhang, and Y. Chen, “Sketch-based change detection: Methods, evaluation, and applications,” in
Proceedings of the 3rd ACM SIGCOMM Conference on Internet Measurement, IMC ’03. ACM, October 2003, pp. 234–247.
[6] Q. Zhao, A. Kumar, and J. Xu, “Joint data streaming and sampling techniques for detection of super sources and destinations,” in
Proceedings of the 5th ACM SIGCOMM Conference on Internet Measurement, IMC ’05. Berkeley, CA, USA: USENIX Association,
2005, pp. 77–90.
[7] “Cisco NetFlow http://www.cisco.com.”
[8] M. Yu, L. Jose, and R. Miao, “Software defined traffic measurement with OpenSketch,” in Proceedings of the 10th USENIX Conference on Networked Systems Design and Implementation, NSDI’13. Berkeley,
CA, USA: USENIX Association, 2013, pp. 29–42.
[9] V. Sekar, M. K. Reiter, and H. Zhang, “Revisiting the case for a minimalist approach for network flow monitoring,” in Proceedings of the 10th ACM SIGCOMM Conference on Internet Measurement, IMC
’10. New York, NY, USA: ACM, 2010, pp. 328–341.
[10] Z. Liu, G. Vorsanger, V. Braverman, and V. Sekar, “Enabling a ”RISC” Approach for Software-Defined Monitoring Using Universal Streaming,” in Proceedings of the 14th ACM Workshop on Hot Topics in Networks, HotNets-XIV. New York, NY, USA: ACM, 2015, pp. 21:1–
21:7.
[11] G. Cormode and S. Muthukrishnan, “An improved data stream summary: The count-min sketch and its applications,” Journal of Algorithms, vol. 55, no. 1, pp. 58–75, April 2005.
[12] T. Wellem, G.-W. Li, and Y.-K. Lai, “Superspreader Detection System on NetFPGA Platform,” in Proceedings of the Tenth ACM/IEEE Symposium on Architectures for Networking and Communications Systems, ANCS ’14. New York, NY, USA: ACM, 2014, pp. 247–248.
[13] M. Yoon, T. Li, S. Chen, and J.-K. Peir, “Fit a Compact Spread Estimator in Small High-Speed Memory,” IEEE/ACM Transactions on Networking, vol. 19, no. 5, pp. 1253–1264, Oct. 2011.
[14] M. Thorup and Y. Zhang, “Tabulation based 5-universal hashing and linear probing,” in Proceedings of the 12th Workshop on Algorithm Engineering and Experiments ALENEX, 2010, pp. 62–76.
[15] “NetFPGA project site. http://netfpga.org.”
[16] J. Carter and M. Wegman, “Universal classes of hash functions,”
Journal of Computer and System Sciences, vol. 18, no. 2, pp. 143–154,
1979.
[17] C. Shannon, “A mathematical theory of communication,” The Bell System Technical Journal, vol. 27, pp. 379–423, July 1948.
[18] S. Venkataraman, D. Song, P. B. Gibbons, and A. Blum, “New streaming algorithms for fast detection of superspreaders,” in
Proceedings of Network and Distributed System Security Symposium (NDSS), 2005, pp. 149–166.
[19] “CAIDA UCSD Anonymized Internet Traces,” 2012.
[20] MAWI Working Group, “MAWI Working Group Traffic Archive,” http://mawi.wide.ad.jp/mawi/samplepoint-B/20030227/.
[21] M. Charikar, K. Chen, and M. Farach-Colton, “Finding frequent items in data streams,” in Proceedings of the 29th International Colloquium on Automata, Languages and Programming. Springer-Verlag, July 2002,
pp. 693–703. Fig. 7. Exact vs. estimated fan-out in Interval 4. The number of actual
superspreaders in this interval is 76.
Superspreader detection
200 500 1250 2500
200 500 1250 2500
Esti
m
ated sour
ce
fan-out
Exact source fan-out 2HashTable
200 500 1250 2500
200 500 1250 2500 Exact source fan-out
