Besides the methods on System.Object, any type that implements IEnumerable<T> is required to implement only one other method,
GetEnumerator(). Yet, doing so makes more than 50 methods available to all types implementing IEnumerable<T> , not including any
overloading—and this happens without needing to explicitly implement any method except the GetEnumerator() method. The additional
functionality is provided through extension methods and resides in the class System.Linq.Enumerable . Therefore, including the using declarative for System.Linq is all it takes to make these methods available.
8. Starting in C# 3.0.
Each method on IEnumerable<T> is a standard query operator; it provides querying capability over the collection on which it operates. In the following sections, we examine some of the most prominent of these
standard query operators. Many of these examples will depend on an Inventor and/or Patent class, both of which are defined in LISTING
15.9 with OUTPUT 15.1.
LISTING 15.9: Sample Classes for Use with Standard Query Operators
8
using System;
using System.Collections.Generic;
public class Program {
public static void Main() {
IEnumerable<Patent> patents = PatentData.Pate Print(patents);
Console.WriteLine();
IEnumerable<Inventor> inventors = PatentData.
Print(inventors);
}
private static void Print<T>(IEnumerable<T> items {
foreach(T item in items) {
Console.WriteLine(item);
} } }
OUTPUT 15.1
Bifocals (1784) Phonograph (1877) Kinetoscope (1888)
Electrical Telegraph (1837) Flying Machine (1903)
Steam Locomotive (1815)
Droplet Deposition Apparatus (1989) Backless Brassiere (1914)
Benjamin Franklin (Philadelphia, PA) Orville Wright (Kitty Hawk, NC)
Wilbur Wright (Kitty Hawk, NC) Samuel Morse (New York, NY)
George Stephenson (Wylam, Northumberland) John Michaelis (Chicago, IL)
Mary Phelps Jacob (New York, NY)
Filtering with Where()
To filter out data from a collection, we need to provide a filter method that returns true or false, indicating whether or not a particular element should be included. A delegate expression that takes an argument and returns a Boolean value is called a predicate, and a collection’s Where() method depends on predicates for identifying filter criteria, as shown in LISTING 15.10. (Technically, the result of the Where() method is an
object that encapsulates the operation of filtering a given sequence with a given predicate.) The results appear in OUTPUT 15.2.
LISTING 15.10: Filtering with System.Linq.Enumerable.Where()
using System;
using System.Collections.Generic;
using System.Linq;
public class Program {
public static void Main() {
IEnumerable<Patent> patents = PatentData.Pate patents = patents.Where(
patent => patent.YearOfPublication.Starts Print(patents);
}
// ...
}
OUTPUT 15.2
Phonograph (1877) Kinetoscope (1888)
Electrical Telegraph (1837) Steam Locomotive (1815)
Notice that the code assigns the output of the Where() call back to IEnumerable<T> . In other words, the output of
IEnumerable<T>.Where() is a new IEnumerable<T> collection. In LISTING 15.10, it is IEnumerable<Patent> .
Less obvious is that the Where() expression argument has not necessarily been executed at assignment time. This is true for many of the standard query operators. In the case of Where(), for example, the expression is passed into the collection and “saved” but not executed. Instead, execution of the expression occurs only when it is necessary to begin iterating over the items within the collection. For example, a foreach loop, such as the one in Print() (in LISTING 15.9), can trigger the expression to be
evaluated for each item within the collection. At least conceptually, the Where() method should be understood as a means of specifying the query regarding what appears in the collection, not the actual work involved with iterating over the items to produce a new collection with potentially fewer items.
Projecting with Select()
Since the output from the IEnumerable<T>.Where() method is a new IEnumerable<T> collection, it is possible to again call a standard query operator on the same collection. For example, rather than just filtering the
data from the original collection, we could transform the data (see LISTING
15.11).
LISTING 15.11: Projection with
System.Linq.Enumerable.Select()
using System;
using System.Collections.Generic;
using System.Linq;
public class Program {
public static void Main() {
IEnumerable<Patent> patents = PatentData.Pate IEnumerable<Patent> patentsOf1800 = patents.W patent => patent.YearOfPublication.Starts IEnumerable<string> items = patentsOf1800.Sel patent => patent.ToString());
Print(items);
}
// ...
}
In LISTING 15.11, we create a new IEnumerable<string> collection. In this case, it just so happens that adding the Select() call doesn’t change the output—but only because Print()’s Console.WriteLine() call used ToString() anyway. Obviously, a transform still occurred on each item from the Patent type of the original collection to the string type of the items collection.
Consider the example using System.IO.FileInfo in LISTING 15.12.
Here fileList is of type IEnumerable<string>. However, using the projection offered by Select , we can transform each item in the
collection to a System.IO.FileInfo object.
Lastly, capitalizing on tuples, we can create an IEnumerable<T>
collection where T is a tuple (see LISTING 15.13 and OUTPUT 15.3).
LISTING 15.12: Projection with
System.Linq.Enumerable.Select() and new
//...
IEnumerable<string> fileList = Directory.EnumerateFil rootDirectory, searchPattern);
IEnumerable<FileInfo> files = fileList.Select(
file => new FileInfo(file));
//...
LISTING 15.13: Projection to Tuple
//...
IEnumerable<string> fileList = Directory.EnumerateFil rootDirectory, searchPattern);
IEnumerable<(string FileName, long Size)> items = fil file =>
{ FileInfo fileInfo = new FileInfo(file);
return ( FileName: fileInfo.Name, Size: fileInfo.Length );
});
//...
OUTPUT 15.3
FileName = AssemblyInfo.cs, Size = 1704
FileName = CodeAnalysisRules.xml, Size = 735 FileName = CustomDictionary.xml, Size = 199 FileName = EssentialCSharp.sln, Size = 40415
FileName = EssentialCSharp.suo, Size = 454656 FileName = EssentialCSharp.vsmdi, Size = 499 FileName = EssentialCSharp.vssscc, Size = 256
FileName = intelliTechture.ConsoleTester.dll, Size = FileName = intelliTechture.ConsoleTester.pdb, Size =
The output of an anonymous type automatically shows the property names and their values as part of the generated ToString() method associated with the anonymous type.
Projection using the Select() method is very powerful. We already saw how to filter a collection vertically (reducing the number of items in the collection) using the Where() standard query operator. Now, via the
Select() standard query operator, we can also reduce the collection horizontally (making fewer columns) or transform the data entirely. In combination, Where() and Select() provide a means for extracting only those pieces of the original collection that are desirable for the current algorithm. These two methods alone provide a powerful collection
manipulation API that would otherwise result in significantly more—and less readable—code.
Advanced Topic: Running LINQ Queries in Parallel
With the widespread availability of computers having multiple processors and multiple cores within those processors, the ability to easily take
advantage of the additional processing power becomes far more important.
To do so, programs need to support multiple threads so that work can happen simultaneously on different CPUs within the computer. LISTING
15.14 demonstrates one way to do this using Parallel LINQ (PLINQ).
LISTING 15.14: Executing LINQ Queries in Parallel
//...
IEnumerable<string> fileList = Directory.EnumerateFil rootDirectory, searchPattern);
var items = fileList.AsParallel().Select(
file =>
{
FileInfo fileInfo = new FileInfo(file);
return new {
FileName = fileInfo.Name, Size = fileInfo.Length };
});
//...
As LISTING 15.14 shows, only a minimal change in code is needed to enable parallel support. All that this example uses is a Microsoft .NET Framework 4–introduced standard query operator, AsParallel() , on the static class
System.Linq.ParallelEnumerable . Using this simple extension
method, the runtime begins executing over the items within the fileList collection and returning the resultant objects in parallel. Each parallel
operation in this case isn’t particularly expensive (although it is relative to the other execution taking place), but consider the burden imposed by CPU- intensive operations such as encryption or compression. Running the query in parallel across multiple CPUs can decrease execution time by a factor corresponding to the number of CPU cores.
An important caveat to be aware of (and the reason why AsParallel() appears as an Advanced Topic rather than in the standard text) is that parallel execution can introduce race conditions, such that an operation on one thread can be intermingled with an operation on a different thread, causing data corruption. To avoid this problem, synchronization
mechanisms are required on data with shared access from multiple threads to force the operations to be atomic where necessary. Synchronization itself, however, can introduce deadlocks that freeze the execution, further
complicating the effective parallel programming.
More details on this and additional multithreading topics are provided in Chapters 19 through 22.
Counting Elements with Count()
Another query frequently performed on a collection of items is to retrieve the count. To support this type of query, LINQ includes the Count() extension method.
LISTING 15.15 demonstrates that Count() is overloaded to simply count all elements (no parameters) or to take a predicate that counts only items identified by the predicate expression.
LISTING 15.15: Counting Items with Count()
using System;
using System.Collections.Generic;
using System.Linq;
public class Program {
public static void Main() {
IEnumerable<Patent> patents = PatentData.Pate Console.WriteLine($"Patent Count: { patents.C Console.WriteLine($@"Patent Count in 1800s: { patents.Count(patent =>
patent.YearOfPublication.StartsWith("
In spite of the apparent simplicity of the Count() statement,
IEnumerable<T> has not changed, so the executed code still iterates over all the items in the collection. Whenever a Count property is directly available on the collection, it is preferable to use that rather than LINQ’s
Count() method (a subtle difference). Fortunately, ICollection<T>
includes the Count property, so code that calls the Count() method on a collection that supports ICollection<T> will cast the collection and call
Count directly. However, if ICollection<T> is not supported, Enumerable.Count() will proceed to enumerate all the items in the collection rather than call the built-in Count mechanism. If the purpose of checking the count is just to see whether it is greater than zero
( if(patents.Count() > 0){...} ), the preferred approach would be to use the Any() operator ( if(patents.Any()){...} ). Any() attempts to iterate over only one of the items in the collection to return a true result, rather than iterating over the entire sequence.
Guidelines
DO use System.Linq.Enumerable.Any() rather than calling patents.Count() when checking whether
there are more than zero items.
DO use a collection’s Count property (if available) instead of calling the
System.Linq.Enumerable.Count() method.
Deferred Execution
One of the most important concepts to remember when using LINQ is deferred execution. Consider the code in LISTING 15.16 and the
corresponding output in OUTPUT 15.4.
LISTING 15.16: Filtering with System.Linq.Enumerable.Where()
using System;
using System.Collections.Generic;
using System.Linq;
// ...
IEnumerable<Patent> patents = PatentData.Pate bool result;
patents = patents.Where(
patent =>
{
if(result =
patent.YearOfPublication.StartsWi {
// Side effects like this in a pr // are used here to demonstrate a
// principle and should generally // avoided
Console.WriteLine("\t" + patent);
}
return result;
});
Console.WriteLine("1. Patents prior to the 19 foreach(Patent patent in patents)
{ }
Console.WriteLine();
Console.WriteLine(
"2. A second listing of patents prior to Console.WriteLine(
$@" There are { patents.Count() } patents prior to 1900.");
Console.WriteLine();
Console.WriteLine(
"3. A third listing of patents prior to t patents = patents.ToArray();
Console.Write(" There are ");
Console.WriteLine(
$"{ patents.Count() } patents prior to 19
//...
OUTPUT 15.4
1. Patents prior to the 1900s are:
Phonograph (1877) Kinetoscope (1888)
Electrical Telegraph (1837) Steam Locomotive (1815)
2. A second listing of patents prior to the 1900s:
Phonograph (1877) Kinetoscope (1888)
Electrical Telegraph (1837) Steam Locomotive (1815)
There are 4 patents prior to 1900.
3. A third listing of patents prior to the 1900s:
Phonograph (1877) Kinetoscope (1888)
Electrical Telegraph (1837) Steam Locomotive (1815)
There are 4 patents prior to 1900.
Notice that Console.WriteLine("1. Patents prior...) executes before the lambda expression. This characteristic is very important to
recognize because it is not obvious to those who are unaware of its
importance. In general, predicates should do exactly one thing—evaluate a condition—and should not have any side effects (even printing to the
console, as in this example).
To understand what is happening, recall that lambda expressions are delegates— references to methods—that can be passed around. In the context of LINQ and standard query operators, each lambda expression forms part of the overall query to be executed.
At the time of declaration, lambda expressions are not executed. In fact, it isn’t until the lambda expressions are invoked that the code within them begins to execute. FIGURE 15.2 shows the sequence of operations.
FIGURE 15.2: Sequence of operations invoking lambda expressions
As FIGURE 15.2 shows, three calls in LISTING 15.14 trigger the lambda
expression, and each time it is fairly implicit. If the lambda expression were expensive (such as a call to a database), it would therefore be important to minimize the lambda expression’s execution.
First, the execution is triggered within the foreach loop. As described earlier in the chapter, the foreach loop breaks down into a
MoveNext() call, and each call results in the lambda expression’s execution for each item in the original collection. While iterating, the
runtime invokes the lambda expression for each item to determine whether the item satisfies the predicate.
Second, a call to Enumerable ’s Count() (the function) triggers the lambda expression for each item once more. Again, this is subtle behavior because Count (the property) is very common on collections that have not been queried with a standard query operator.
Third, the call to ToArray() (or ToList(), ToDictionary() , or ToLookup()) evaluates the lambda expression for each item. However, converting the collection with one of these “To” methods is extremely helpful. Doing so returns a collection on which the standard query operator has already executed. In LISTING 15.14, the conversion to an array means that when Length is called in the final Console.WriteLine() , the underlying object pointed to by patents is, in fact, an array (which obviously implements IEnumerable<T> ); in turn, System.Array ’s implementation of Length is called and not
System.Linq.Enumerable ’s implementation. Consequently, following a conversion to one of the collection types returned by a “To” method, it is generally safe to work with the collection (until another standard query operator is called). However, be aware that this will bring the entire result set into memory (it may have been backed by a database or file prior to this step). Furthermore, the “To” method will take a snapshot of the underlying data, so that no fresh results will be returned upon requerying the “To”
method result.
We strongly encourage you to review the sequence diagram in FIGURE 15.2 along with the corresponding code and recognize that the deferred
execution of standard query operators can result in extremely subtle
triggering of the standard query operators; therefore, developers should use caution and seek to avoid unexpected calls. The query object represents the query, not the results. When you ask the query for the results, the whole query executes (perhaps even again) because the query object doesn’t know that the results will be the same as they were during a previous execution (if one existed).
Note
To avoid such repeated execution, you must cache the data retrieved by the executed query. To do so, assign the data to a local collection using one of the “To” collection methods. During the assignment call of a “To” method, the query obviously executes. However, iterating over the assigned
collection after that point will not involve the query expression any further.
In general, if you want the behavior of an in-memory collection snapshot, it is a best practice to assign a query expression to a cached collection to avoid unnecessary iterations.
Sorting with OrderBy() and ThenBy()
Another common operation on a collection is to sort it. Sorting involves a call to System.Linq.Enumerable ’s OrderBy() , as shown in LISTING
15.17 and OUTPUT 15.5.
LISTING 15.17: Ordering with
System.Linq.Enumerable.OrderBy() /ThenBy()
using System;
using System.Collections.Generic;
using System.Linq;
// ...
IEnumerable<Patent> items;
Patent[] patents = PatentData.Patents;
items = patents.OrderBy(
patent => patent.YearOfPublication) .ThenBy(
patent => patent.Title);
Print(items);
Console.WriteLine();
items = patents.OrderByDescending(
patent => patent.YearOfPublication) .ThenByDescending(
patent => patent.Title);
Print(items);
//...
OUTPUT 15.5
Bifocals (1784)
Steam Locomotive (1815) Electrical Telegraph (1837) Phonograph (1877)
Kinetoscope (1888) Flying Machine (1903) Backless Brassiere (1914)
Droplet Deposition Apparatus (1989)
Droplet Deposition Apparatus (1989) Backless Brassiere (1914)
Flying Machine (1903) Kinetoscope (1888) Phonograph (1877)
Electrical Telegraph (1837) Steam Locomotive (1815) Bifocals (1784)
The OrderBy() call takes a lambda expression that identifies the key on which to sort. In LISTING 15.17, the initial sort uses the year in which the patent was published.
Notice that the OrderBy() call takes only a single parameter,
keySelector , to sort on. To sort on a second column, it is necessary to use a different method: ThenBy() . Similarly, code would use ThenBy() for any additional sorting.
OrderBy() returns an IOrderedEnumerable<T> interface, not an IEnumerable<T> . Furthermore, IOrderedEnumerable<T> derives from IEnumerable<T> , so all the standard query operators (including
OrderBy() ) are available on the OrderBy() return. However, repeated calls to OrderBy() would undo the work of the previous call such that the end result would sort by only the keySelector in the final
OrderBy() call. For this reason, you should be careful not to call OrderBy() on a previous OrderBy() call.
Instead, you should specify additional sorting criteria using ThenBy() . Although ThenBy() is an extension method, it is not an extension of
IEnumerable<T> but rather of IOrderedEnumerable<T>. The method, also defined on System.Linq.Extensions.Enumerable , is declared as follows:
public static IOrderedEnumerable<TSource>
ThenBy<TSource, TKey>(
this IOrderedEnumerable<TSource> source, Func<TSource, TKey> keySelector)
In summary, use OrderBy() first, followed by zero or more calls to ThenBy() to provide additional sorting “columns.” The methods
OrderByDescending() and ThenByDescending() provide the same functionality except that they sort items in descending order. Mixing and matching ascending and descending methods is not a problem, but if sorting
items further, you would use a ThenBy() call (either ascending or descending).
Two more important notes about sorting are warranted. First, the actual sort doesn’t occur until you begin to access the members in the collection, at which point the entire query is processed. You can’t sort unless you have all the items to sort, because you can’t determine whether you have the first item. The fact that sorting is delayed until you begin to access the members is due to deferred execution, as described earlier in this chapter. Second, each subsequent call to sort the data (e.g., Orderby() followed by
ThenBy() followed by ThenByDescending()) does involve additional calls to the keySelector lambda expression of the earlier sorting calls.
In other words, a call to OrderBy() will call its corresponding
keySelector lambda expression once you iterate over the collection.
Furthermore, a subsequent call to ThenBy() will again make calls to OrderBy() ’s keySelector .
Guidelines
DO NOT call an OrderBy() following a prior
OrderBy() method call. Use ThenBy() to sequence items by more than one value.
Beginner Topic: Join Operations
Consider two collections of objects as shown in the Venn diagram in FIGURE
15.3. The left circle in the diagram includes all inventors, and the right circle contains all patents. The intersection includes both inventors and patents, and a line is formed for each case where there is a match of inventors to patents. As the diagram shows, each inventor may have multiple patents and each patent can have one or more inventors. Each patent has an inventor, but in some cases, inventors do not yet have patents.
FIGURE 15.3: Venn diagram of inventor and patent collections
Matching up inventors within the intersection to patents is an inner join.
The result is a collection of inventor/patent pairs in which both patents and