Using library algorithms
6.1 Analyzing strings
6.1.3 Finding URLs
As the last of our examples of character manipulation, let's write a function that finds Web addresses, called uniform resource locators (URLs), that are embedded in a string. We might use such a function by creating a single string that holds the entire contents of a document.
The function would then scan the document and find all the URLs in it. A URL is a sequence of characters of the form
protocol-name: //resource-name
where protocol-name contains only letters, and resource-name may consist of letters, digits, and certain punctuation characters. Our function will take a string argument and will look for instances of :// in that string. Each time we find such an instance, we'll look for the
protocol-name that precedes it, and the resource-name that follows it.
Because we want our function to find all the URLs in its input, we'll want it to return a
vector<string>, with one element for each URL. The function executes by moving the iterator b through the string, looking for the characters :// that might be a part of a URL. If we find
these characters, it looks backward to find the protocol-name, and it looks forward to find the resource-name:
vector<string> find_urls(const string& s) {
vector<string> ret;
typedef string::const_iterator iter;
iter b = s.begin(), e = s.end();
// look through the entire input while (b != e) {
// look for one or more letters followed by ://
b = url_beg(b, e);
// if we found it if (b != e) {
// get the rest of the URL iter after = url_end(b, e);
// remember the URL
ret.push_back(string(b, after));
// advance b and check for more URLs on this line b = after;
} }
return ret;
}
We start by declaring ret, which is the vector into which we will put the URLs as we find them, and by obtaining iterators that delimit the string. We will have to write the url_beg and
url_end functions, which will find the beginning and end of any URL in the input. The url_beg function will be responsible for identifying whether a valid URL is present and, if so, for
returning an iterator that refers to the first character of the protocol-name. If it does not identify a URL in the input, then it will return its second argument (e in this case) to indicate failure.
If url_beg finds a URL, the next task is to find the end of the URL by calling url_end. That function will search from the given position until it reaches either the end of the input or a character that cannot be part of a URL. It will return an iterator positioned one past the last character in the URL.
Thus, after the calls to url_beg and url_end, the iterator b denotes the beginning of a URL, and the iterator after denotes the position one past the last character in the URL:
We construct a new string from the characters in this range, and push that string onto the back of ret.
All that remains is to increment the value of b and to look for the next URL. Because URLs cannot overlap one another, we set b to (one past) the end of the URL that we just found, and continue the while loop until we've looked at all the input. Once that loop exits, we return the vector that contains the URLs to our caller.
Now we have to think about url_beg and url_end. The url_end function is simpler, so we'll start there:
string::const_iterator
url_end(string::const_iterator b, string::const_iterator e) {
return find_if(b, e, not_url_char);
}
This function just forwards its work to the library find_if function, which we used in §6.1.1/103.
The predicate that we pass to find_if is one that we will write, named not_url_char. It will return true when passed a character that cannot be in a URL:
bool not_url_char(char c) {
// characters, in addition to alphanumerics, that can appear in a URL static const string url_ch = "~;/?:@=&$-_.+!*'(),";
// see whether c can appear in a URL and return the negative return !(isalnum(c) ||
find(url_ch.begin(), url_ch.end(), c) != url_ch.end());
}
Despite being small, this function uses a fair bit of new material. First is the use of the static storage class specifier. Local variables that are declared to be static are preserved across invocations of the function. Thus, we will create and initialize the string url_ch only on the first
call to not_url_char. Subsequent calls will use the object that the first call created. Because url_ch is a const string, its value will not change once we have initialized it.
The not_url_char function also uses the isalnum function, which the <cctype> header defines. This function tests whether its argument is an alphanumeric character (a letter or a digit).
Finally, find is another algorithm that we haven't used yet. It is similar to find_if, except that instead of calling a predicate, it looks for the specific value given as its third argument. As with find_if, if the value that we want is present, the function returns an iterator denoting the first occurrence of the value in the given sequence. If the value is not found, then find returns its second argument.
With this information in hand, we can now understand the not_url_char function. Because we negate the value of the entire expression before we return it, not_url_char will yield false if c is a letter, a digit, or any of the characters in url_ch. If c is any other value, the function returns true.
Now the hard part begins: implementing url_beg. This function is messy, because it must deal with the possibility that the input might contain :// in a context that cannot be a valid URL. In practice, we'd probably have a list of acceptable protocol-names and look only for those. For simplicity, though, we'll limit ourselves to being sure that one or more letters precede the ://
separator, and at least one character follows it:
string::const_iterator
url_beg(string::const_iterator b, string::const_iterator e) {
static const string sep = "://";
typedef string::const_iterator iter;
// i marks where the separator was found iter i = b;
while ((i = search(i, e, sep.begin(), sep.end())) != e) {
// make sure the separator isn't at the beginning or end of the line if (i != b && i + sep.size() != e) {
// beg marks the beginning of the protocol-name iter beg = i;
while (beg != b && isalpha(beg[-1])) --beg;
// is there at least one appropriate character before and after the separator?
if (beg != i && !not_url_char(i[sep.size()])) return beg;
}
// the separator we found wasn't part of a URL advance i past this separator i += sep.size();
}
return e;
}
The easy part is to write the function header. We know that we'll be passed two iterators denoting the range in which to look, and that we'll return an iterator that denotes the beginning of the first URL in that range, if one exists. We also declare and initialize a local string, which will hold the characters that make up the separator that identifies a potential URL. Like url_ch in the not_url_char function (§6.1.3/107), this string is static and const. Thus, we will not be able to change the string, and its value will be created only on the first invocation of url_beg. The function executes by placing two iterators into the string delimited by b and e:
The iterator iwill denote the beginning of the URL separator, if any, and beg will indicate the beginning of the protocol-name, if any.
The function first looks for the separator, by calling search, a library function that we haven't used before. This function takes two pairs of iterators: The first pair denotes the sequence in which we are looking, and the second pair denotes the sequence that we wish to locate. As with other library functions, if search fails, it returns the second iterator. Therefore, after the call to search, either i denotes (one past) the end of the input string, or it denotes a : that is followed by //.
If we found a separator, the next task is to get the letters (if any) that make up the
protocol-name. We first check whether the separator is at the beginning or end of the input. If the separator is in either of those places, we know that we don't have a URL, because a URL has at least one character on each side of its separator. Otherwise, we need to try to position
the iterator beg. The inner while loop moves beg backward through the input until it hits either a nonalphabetic character or the beginning of the string. It uses two new ideas: The first is the notion that if a container supports indexing, so do its iterators. In other words, beg[-1] is the character at the position immediately before the one that beg denotes. We can think of beg[-l]
as an abbreviation for *(beg - 1). We'll learn more about such iterators in §8.2.6/148. The second new idea is the isalpha function, defined in <cctype>, which tests whether its argument is a letter.
If we were able to advance the iterator over as much as a single character, we assume that we've found a protocol-name. Before returning beg, we still have to check that there's at least one valid character following the separator. This test is more complicated. We know that there is at least one more character in the input, because we're inside the body of an if that
compares the value of i + sep.size() with e. We can access the first such character as
i[sep.size()], which is an abbreviation for *(i + sep.size()). We test whether that character can appear in a URL by passing the character to not_url_char. This function returns true if the character is not valid, so we negate the return to check whether the character is valid.
If the separator is not part of a URL, then the function advances i past the separator and keeps looking.
This code uses the decrement operator, which we mentioned in the operator table in
§2.7/32, but which we have not previously used. It works like the increment operator, but it decrements its operand instead. As with the increment operator, it comes in prefix and postfix versions. The prefix version, which we use here, decrements its operand and returns the new value.
6.2 Comparing grading schemes
In §4.2/61, we presented a grading scheme that based students' final grades, in part, on their median homework scores. Devious students can exploit this scheme by deliberately not turning in all their homework assignments. After all, the bottom half of their homework grades has no effect on their final grade. If they've done enough homework to ensure a good grade, why not stop doing homework altogether?
In our experience, most students do not exploit this particular loophole. However, we did have occasion to teach one class that gleefully and openly did so. We wondered whether the
students who skipped homework had, on average, different final grades than those who did all the homework. While we were thinking about how to answer that question, we decided that it might be interesting to see what the answer would be if we used one of two alternative grading schemes:
Using the average instead of the median, and treating those assignments that the student failed to turn in as zero
Using the median of only the assignments that the student actually submitted
For each of these grading schemes, we wanted to compare the median grade of the students who turned in all their homework with the median grade of the students who missed one or more assignments. We wound up with a program that had to solve two distinct subproblems:
Read all the student records, separating the students who did all the homework from the others.
1.
Apply each of the grading schemes to all the students in each group, and report the median grade of each group.
2.