reluctant to rely on computers for composing and communicating their texts often end up wondering how they could have ever coped without incorporating these machines into their lives. They have effectively adapted their preferences to the new world they inhabit.
Indeed, many activities that currently require direct contact with other people will probably come to be “virtualized” through human–computer interaction. A remarkably prescient analysis of this prospect was presented a half-century ago by the theologian- turned-sociologist Jacques Ellul (1965), who has been often cited as an inspiration to the countercultural movements of the 1960s.
However, as we shall now see, the 1960s also marked the dawn of virtualization.
developments have not added up to a harmonious social order sup- ported by a benevolent technocracy.
To be sure, there are those who regarded Bell’s dream as a night- mare from the start. Such anxieties were largely responsible for the widespread campus unrest in the late 1960s, a point Bell (1966) himself admitted in the report he did for Columbia University, The Reforming of General Education. Perhaps because many of these critics were trained in the humanities or the “softer” social sciences, there has been a tendency to caricature their objections as anti- scientific and Luddite. However, a perusal of a major manifesto of the period, Theodore Roszak’s (1967) The Dissenting Academy, reveals a more nuanced concern. Roszak and his comrades feared that the post-industrial mentality sacrificed the critical mission of science—associated with the Enlightenment motto “Dare to know!”
—in the name of increased productivity, which, in turn, strengthened the grip of the emerging “military–industrial complex” on American society. Technocracy for them was not science applied but science betrayed.
The disagreement here was more over interpretation than facts.
The prophets of post-industrial society did not hide the fact that the university figured so prominently in their vision because embodied information—both human and machine—was quickly becoming the leading factor of production in the economy. The dispute between the prophets and their critics turned on the quality of the products—
again, both human and machine—that this new economy was gen- erating. In particular, the dissenters were worried about the kind of people that such an economy produced.
Although artificial intelligence (or AI) research was still very much in its infancy in the 1960s, it was already clear that even relatively unintelligent computerized systems could perform well in environ- ments that simulated the complex problems facing a harried bureau- crat, manager, or field commander who cannot wait for perfect information before deciding on a course of action. The advertised virtue of these machines was, in large measure, a function of their very stupidity. Precisely because these machines could not capture the full range of factors that would influence a human decision-maker, they ended up saving time and resources by avoiding potentially ir- relevant lines of thought.
The obvious but politically abrasive point in all this was that these machines could not effectively operate without a hospitable human
environment. In other words, the attending humans had to be just as focused on the goals pursued by the computers as the computers themselves were. Any critical interrogation or creative reconfigura- tion of the software would defeat the purpose of “informatization.”
However, the prophets of post-industrial society realized that old human habits die hard, and so new forms of research and teaching entered academic life that soon made “simulation” seem like second nature. Business schools were in the vanguard of these develop- ments, with Herbert Simon at Carnegie Mellon University playing a leading role.
In the past 30 years, computer simulations have been increasingly used as research tools in the natural and social sciences. Indeed, the ease with which scientists today confer on simulations the title of
“experiments” speaks volumes about how “natural” the artificial has become. Simulations nowadays often replace costly laboratory exper- iments, which a couple of generations earlier would have themselves been regarded as lacking naturalness or “external validity.” However, dissenters from the post-industrial dream were originally more con- cerned about the role of simulations in the classroom. Here I mean not the actual use of computers, but the attempt to get teachers and students to think more like computers.
These simulations often assumed such humanistic guises as
“games” and “role-playing,” but were in fact no different from what military and business strategists had begun to call “scenarios.” The players of these games occupied positions in an artificially con- structed situation calling for a decision. However, the abstract setting sufficiently resembled the “real world” to be of some pedagogical value. Each position came built with assumptions that constrained the player’s deliberations. Adeptness at play came from making the best decision possible operating exclusively within those constraints.
In fact, players would be faulted for “breaking frame” and intro- ducing “extraneous information” that nevertheless would be perti- nent in the real-world setting. Sometimes these faults would be diagnosed as “errors in formal reasoning,” if not signs of outright
“irrationality.”
Regularity and reproducibility—the virtues that Bell had identified in the new intellectual technologies—appeared, in the eyes of his critics, as the latest trained incapacities of the learned classes. Class- room simulations favored those whose conception of accountability was limited to their assigned frame of reference. The implicit message
of these exercises was that a complex decision should be taken at some distance removed from the issues—and perhaps even the very parties—potentially affected by the decision. So, whereas the fuzzy- minded would deal directly with the concrete lives at risk when decid- ing how to trade off productivity against safety in the workplace, the rigorous simulator would start by translating these people into more abstract units of analysis. From under the veil of the algorithm, then, post-industrial reasoners were to be insulated from the remote con- sequences of their decisions.
The discontent generated by the new intellectual technologies in the 1960s was epitomized in a word: alienation. However, this word was used in so many different ways that the post-industrial prophets and their critics ended up arguing at cross-purposes. The critics com- plained mainly about how the computerized environment disabled the skills classically associated with participatory democracy, espe- cially critical deliberation in a public setting. Their fear was that, given the “proper training,” people would come to lose any interest in collectively questioning the ends that their work serves. It would be enough for them to do the work as efficiently as possible. Bell and his defenders interpreted these complaints to be more about the char- acter of post-industrial work itself—whether it retained enough
“craft” elements to give aesthetic satisfaction to the people doing it.
At the most basic level, “alienation” refers to the process by which people come to lose what they regard as most human. But, of course, there are as many senses of “being human” as there are schools of philosophy. Addressing the alienation of Aristotle’s zoon politikon—
the concern of the dissenting academicians—is quite different from addressing the alienation of Marx’s homo faber—the concern of the post-industrial prophets. However, in the confused heat of the debate, the latter concern came to dominate the former. In large part, this shift reflected a widespread belief that the social order projected by the post-industrial prophets was, indeed, an inevitability, and that the best one could do was to adapt one’s sense of intellectual integrity to it. The call to craftsmanship was one such adaptation.
Bell’s inspiration for introducing craft elements into technocratic work was the attractive picture of “normal science” presented by Thomas Kuhn (1970) in his blockbuster book The Structure of Sci- entific Revolutions. Instead of making all scientists look like such
“countercultural” figures as Galileo and Darwin, Kuhn portrayed the average scientist as focused on technical puzzles that make sense only
in the context of the “paradigm” under which she is laboring. As for
“scientific revolutions,” Kuhn said that scientists studiously avoid them until too many insoluble problems arise. But even then, a rev- olution should end as quickly as possible so that normal science can resume, now in a field that may be subdivided into two peacefully coexisting specialties. Despite restricting his own account to the natural sciences, Kuhn’s tendency to identify a paradigm with a self- selecting, self-governing community of inquirers helped persuade many humanists and social scientists that they, too, could enjoy sci- entific status—or at least raise their standing in the university—by doing things that look like normal science (Fuller 2000b, Chapter 5).
Thus, by the 1970s, the rhetoric of the dissenting academy had subtly shifted from claims that the sciences were allowing the military–industrial complex to colonize the university to claims that one’s own field had the right to pursue its paradigm alongside the sciences.
The rhetoric of “If you can’t beat ’em, join ’em” worked well in this time of plenty for the universities, when ideological enmities could be resolved by the creation of separate departments. However, the fiscal contractions of the post–Cold War period have forced some- what different conclusions to be drawn from the normal science mode of inquiry. A paradigm in the arts and sciences not only marks its practitioners as disciplined and its practices as self-contained, but also increases the likelihood that both will be dispensable from the standpoint of general education and the public mission of the uni- versity. In the case of the professional schools, the conversion of their expertises to normal science renders them tractable to the class of intellectual technologies—expert systems—that similarly threaten the employment prospects of therapists, medical technicians, engineers, financial planners, legal advisors, and many more.
In Herbert Simon’s (1981) terms, the post-industrial prophets managed to persuade academics to think about their work as con- stituting a “near-decomposable” system. A paradigm is essentially a module of knowledge production whose business can be conducted largely in isolation from other bodies of knowledge. The source of its autonomy—a limited domain and well-defined procedural rules of inquiry—is also the source of its replaceability, either by a successor paradigm, in the case of revolutionary science, or by an expert system, in the case of normal science. In both cases, however, the rest of the knowledge system can proceed as if nothing has happened.
In hindsight, the great irony of the debate between the post- industrial prophets and their critics was that both sides presumed that the worst that could happen to knowledge work was that it would become machinelike, and knowledge workers would suffer the intellectual equivalent of “deskilling.” No one originally suspected that knowledge workers could themselves be replaced by machines.
In other words, neither Bell nor Roszak took seriously enough the fact that intellectual technologies are primarily technologies, and hence likely to have the same effects on academic capital and labor as other technologies have had on other socioeconomic factors. Thus, from the standpoint of academics worrying about their job prospects, a Luddite response to the introduction of computerization in the 1960s would have been remarkably prescient.
However, the Luddite response was not forthcoming because aca- demics allowed metaphysics and epistemology to do the work of eco- nomics and politics. Epitomizing this tendency is the philosophical debate over “the very idea of artificial intelligence,” which has fas- cinated practitioners in many fields (Haugeland 1984). Indeed, AI research entered the world already accompanied by skeptics who have tried to come up with “tests” which only human intelligence can pass, not its machine simulations. Championing these skeptics for the past 30 years has been the Berkeley phenomenologist Hubert Dreyfus (1992), whose 1972 book, What Computers Can’t Do, pop- ularized the idea that computers cannot think unless they can repro- ducehuman thought processes. Yet, from the 1950s onward, military, industrial, and ultimately academic interest in AI research has been motivated by the idea that machines can replace human thought processes in a variety of situations calling for quick and focused deci- sions. The advent of the automated workplace had demonstrated that machines could replace humans without necessarily reproducing exactly what the humans had been doing. Indeed, the economic lure of automation is precisely the prospect that machines can avoid certain endemic features of human performance—from fatigue to feelings—that limit productivity.
From an engineering perspective, human beings are built to do many things passably well but few things exceptionally well. A Swiss army knife comes to mind as the model of Homo sapiensimplied in humanity’s collective technological trajectory. The history of tech- nology has been one long attempt to disentangle human capacities into separable skills with specific goals, each of which can then be
simulated and refined to order. AI research is only the latest development along this trajectory. From that standpoint, Dreyfus’s arguments against the possibility of AI sound a bit like the centuries- old complaint that humans will never be able to fly because we are physically incapable of doing what birds do. Such complaints are beside the point, once agreement is reached on which aspects of a bird’s flight are worth simulating. Not all of them are of equal inter- est. Similarly, an expert system designed to diagnose mental illness may lack the subtlety and intuition of an experienced therapist, but if the goal is to classify large numbers of patients accurately and effi- ciently, then a calculator of symptoms might do that particular job better than the caring person (Faust 1985). What, then, should we make of Dreyfus’s efforts at troubleshooting expert systems by asking the machines about the background conditions that make their exper- tise meaningful? Not surprisingly, machines fail to provide suitable answers, as they were not programmed to reflect upon the presup- positions of their claims to knowledge. The unspoken assumption here is that human experts would have done better. But would they?
With Bell and Kuhn as our guides, we can imagine that a human expert would refuse to answer Dreyfus’s queries on the grounds that they violate the public trust and collegial authority that make exper- tise sociologically possible. Expertise, in the sense that makes “expert systems” even conceivable, requires a strong sense of who is “inside”
and “outside” the body of expert knowledge. Colleagues question each other’s judgments only within the approved parameters of the
“peer-review” process. As lawyers and journalists have shown all too well, whenever experts agree not to limit the kinds of questions they answer, their expertise is soon eroded.
People seek out experts in the hope that the experts understand their problems better than they themselves do. Thus, a client is no more likely than a fellow expert to engage in acts of Dreyfusian impertinence. On the contrary, the client may well blame herself if expert advice fails. Perhaps she misunderstood what the expert had said or even misstated what her original problem was. Like the expert system, then, the human expert must control the frame of reference in terms of which questions are posed. Dreyfus comes across as much too eager to “break frame.”
Despite being unwilling to test it in their daily practice, most people seem to agree with Dreyfus that there is something special about human experts that computerized versions could never fully
replace. This is the “tacit dimension” of genuinely expert knowledge, theje ne sais quoithat supposedly distinguishes someone truly gifted in her field from one who is merely proficient at the rules. On this view, the plodding expert system has met its match in the inspired expert human. But how realistic is this distinction? And, to the extent that it does capture something real, how is it to be explained?
Experts flourish when they enjoy discretionary control over the domains in which they operate. Their ability to give good advice is matched only by their “meta-ability” to determine when they give advice at all. There are certain clients that smart therapists, lawyers, or accountants will not take—regardless of the promised remunera- tion—because, for whatever reason, the cases do not seem tractable to their expertise. Such rejections are considered signs of “wisdom”
on the part of the expert human, yet when an expert system rejects the solicitations of a client, it is counted as a strike against the system.
For this very reason, the first stage in designing an expert system is figuring out how a human expert decides whether a case is rele- vant to her expertise. This typically involves extended interviews, in which the expert system designer, or “knowledge engineer,” tries to map an expert’s mindscape. To be sure, the assignment is far from simple, since, in large part, the tacit dimension of the expert’s knowl- edge lies in her inability to articulate the principles that govern her practice. Thus, the knowledge engineer often needs some je ne sais quoi of her own—or at least methods that approach psychoanalysis in their indirectness—in order to retrieve expertise from its human container. This stage is followed by the construction of a computer interface that enables the expert system’s prospective user to access that expertise as effortlessly as possible, presumably by building on what the user already knows and her interests in wanting such a system designed in the first place.
The two remarkable features of this process—that experts would grant interviews to knowledge engineers and that expert systems would be customized to user specifications—have begun to transform our conceptions of knowledge in ways that go beyond the original post-industrial prophecies. These developments parallel ones that have increasingly allowed the conversion of knowledge from embo- died ideas to intellectual property. Together they challenge the as- sumption that knowledge is a “public good.” Whatever else may be meant by this expression, it implies that once the good has been pro- duced, no additional costs are incurred for additional people consum-
ing the good. For example, an insight that originally took the genius of an Einstein or an Edison is eventually accessible to anyone who can read a physics textbook. If students today need to reproduce Ein- stein’s original reasoning, they are probably proving a point about the psychology of scientists, not “earning” their knowledge of relativity theory. This idea plays to the classical notion that genuine knowledge is, at least in principle, equally accessible to all. In economic terms, the value of knowledge is not affected by its distribution.
However, the advent of expert systems challenges precisely this notion. In a slogan: The multiplication of expert systems divides the value of expertise. Like all slogans, it covers a complex of effects.
The spread of expert systems in a given domain is likely to diminish not only the need for the corresponding human experts but also the value of the knowledge that each remaining expert has. However, these effects will have little to do with the ability of the expert systems to reproduce human expertise. Rather, they will result from people coming to realize that it is not worth paying the additional cost of consulting the human expert. The customized computer is likely to address the client’s needs at minimal cost. In that case, professional consultations may go the way of hand-crafted furniture, the pursuit of a few artisans who eke out a modest living off the patronage of the rich.
Nevertheless, it would be a mistake to suppose that only the human expert experiences a depreciation in the value of her exper- tise. As more people have access to an expert system, the value which they can derive from its knowledge base will likewise decline. This is for two reasons. On the one hand, many people will be in a posi- tion to act on the basis of the same knowledge at roughly the same time, thereby diminishing whatever advantage that knowledge may have brought for its possessor in the past. On the other hand, the amount of knowledge that will be available to a given individual from the various expert systems at her disposal will make it increasingly difficult to locate what one needs to know in order to act decisively.
Public ignorance thus ascends to the meta-level of not knowing the expert system in which the relevant expertise is contained.
If these emerging problems of knowledge seem strange, it is because they reveal that knowledge is less a public than a positional good (Hirsch 1977). In other words, the value of a piece of knowl- edge cannot be determined without knowing how many people have access to it, and how many other pieces of knowledge they have