5.10 TEMPORAL ORDERING
5.10.2 T HE B ODY
There are several ways in which the body of a literature review may be presented. The job here is to decide which approach offers the clearest presentation of the material at hand. One common problem with writing a literature review is that when the author has read a really good literature review it becomes difficult to present the same material (with some additions) in a different format.
Unfortunately, it can be very difficult to write in a similar presentation style as another author without lapsing into rewriting the paper with synonyms in the actual text. This is not
acceptable and we strongly advise that if the material being reviewed is very similar to the work of an existing author, take the time to reframe it completely. Although this is more work, it removes any suspicion about ‘‘borrowed’’ work. Besides, if the material has already been presented in a given temporalflow, it may make the material accessible to a different audience by presenting it in a different fashion.
5.10.3 ARRANGING THEBODY
As discussed earlier, there are many logical ways to order a literature review. We previously discussed the differences between two types of approaches—an intellectual history and the history of ideas. However, there are additional approaches that one can consider. Among them are chronological, by topic, and methodological approaches (The Writing Center, 2006).
5.10.3.1 Chronological
A chronological approach tracks events through time. This approach is good if you want to trace the evolution of an event or theory, but may become confusing if there is a lot of information about a subject over a short period. For example, if an author wanted to trace the evolution of public administration, they could start with Wilson (1887) andfinish with Frederickson and Smith (2003), hitting such greats as White (1926), Gulick (1937), Stillman (1987), and Kettl and Milward (1996) along the way.
5.10.3.2 Topic
Often, telling a tale as it happens is not the best way to emphasize specific events. The same is true with writing. Often it is more helpful to discuss the advent of a given phenomenon in terms of the components that drove it. For instance, Shafritz and Hyde (1997) offer the contents of their reader in both chronological and topical orders. The latter allows the reader to only read the sections dealing with human resource or organization theory. By seeing the development of the parts, it is easier to understand the development of the whole.
5.10.3.3 Methodological Approaches
While the first two approaches are more common, the constant expansion of methodological approaches is becoming ever more confusing in thefield of public administration. If there are several sets of methodological approaches, it is often to group the literature by these approaches so the reader can compare apples to apples and so the author does not have to be redundant in the explanations of a technology. It is also common to blend these approaches to better tailor the discussion to the question at hand. Above all, there are a few points that should be universal in writing.
5.10.4 UNIVERSALS
5.10.4.1 Synthesize
Always work to be succinct in writing. It is the writer’s job to synthesize the material, make sense of it, and make it one’s own. There is nothing more valuable than a good literature review and little as punishing as a bad one. Unfortunately there is not much middle ground between the two. Think back to a young child trying to tell a story about a series of events that hurt its feelings.‘‘Bobby said this, and Suzie said this and Frank said this, and Krystle said this, and Maggie said this . . .’’It is truly hard to be compassionate when a story becomes this annoying. Far better would be something like, ‘‘Bobby, Frank, Maggie, and Krystle all agreed that . . . While Suzie and I think this . . .’’; parsimonious, efficient, and effective (although it is difficult to get adults to speak so clearly at times, this is probably an impossibility for an emotional youth).
5.10.4.2 Keep Your Own Persona
There is nothing as bad as seeing a bad impersonator doing an impression of someone you like. By the same measure, it is terrible to read a paper that is trying to sound like someone they are not. This is probably one of the biggest shortcomings in writing today. Far too many people believe that they must write like the work they read, when in fact this is a horrible approach. Remember, the reader wants your personality andflavor. Some of the best scholars in thefield are not great because they are great thinkers, but because they are great communicators. Emulate that quality instead of trying to impersonate someone else.
5.10.4.3 Provide Structure
Very few people can write so well that they do not need to impose structure in the writing. Structure guides your thought while you are creating and, more importantly, it helps the reader through the work. It also allows the reader to jump back to previous sections, provides natural stopping points, and gives an overview of the work. Earlier we advocated the use of outlines in your writing. We also suggest using the outline to create section headings. The really fun part of this is that using the headings feature in your word processor allows the author to jump around within the outline and make automatic tables and indexes.
5.10.4.4 Give Credit
Someone once said that if you copy from one it is plagiarism, but if you copy from many it is research. Although that is not exactly true, there is some merit to the saying. The most important thing to do is make sure everything used is cited. We cannot think of a reason to omit citations (with the possible exception that the author did not track them properly and has lost the reference). Err to the side of citing too much, and if in doubt, seek advice (you only have to cite advice if you are going to publish the paper).
5.10.4.5 Revise
Never expect that afirst draft of anything is good enough. Make it a habit of writing papers well before due dates and take a break from the paper. Set it aside for a day or two so as to completely clear the mind, and then go back to it. The more fresh looks one can take at a paper, the better the final product will be.
5.10.4.6 Share the Pain!
Before sending anything anywhere for evaluation (be it a conference, class, or to a journal), one should at a minimum read the paper aloud. If the author is unable to do this, then the paper probably does notflow very well. An even better approach to evaluating writing is to have a friend read the paper aloud. This will let the author hear how the prose flows and allow them to identify where others will stumble on the writing. As well as improving your paper, this will create a way to gauge writing proficiency over time.
REFERENCES
Fink, A. 2005.Conducting Research Literature Reviews. Thousand Oaks: Sage.
Frederickson, G.H. and Smith, K.B. (Eds.). 2003. The Public Administration Theory Primer. Cambridge:
Westview Press.
Gulick, L. 1937. Notes on the theory of organization. In L. Gulick and L. Urwick (Eds.),Papers on the Science of Administration, pp. 1–89. New York: Institute of Public Administration.
Kettl, D.F. and Milward, H.B. (Eds.). 1996. The State of Public Management. Baltimore: Johns Hopkins University Press.
Lovejoy, A.O. 1936. The great chain of being; a study of the history of an idea. Cambridge, Mass: Harvard University Press.
Shafritz, J.M. and Hyde, A.C. (Eds.). 1997.Classics of Public Administration. Fort Worth: Harcourt Brace.
Spicer, M. 2004. Public administration, the history of ideas, and the reinventing government movement.Public Administration Review64(3), 353–362.
Stillman, R.J. 1987.The American Bureaucracy. Chicago: Nelson-Hall.
The Writing Center, U. o. N. C. a. C. H. 2006. Literature Reviews.
Van Wart, M. 2003. Public sector leadership theory: An assessment. Public Administration Review 63(2), 214–228.
White, L.D. 1926.Introduction to the Study of Public Administration. Saddle River: Prentice Hall.
Wilson, W. 1887. The study of administration.Political Science Quarterly 2(June), 197–222.
Part II
Research Design
6 Purpose and Signi fi cance of Research Design
Jonathan B. Justice
CONTENTS
6.1 Introduction ... 75 6.2 Purpose and Significance of Research Design ... 77 6.3 Elements of Research Design ... 79 6.3.1 Lurking below the Surface: Causality and Variables... 81 6.3.2 Getting to the Point: Topics, Problems, Questions, Arguments, and Answers ... 82
6.3.2.1 Getting into the Conversation: Theories, Frameworks,
and Previous Research... 83 6.3.2.2 Contributing to the Conversation: Research Problems and Questions ... 84 6.3.2.3 Designing a Strong Argument... 86 6.3.2.4 Research Ethics and Writing ... 88 6.4 Conclusion ... 89 Exercises ... 90 Endnotes... 90 References ... 91 Bibliography... 92 6.1 INTRODUCTION
The phrase ‘‘research design’’ denotes both a process and a product aimed at facilitating the construction of soundarguments. An argument is a logical structure that marshalls both evidence and reasons why that evidence supports some claim or point. A sound argument is one that supports its claim in a way which is, even in the eyes of a skeptical and well-informed audience, credible and useful to the greatest degree feasible given the resources we have available for gathering and analyzing evidence in support of that claim. Research involves using a transparent and systematic approach to conducting inquiries that answer questions and solve problems by means of claims that are supported well enough to be treated reasonably as knowledge rather than mere assertion. The process of designing research can be difficult and at times frustrating—messy, seemingly inefficient, nonlinear, and even repetitive. If we invest the time and effort to do research design well, however, the resulting plan for conducting research will allow us to be more orderly, efficient, and linear in amassing and interpreting evidence and using it to construct a sound argument. If we design research poorly or not at all, we may get lucky and be able to generate a valid argument anyway, or we may be unlucky and end up having conducted an investigation that fails to support a claim adequately. Well-designed research leaves less to chance and thereby reduces the risk of wasting time and effort on pointless research.
The purposeof research design, then, is to define the structure of an inquiry into a research problem that will produce a persuasive, valid, and demonstrably useful argument in the eyes of the researcher’s audience, yet can feasibly be carried out within the bounds of the material and 75
intellectual resources, and time, available. Like any kind of design, research design is a form of constrained problem solving, used to prescribe a way to use available materials to yield some desired product or outcome. For an engineer, the problem may be to design a bridge that can carry particular vehicles over a particular river at a particular height, and which also can be constructed within specified limits on cost and schedule. The bridge once constructed will solve the problem of how to get those vehicles over that river. For the researcher, the problem may be to design a research project that will provide the best possible—for a defined audience and application—answer to a question, or the best possible confirmation or disconfirmation of a claim, within resource limits. In public administration, the underlying problem may be a practical one confronting administrators or policy makers (e.g., whether to privatize trash collection) or a theoretical one confronting researchers (e.g., whether‘‘rational choice’’theorists’assumptions about human nature are correct).
The significance of research design is thus twofold. First, the quality of the design will be a determinant of the quality of the product. It is usually easier to produce a good bridge or a demonstrably sound argument if you start with a carefully thought-out design. Second, by narrow- ing inquiry to a specific combination of particular theories, frameworks, research question(s), evidence, methods of drawing inferences from evidence, and audience, the design determines what arguments can and cannot convincingly be made, what uses can and cannot reasonably be made of the researchfindings, and by whom. (Will the bridge be designed to carry pedestrians, trains, and bicycles, or only cars, trucks, and buses?) The process of design is a process of trading off competing values and purposes: perfection against feasibility, breadth against depth, accessibil- ity to an audience of practicing managers versus some of the qualities sought by academic social scientists, parsimony versus realistic description, statistical (Yin, 2003, p. 10) generalizability versus fine-grained accuracy and relevance to a particular case, and so on.
In this chapter, I will emphasize the problem-solving aspects of research design in public administration (PA), as a somewhat iterative and often messy process which is used to solve the second-order problem of how to solve research problems by delineating credible and feasible procedures for investigation and argumentation in both the practice and study of PA. The nature of PA as both afield of scholarly inquiry and afield of professional practice dictates the use of a diversity of paradigms, designs, and methods of research according to the specific purposes and audiences at hand. Thus we should not rush to privilege one form or source of knowledge over another (Schmidt, 1993). The implication of this for the design and conduct of research in PA, however, is not that fundamentally different standards of reasoning and argumentation apply in different applications, nor that scholars need somehow to adopt less rigorous standards and procedures or are justified in so doing. Rather, it means that we should seek as researchers to employ, and as consumers of research to demand, logic, argumentation, and evidence that reflect the standards of validity and reliability, susceptibility to scrutiny, and willingness to yield to demon- strably better supported claims that we tend to associate with‘‘scientific’’research.
This should not be intimidating for students or practicing managers, however: the systematic approach and publicness of data and methods associated with‘‘scientific’’rigor are meant precisely to make it easier for us mortals to maximize the quality of our arguments. Applying scientific rigor to research requires using a tested set of procedural guidelines to channel as well as to stimulate creativity. Accordingly, a good, scientific (meaning systematic, explicit, and transparently pre- sented) research design can make the construction of useful1 knowledge in (and of ) public administration easier than it would otherwise have been, for managers and students as well as for specialist researchers.
My primary agenda in this chapter, therefore, is to present the fundamental logic of research design for public administration as involving simply the work of devising a sound plan for constructing a good argument to support a claim that will be useful to some audience or audiences.
I will try here to demystify some of the social-scientific jargon usually employed in courses in research design and methods by showing how it is related to the more general problem of devising adequate arguments to support claims presented to skeptical audiences. Audiences of administrators
and audiences of academics may well have different standards of evidence and inference but these are differences in technique rather than in the underlying logic of inquiry. Further, I would suggest that administrative researchers and administrative consumers of research would do well to design and evaluate administrative arguments with the same degree of concern for making their assump- tions and limitations explicit and their logics of inquiry systematic that academic researchers (are supposed to) employ.
Courses in research design, like the chapters in this volume, discuss scientific procedures in terms of such qualities as validity, reliability, inference, induction, deduction, levels of measure- ment, operations, instrumentation, and so on. In practice, this language can be understood as concerned with the fundamental structural elements of an argument: claims, evidence (or data), inferential warrants, and qualifications (see Booth et al, 1995, pp. 85–148, for an updated discussion of this general structure grounded in a classic essay by Toulmin [1958, pp. 94–145]).Claimsare the points we want to make: our proposed solutions to problems; answers to questions; assertions about what our audience should be paying attention to, or choosing, or doing; or simple descriptive statements (e.g., Toulmin’s ‘‘Harry is a British subject’’). Evidence is the observable data on which claims are ultimately based (‘‘since Harry was born in Bermuda’’). Inferentialwarrantsare the accepted or acceptable logical principles that allow us to justify particular claims on the basis of particular evidence: generally recognized or defensible theories and principles that allow us to draw conclusions on the basis of observations (‘‘because those born in Bermuda are British subjects’’). Warrants in turn may be supported by implicit or explicit backing that justifies the warrant itself (‘‘on account of the particular legal provisions that make those born in Bermuda British’’). Qualifications describe limits to the force or breadth of application of our claims: the evidence and warrants I am able to muster in support of my claim may not be conclusive, there may be individual exceptions to a general rule, or a rule may apply only to a degree or only under certain conditions or in certain settings (‘‘presumably,’’ ‘‘unless he has become a naturalized American’’).
Researchers will also try to anticipate any reasonable rebuttalsthat a skeptical and well-informed audience might offer (that Harry wasn’t really born in Bermuda, that my reading of the law is incorrect, that Harry has in fact become an American, and so on). What distinguishes research-based arguments and claims from those based on faith or on mere assertion is the researcher’s explicit attention to grounding them by means of adequate and soundly backed warrants in appropriate evidence, to anticipating and responding to reasonable rebuttals, and to honestly acknowledging their limitations.
Another distinguishing characteristic of research arguments is that they are formally structured as inquiries rather than solely as means to justify already decided-upon claims. Thus, the formal jumping-off point for a research-based argument will be a central research question. When your research argument is constructed in itsfinal form, therefore, it must have a question as well as an answer (your claim), and answering the question must lead demonstrably to the key points and conclusion of your argument. Procedurally, however, you may begin either with a pure question, andfind your way to an argument in the course of answering that question, or with a point you want or need to make.2In either case, answering the research question should solve, or at least elucidate, a problem that matters to your audience, and a research design provides the road map for an argument that will arrive at that solution. The purpose of research design is to devise procedures for constructing an argument, adapted to a particular audience for and problem of research, given the applicable constraints. The significance of research design is that it determines whether, by whom, and for what purposes your research results will be valued.