There exist several important steps in writing a literature review. Given that what is to be done is to review literature, the first and most important step is finding that important literature. Currently, electronic access is probably the source for the majority of literature cited in student research today.
There are several different types of electronic access, and each offers different strengths and weaknesses.
5.8.1 GATHERSOURCEMATERIAL
5.8.1.1 Electronic Resources
First-line searches are typically done with either a search engine or an index. The search engine actually goes out and searches the World Wide Web for pages that reference the specified search criteria. Oftentimes, the types of searches can lead to a long and fruitless path. Bear in mind, that while the Internet offers a tremendous amount of information, there is nofilter for this information.
By this we mean that there is no check on the veracity of what is there. While some very bright, informed individuals poach research on the World Wide Web, it is also home to a great deal of‘‘less than accurate’’information. One notable exception is Google Scholar.
Indices, on the other hand, are searchable databases from known sources. Typically these sources are from refereed journals, official documents, or bona fide news outlets. JSTOR, one of
the oldest scholarly journal indices, offers a list of the journals included. Other indices do one better.
Blackwell Synergy offers not only full text access to all Blackwell journals but it also offers hyperlinks for the citations. The major downside to these types of indices is that they give the illusion of total coverage. What we mean is that a typical search on any of the commercially available indices will usually result in tens of pages of article listings. However, important literature, especially that which predates the electronic era, often can easily be missed. Also, topics that are reserved for specialty journals typically do not fare as well when searches are limited to electronic indices. Some popular indices include
Blackwell Synergy: http:==www.blackwell-synergy.com= ERIC: http:==www.eric.ed.gov=ERICWebPortal=Home.portal First Search: http:==firstsearch.oclc.org=
JSTOR: http:==www.jstor.org=
Lexis=Nexis: http:==global.lexisnexis.com=us 5.8.1.2 Old School Library Usage
Before 1990, it was easy to tell when class papers were due at major universities. That would be the day that college professors and students fought for space at the large tables in the library. The advent of electronic access has all but done away with this seasonal trend. This is unfortunate, as some of the best access to information is still only available from a trip to the library. Often, one will be able tofind information from state and local governments, which is unavailable online in the library.
Older information, including government documents and reports, is often stored as microfiche which must be accessed in the library.
Unique information, not available online, is certainly a good reason to visit the library; however, it is not the best. The most important resource in the library offers its patrons are librarians. Ever since manfirst started storing knowledge, we have continually upgraded the storage, medium, and delivery of knowledge. For this reason, librarians always know the best way to locate information.
Furthermore, they will often steer you to information you had not originally thought of. Librarians are masters of far more than the Dewey decimal system, and every researcher would be well served to be able to call at least one friend.
5.8.1.3 Find a Friend!
Ever noticed that senior professors seem to be far more prolific in their writing than the junior professors? There are several reasons for this:first and most obvious is that they have been doing it for a long time, and have read a fair amount of the literature extant. Second and somewhat less obvious is that there are just not that many academics in any specific area. This means that more senior professors likely know everyone who is active in research within their discipline. We should be clear here that when we say know we do not mean know of, but truly know. Through networking, conferences, and social events, most active scholars know everyone who publishes in their area. The long and the short of this is it is easier to remember what your friends are doing than someone you have never met. The third andfinal reason is that scholars tend to focus all their research in a very few areas. This means the longer you focus on a given literature, the better you get to know the literature. We are certainly not suggesting that senior professors should provide every reference needed to read on any given topic area, but any professor should be able to steer an interested party to the body of literature on a particular topic within their individual expertise.
It is not uncommon for researchers to consult one another about existing bodies of literature;
however, a person does not have to be a professor to be able to suggest a place to start. As mentioned earlier, librarians are great for this. Another good source can be classmates or friends who do work in the samefield. Remember, just because two people had the same classes does not necessarily mean they retain the same information.
5.8.1.4 Quality of the Literature
A good literature review does not just sum up everything that exists on a given topic. A good literature review summarizes and synthesizes all the literature that is pertinent to a given question.
With this in mind, it follows that one must be able to identify good research from bad. In some cases this is more difficult than it sounds. Everyone who has ever watched late-night TV has certainly seen an infomercial. The interesting thing about infomercials is that it is often difficult to distinguish them from actual TV news programs. Obviously, this is a marketing ploy. Research can be similar in that often very bad research gets‘‘packaged’’ in such a manner as to increase the face validity of the work. There are some rules of thumb, however, that can help to distinguish the good from the bad (and the ugly).
5.8.2 LITERATURE SOURCES
5.8.2.1 Journals
When we speak of journals, we mean peer-reviewed, scholarly journals. A peer-reviewed journal is one where the manuscripts are sent out for reviews before publication. These reviews are double- blind. This means that neither the authors nor the reviewers know who the other is. This is done in an effort to ensure the quality of the research. Due to the laws of supply and demand, the research found in better journals faces a more scrutinizing review process. Typically, journals can be thought of as a fairly reliable source of scholarly research. It is important to remember all journals are not created equal. There are two easy ways to distinguish between good journals and the rest.
Thefirst is by reputation. In thefield of public administration there are journals heard referenced over and over again. This will includePublic Administration Review,Journal of Public Adminis- tration Research and Theory, Journal of Policy Analysis and Management, Journal of Public Affairs Education, as well as others. One can have more confidence in the quality of research in these better-known journals than the research found in more obscure journals. One major reason for this is the demand for journal space. Each journal has afinite set of resources and can publish only as many articles every year. It follows then, that the more popular a journal becomes, the more manuscripts it receives every year. The larger the number of submissions received, the more selective a journal can be in what it chooses to publish.
The second way to distinguish between journals is to look up the ISI impact factor. These can be found on the ISI Web of Science Web site. The impact factor is a score which reflects how often manuscripts published in a journal are cited in other work. Although this is not a perfect measure, it does give a pretty good indication of the status of the particular journal with other researchers.
5.8.2.2 Books
Books are a good source of information, but they are typically more dated than journal articles. This is because it takes longer to write a book, and it takes longer to review a book. Bearing this in mind, one of the greatest strengths of a book is the coverage it can give to older subjects. Also, books can be a fantastic way tofind a starting point in unfamiliar literature. There are also quality differences in books that can be identified by the type of publisher. The basic rule of thumb is that university presses are more academically rigorous than commercial presses. Also, some commercial presses are better than others. There are a number of good presses, including Lexington, McGraw-Hill, Wiley and Sons, and others that you will see repeatedly as classroom texts. Typically, these produce better scholarship than‘‘prestige’’presses.
5.8.2.3 Conference Proceedings
Many conferences now publish conference papers in either hard or electronic format. This is a good source of cutting-edge research, and often a good way to catch manuscripts before they go to
journals. The easiest way to access conference proceedings is to go to the association Web site. The downside of conference papers is that they have not been through a peer review.
5.8.2.4 Dissertations
Dissertations are a good source of information on a given topic. Typically a dissertation will have a very thorough review of the literature on any given topic. Also, dissertations go through a rigorous reviewing process by the dissertation committee. Unfortunately all committees are not created equal, and it is important to see who the committee was. This information is usually contained within the bound dissertation.
5.8.2.5 Government Research
There is a great deal of government produced or sponsored research that never makes it to a refereed journal but is performed by world-class researchers. This is because the government is typically more interested in outputs and outcomes than the theoretical links, whereas scholarly research is done with the intent to understand theoretical linkages. Much of this is easily available on the Web.
For instance, the Department of Education Web site has links to tremendous amounts of research on US schools. The same is true for almost every other large agency.
5.8.2.6 Internet Sources
Internet sources are a mixed bag at best. While there are some noteworthy research projects out there that will emerge from a general Web search (i.e., Yahoo, Google, Dogpile), the majority of what will result are either hits that are only tertiary to the given subject, or are dangerous. And by dangerous, we mean that they will be convincing enough that you might be inclined to take the site as a reliable source, only tofind out later that the information was wrong.
5.8.2.7 Think Tanks
Research produced by think tanks can be very valuable, as long as one understands the system that produced the product. Think tanks produce research for clients or are based on an ideology. Bear in mind that ideologically driven research starts with a set of assumptions that are typically not altogether solid. For example, there exists literature on teen pregnancy that is built on the assump- tion that life begins at conception. While this is a truth in some religious beliefs, this is a subjective truth that has not been verified by science. The premise of science is that facts that cannot be substantiated objectively are not facts. Hence, research that builds on literature derived from this assumption would be faulty from the start. This is not to say that science is‘‘better’’or‘‘worse’’than metaphysics, only that the fundamental requirements for the two are starkly different and are not interchangeable.