rates or which used data from non-representative samples of respondents and articles which employed correlation and/or regression analyses of variables from data sets that did not make good bedfellows.5
Conclusion
What can be said from this examination of the published scholarship in the nascent field of e-government research? Since we do not want to be criticized for “going a bridge too far”, that is, for going beyond what the data from the articles permit us to say, we will be cautious in drawing conclusions.
Perhaps the first conclusion is that, through the end of 2004 (that is, during argu- ably the first 12 years or so of e-government, and arguably eight years of published e-government research), relatively few (only 57) empirical articles about e-govern- ment have been published in peer-reviewed academic journals. What is more, only seven (13%) of those articles appeared in what are generally considered the lead- ing journals (e.g., Public Administration Review, Journal of Public Administration Research and Theory, and Administration and Society). All of the other articles appeared in lesser-ranked journals.
The Scholarly Lterature on E-Government
Second, the empirical literature on e-government in this period was distributed al- most equally between journals in the social sciences and journals in the information sciences. Within the subset of articles appearing in IT journals, however, more than half (14 out of 27) were published in a single journal, GIQ, which led in this category because of the number of special issues on e-government that it had published.
Third, most of the authors (as determined by lead author’s discipline) of these articles were from the social sciences (51%). This is at least somewhat surprising because funding for e-government research is more available to computer and information scientists than to social scientists.
Fourth, the most common types of empirical data found in the articles were survey data (42%) and Web site content analysis data (39%), each of which were used in 40% of the articles. Case study data (which were mostly from “case stories” and not rigorously-developed case studies) was used less often, appearing in 25% of the articles, while one article used a content analysis of state legislation. Four articles used a combination of survey and Web site analysis.
Fifth, 77% of the articles used some form of quantitative analysis, most frequently descriptive statistics, and often in conjunction with some form of correlation or regression analysis. A majority (57%) employed some form of qualitative analysis.
Twenty articles employed both qualitative and quantitative analyses.
Sixth, only about one-quarter of the articles (25%) formulated new theory, or ex- tended existing theory in a new way, or engaged in explicit testing of theory. Only slightly more (33%) formulated explicit hypotheses or research questions and tested them.
Seventh, of the authors who formulated or extended theory, nearly all (12 of 14, or 86%) formally tested the theory, while all who formed hypotheses or research questions engaged in formal testing. The scant number of articles that formulated theory, hypotheses, and research questions and that tested theory, hypotheses, and research questions suggests a lack of rigor in the published works in the emerging field of e-government research and also the relatively recent origin of the field.
Eighth, for the most part the authors of these articles did not heed Sir Isaac Newton’s advice to “stand on the shoulders of giants”. With exceptions as noted, the reviews of the e-government and the IT and government literatures were unimpressive. Here, we considered only 13 literature reviews in each category (23%) to be adequate (rated as either strong or moderate). The remaining articles (44, or 77%) received
“weak” ratings. The articles were somewhat better when it came to reviewing specialized literatures (e.g., bureaucracy or participation) that were also topics of the articles. Here, 28 (58%) were rated as adequate, while 42% received “weak”
ratings. (Because of their topics, nine articles did not need or contain reviews of specialized literatures.)
Finally, a number of the articles (23, or 41%) went a “bridge too far” and presented conclusions that were not supported by their data or analyses or whose data suf-
Norrs & Lloyd
fered from methodological deficiencies. In only about one-third (18 or 32%) of the articles were conclusions strongly supported by the data or evidence presented, and in a quarter (27%) of them, conclusions were moderately supported.
Our review of the small but growing body of literature in the field of e-government, from its beginning through the end of 2004, strongly suggests that this is, indeed, a new field of scholarship that it is just “getting its legs.” Relatively few articles on e-government had been published in peer-reviewed articles in this period, fewer still were empirical in nature, and yet fewer appeared in leading scholarly journals.
Indeed, the latter finding may help to explain the number of relatively weak articles that were published; they appeared in lesser-ranked journals.
Although the published empirical articles demonstrated a range of (mainly quan- titative) methodologies, few contained strong or even adequate links to previous scholarship in either the e-government or IT and government fields. Additionally, a surprising number of the articles drew conclusions that went beyond what their data or analyses would support.
As e-government itself continues to grow and evolve, what might we expect of the field of e-government research? We would both hope to see more and stronger (that is, more rigorous) articles, articles representing research from a wider range of disciplines, articles that employ a wider variety of research methodologies, and more articles that develop and test theory, hypotheses, and research questions. We would also hope that future articles would be linked more strongly to prior research (a.k.a., the relevant literatures), and that their conclusions would be more strongly supported by their data and analyses.
With the publication of four new journals devoted to e-government research begin- ning in 2004 and 2005, at the very least, the quantity of articles about e-government can be expected to increase. Beyond the issue of quantity, however, and, given the findings of this research, we can only wonder whether the field can sustain four journals or whether their existence will result in the continuing publication of re- search into e-government that lacks rigor and sophistication and is not adequately connected to prior relevant scholarship.
Whether our hopes and expectations for e-government research are fulfilled will require continuing examination of articles published in peer-reviewed journals—the gold standard of research in virtually any field.
References
Anderson, K. V., & Henriksen, H. Z. (2005). The first leg of e-government research: Domains and application areas, 1998-2003. International Journal of Electronic Government Research, 1(4).
The Scholarly Lterature on E-Government Grönlund, Å. (2004). State of the art in e-gov research—A survey. In R. Traunmuller, (Ed.),
EGOV 2004, LNCS 3183 (pp. 178-185). Berlin: Springer-Verlag.
Holden, S., Norris, D. F., & Fletcher, P. D. (2003). Electronic government at the local level.
Public Performance & Management Review, 26(4), 325-344.
Korosec, R. L., & Norris, D. F. (in press). E-government among Florida municipalities: A comparison to national data and trends. Baltimore: Maryland Institute for Policy Analysis and Research, University of Maryland, Baltimore County.
Norris, D. F., & Moon, M. J. (2005). Advancing e-government at the grassroots: Tortoise or hare? Public Administration Review, 65(1), 64-73.
Titah, R., & Barki, H. (2006). E-government adoption and acceptance: A literature review.
International Journal of Electronic Government Research, 2(3), 23-57.
Yin, R. K. (2004). Case study research design and method. Thousand Oaks, CA: Sage Publications.