ContentslistsavailableatScienceDirect
Management Accounting Research
j o ur n a l ho me p a g e :w w w . e l s e v i e r . c o m / l o c a te / m a r
Diversity and validity in positivist management accounting research—A longitudinal perspective over four decades
Maik Lachmann
a,∗, Irene Trapp
b, Rouven Trapp
baTechnicalUniversityofBerlin,DepartmentofAccountingandManagementControl,Straßedes17.Juni135,10623Berlin,Germany
bTUDortmundUniversity,DepartmentofAccountingandManagementControl,Vogelpothsweg87,44227Dortmund,Germany
a r t i c l e i n f o
Articlehistory:
Received27June2015
Receivedinrevisedform25May2016 Accepted17July2016
Availableonline25July2016 Keywords:
Methoddiversity
Positivistmanagementaccountingresearch Theorydiversity
Validity
a b s t r a c t
Thispaperassessesthedevelopmentandstateofpositivistmanagementaccountingresearch(PMAR).
Basedonacontentanalysisof375paperspublishedinnineaccountingjournalsoverfourdecades,we concludethatadiversesetofresearchmethodsandtheories,alongwithaconsiderationofvalidity,are necessaryprerequisitesfortheaccumulationofknowledgeonmanagementaccounting(MA)practice.
Inlightofdiversity,weexaminethestudieswithregardtotheircontents,methodsandtheoreticalper- spectives.Ouranalysesonvaliditycomprisemultiplefacetsofinternal,external,constructandstatistical conclusionvalidity.Regardingdiversity,ourfindingssuggestthatPMARhasrecentlybecomenarrowerin termsoftopicsasitincreasinglyfocusesoncontrolissues.However,PMARcontinuestorelyonavariety ofresearchmethodsandtheoreticalperspectives.Regardingvalidity,wefindimprovementsforallfour typesofvalidityovertime.However,potentialforfurtherprogresspersists.Wediscussourfindingsin lightofrecentdebatesregardingthestateofPMARandhighlightavenuesforfutureresearch.Overall, weconsiderourstudyusefulforassessingthediscipline’sachievementsandevaluatingitsfuturepaths.
©2016ElsevierLtd.Allrightsreserved.
1. Introduction
This paper analyses the diversity and validity in positivist management accounting research (PMAR) papers published in leadingaccountingjournalsoverfourdecades.Thestudyintendsto advanceourknowledgeonthestateanddevelopmentofthisfield ofresearch.Weexpectourpapertobeusefulforacriticalassess- mentofpastachievementsandforareflectiononfutureavenues.
Inparticular,recentlyexpressedconcerns regardingthepathof managementaccounting(MA)research(e.g.,Birnberg,2009;Chow, 2010;Merchant,2013;Salterio,2015)emphasizetherelevanceof theinsightsprovidedbyourstudy.
Ournotionofdiversityreferstotheemploymentofdifferent researchmethodsandtheoreticalperspectivesforinvestigationsof MApractices(Birnbergetal.,1990).Employingdifferenttheoretical perspectivesisimportantaseachtheoryreliesonspecificassump- tionsandthusexplainsMAphenomenaonlypartially(Hoqueetal., 2013;LuftandShields,2002).Similarly,relianceondifferentmeth- odsisimportantduetothelimitationsthateachmethodimplies (MerchantandVanderStede,2006;Shields,2015).Illuminatinga
∗Correspondingauthor.
E-mailaddresses:[email protected](M.Lachmann),
[email protected](I.Trapp),[email protected](R.Trapp).
MAphenomenonfromdifferentperspectivesthusappearsmore likelytocontributetoacomprehensiveunderstandingratherthan toremainentrenchedinone(Chapman,2012;DavilaandOyon, 2008;Hopwood,2008b;Merchantetal.,2003).Validityreferstothe approximatetruthofknowledgeclaimsgatheredthroughPMAR (Shadishetal.,2002,p.34).EvenifMApracticesareilluminated fromdifferentperspectivesbutwithconsiderableinherentflaws, diversityisunlikelytoadvanceourunderstandingintheintended manner. Therefore, we consider diversity and validity primary dimensionsforevaluatingPMAR.Consequently,thesedimensions serveasabackboneofthecontentanalysisofthe375PMARpapers presentedinthisstudy.
In light of recent debates onthestate of MAresearch (e.g., Mittendorf,2015;ScapensandBromwich,2010a),ourevaluation focusesparticularlyontemporaltrends.Insodoing,weplacethe currentstateonalargertemporal scaletoassessachievements anddrawbacks.Moreprecisely,weprovideacomparativeperspec- tiveonfourperiods.Ourtime framereflectsvariouswatershed momentsin thedevelopmentof thediscipline:Thefirstperiod (1980–1982)capturesthe“empiricalturn”ofMAfromaprimarily normativedisciplinethatreliedonanalyticalmodellingtowards empirical research (Hopper et al., 2001; Klemstine and Maher, 1984).Thesecondperiod(1990–1992)reflectstheestablishment of two academic MA journalspropagating openness to diverse researchapproachesandthuspotentiallyreinforcingdiversityin http://dx.doi.org/10.1016/j.mar.2016.07.002
1044-5005/©2016ElsevierLtd.Allrightsreserved.
PMAR(LindquistandSmith,2009;ScapensandBromwich,2001).
Thethirdperiod(2000–2002)representsatimethatisclaimedtobe characterizedbydiverseresearchapproaches(Scapens,2006).The fourthperiod(2010–2012)isexpectedtomirroranincreasingtopi- calandmethodologicalnarrowness(e.g.,Birnberg,2009;Hopwood, 2008a; Lukka, 2010; Merchant, 2013; Scapens and Bromwich, 2010b).
Our paper differs from previous studies primarily in two respects.First,weconsideraplethoraofresearchdesignchoices beyondtheclassificationsoftopics,theoriesandmethods.Thus,our studygoesconsiderablyfurtherthanpreviousbibliometricstud- ies(e.g.,Hesfordetal.,2007;LindquistandSmith,2009;Scapens andBromwich,2010b).Moreover,itcapturesalongertimeframe andalargerselectionofjournals.Second,ourstudycomplementsa seriesofpapersthatdiscussmethodologicalissues(e.g.,Ahrensand Chapman,2006;ChenhallandMoers,2007;LuftandShields,2014) byadoptingan“expost”perspectiveonthevalidityofpublished papers.1OurpaperthusreflectsthestateoftheartofPMARand mayraiseawarenessofissuesthatrequireconsiderationinfuture research(Brutusetal.,2013).Moreover,itprovidesanempirical contributiontothedebateonwhetherPMARislosingitsopenness todifferentresearchmethodsandrelevancetopractice(e.g.,Chow, 2010;Krishnan,2015;Merchant,2010;Salterio,2015;Tuckerand Lowe,2014).
Overall,ourfindingssuggestthatPMARisbecomingincreas- inglymature.Regardingdiversity,ourfindingsindicatethatthe rangeoftopicsstudiedbyPMARhasbecomenarrowerovertime.
However, PMARappears diversified, both in terms of research methods and theoretical perspectives. Thus, there is little evi- dencethatPMARfollowsthepathoffinancialaccountingresearch towardsmethodologicalandepistemologicalmonism(e.g.,Bonner etal.,2012;Oleretal.,2010).Regardingvalidity,ourfindingssug- gest thatalltypesof validitytend tobeincreasinglyaddressed overtime.However,thereisstillpotentialforsubstantialimprove- ment.Correspondingly,weidentifyaseriesoftopicalaswellas methodologicalissuesthatrequirefurtherconsideration.
Thispaperisstructuredasfollows.InSection2,wediscussdiver- sityandvalidityandderivepropositionsonthedevelopmentof PMARbasedontherelatedliterature.InSection3,weexplainour datacollectionandanalysis.WepresentourfindingsinSection4 anddiscusstheminSection5.
2. Backgroundandresearchpropositions
2.1. PurposeandprocessofPMAR
OurstudyreliesontheassumptionthatPMARintendstoacquire anin-depthunderstandingofMApractices(MalmiandGranlund, 2009;VanderStede,2015).PMARfocusesoncausalexplanations ofMAphenomenathatarecommoninmanyinstances.Therefore, itdrawsinferencesfromasampleofspecificobservationstothe general(Ittner,2014;Luftand Shields,2014).2 Modelsofscien- tificenquiry(e.g.,GioiaandPitre,1990;Kaplan,1986;Snowand Thomas,1994)suggestthattheaccumulationofknowledgetyp- icallybeginswithadescriptionofaMAphenomenonandtends tomoveincrementallyforwardtoitsexplanation(Kaplan,1986).
Forthisreason,researchcommonlydevelopsmeasuresthatreflect
1ExceptionsincludeBisbeetal.(2007),HartmannandMoers(1999),Modell (2005)andVanderStedeetal.(2005),whoreviewhowparticularmethodological issueshavebeenaddressedbypreviouslypublishedpapers.
2OurstudyexcludesinterpretiveMAresearchbecausethetypesofvaliditycon- sideredinourstudyaretypicallyonlyofconcerntopositivistresearchers(Birnberg etal.,1990).Bycontrast,interpretiveresearchersfocusonthe“trustworthiness”of theirresearch(AhrensandChapman,2006;DavilaandOyon,2008).
the phenomenon of interest and explores its associations with othervariables(SnowandThomas,1994).Asresearchprogresses, itculminatesatbestinthebuildingoftheoriesconsistingofpropo- sitionsthatexplainMAphenomenaacrossavarietyofconditions (ColoquittandZapata-Phelan,2007;MalmiandGranlund,2009).
CommentariesonthedevelopmentofMAresearchintheUSand UKsupportthisperspective.TheysuggestthatPMARwasmostly descriptiveafterthe“empiricalturn”intheearly1980sandgradu- allybecamemoreexplanatory(Maher,2001;Otley,2003;Scapens, 2006;Zimmerman,2001).Thisunderstandingoftheresearchpro- cessconstitutesthebackgroundforthefollowingpropositionsof howPMARhasevolvedwithregardtodiversityandvalidity.3
2.2. PropositionsonthedevelopmentofdiversityinPMAR
Knowledgebuildingmaybenefitfromtheemploymentofdif- ferentresearchmethodstoinvestigateaMAphenomenonbecause each issubject tostrengthsand limitations(e.g.,Merchant and Van der Stede, 2006; Shields,2015). We define theappropria- tionofdifferentresearchmethodsasmethoddiversityandexpect thatitincreasesinPMARovertime.First,thelimitationsofone methodmay serveas aninitialpoint for future studiesrelying onotherresearchdesigns.Forinstance,casestudiesallowforan in-depth exploration of MA innovationsin theirorganizational settings (Davila and Oyon, 2008; Merchant and Van der Stede, 2006).Althoughthesestudiesmayalsocontributetotheoryrefine- ment(Keating,1995;SnowandThomas,1994),providingevidence on thewider applicability of propositions requires larger-scale methods(Birnbergetal.,1990;Modell,2005).Surveystudies,for example,mayconfirmassociationsamonglargesamplesoffirms (LillisandMundy,2005;VanderStedeetal.,2005).However,due totheircross-sectionaldesign,mostcannotestablishcausalinfer- ences(Van derStede,2014).Employingdifferentmethods may offsetthelimitationsofindividualmethods.
Second,weexpectthattherangeofissuesstudiedbyPMAR,i.e., itscontentdiversity,increasesovertime.IfPMARintendstodevelop anin-depthunderstandingofMApractice,weassumethatdevel- opments inpracticeshape theresearchagendas(Baldvinsdottir et al., 2010; Mitchell, 2002). Therefore, the emergence of MA practices,suchasthebalancedscorecard(Kaplan,1994),strate- gicMA(Langfield-Smith,2008)orriskmanagementandcontrol (SoinandCollier,2013;VanderStede,2011),mayhavebroadened theagendaofPMAR.Astherespectivebodiesofknowledgedif- fer,weexpectthatdifferentMAphenomenaareinvestigatedby employingdiverseresearchmethods.Therefore,weanticipatethat methoddiversityincreasesovertime.Indeed,anecdotalandbiblio- metricevidencesuggeststhatPMARdiversifiedduringthe1980s and1990sregardingitsresearchmethods(Bhimani,2002;Hesford etal.,2007;HopperandBui,2016;Hopperetal.,2001;Scapens, 2006).
Knowledgebuildingmayalsobenefitfromrelianceonavari- etyoftheoreticalperspectives(e.g.,LuftandShields,2002;Lukka and Granlund,2002).For instance,Covaleski etal. (2003)com- pare budgeting research across the theoretical perspectives of economics,psychology,andsociology.Theyexplainthateachper- spectiveimpliesdistinctiveresearchquestionsandreferstospecific
3FollowingKaplan(1986),wedonotarguethattheresearchprocessmoves sequentiallyfromonestagetoanother.Inmostcases,itwillbeiterative,asresearch findingsatlaterstagesmayrequiretheoryrefinementandfurtherdescriptionofthe MAphenomenonunderstudy(FerreiraandMerchant,1992).However,weexpect thatPMARingeneralevolvesasitaccumulatesestablishedknowledgeinpartic- ularareasovertime.Weexpectthatstudiesintheseareasincreasinglyconsistof explanatoryresearch(seeforthesetopicsKrishnan,2015;Shields,2015;Vander Stede,2015),whereasemergingissuesarelikelytobesubjecttomoreexploratory research(MerchantandVanderStede,2006).
levelsofanalysis.Relianceononeperspectivethusprovidesonly partialexplanationsforthephenomenonunderstudy(Lukkaand Mouritsen,2002;Shields,1997; UnermanandChapman, 2014).
For this reason, illuminatinga MA phenomenon fromdifferent theoreticalperspectivesislikelytocontributetoamoreholistic understanding(LuftandShields,2002;Merchantetal.,2003).In thisstudy,werefertotheemploymentofdifferenttheoreticalper- spectivesastheorydiversity.WearguethattheorydiversityinPMAR increasesovertime.Onceknowledgeisestablishedaboutrelation- shipsinformedbyaparticulartheoreticalperspective,additional insightsmaybegeneratedbasedonilluminatingthephenomenon from other perspectives. Theory diversity appears particularly likelytoincreasewhenempiricalfindingsareinconsistentwith previouslyemployedperspectives.Commentariesonthedevelop- mentofMAresearchinAnglophonecountriessuggestthatduring itspenetrationintheearly1980s,PMARwasmostlybasedoneco- nomictheories(Hopperetal.,2001;KlemstineandMaher,1984).
Thisbasethenbecamemorediversifiedaspsychologicalandsoci- ologicaltheorieswereincreasinglyemployed(ChenhallandSmith, 2011;Maher,2001;Scapens,2006).Correspondingly,bibliometric studiesonMAresearchindicateanextensiverelianceoneconomic, sociologicalandpsychologicalperspectivesinthe1980sand1990s (Hesfordetal.,2007;HopperandBui,2016;Shields,1997).
Insummary,weexpectthatPMARreferstoanincreasingrange oftopicsandbecomesmorediversified withregardtomethods andtheoreticalperspectivesovertime.However,ourrelianceon modelsofscientificenquirydisregardsthataresearcher’schoice oftopics and researchapproaches doesnotdependentirelyon
“curiosity”but alsoonpersonalincentives, i.e.,tenureandpro- motioncriteria(GrayandMilne,2015;Hopwood,2008a;Shields, 1997).Thisissuehasbeenthesubjectofconsiderabledebatein recentyears(e.g.,Birnberg,2009;Chow,2010;Hermanson,2015;
Lukka,2010;Merchant,2010).CareeradvancementinUSbusiness schoolsissaidtobeincreasinglydependentonpublishinginasmall setofjournals(HumphreyandGendron,2015;Merchant,2013) andisprogressivelymimickedbyEuropeanuniversities(Hopwood, 2008a;Messner,2015;terBogtandScapens,2012).Paperspub- lishedinthesejournalsrelymainlyoneconomictheoryanddraw onlargearchivaldatasets(Bonneretal.,2012;Merchant,2010;
Oleretal.,2010).Facedwiththepressuretopublishinthesejour- nals,anumberofauthorsarguethatresearchersareincentivized toconductresearchthatbestaligns–concerningbothtopicsand researchapproaches–withpaperspreviouslypublishedbythetar- getjournals(ChenhallandSmith,2011;Maher,2001;Merchant, 2010;Merchant,2013;Hermanson,2015).Consequently,content, methodandtheorydiversityislikelytodecline.Accordingly,such pressuresputtheaforementionedexpectationsintoperspective.
Takingourexpectationsbasedonmodelsofscientificenquiryand thepreviouslyoutlinedconcernstogether,wespecifythefollowing propositions:4
Proposition1.
ContentdiversityofPMARincreasesamongthe1980s,1990sand 2000sanddecreasesbetweenthe2000sand2010s.
Proposition2.
MethoddiversityofPMARincreasesamongthe1980s,1990sand 2000sanddecreasesbetweenthe2000sand2010s.
Proposition3.
TheorydiversityofPMARincreasesamongthe1980s,1990sand 2000sanddecreasesbetweenthe2000sand2010s.
4 For simplicity, we refer to “the 1980s” (etc.), which reflects the period 1980–1982.Weacknowledgethatthistermdoesnotimplyareferencetotheentire 1980–1989period.
2.3. PropositionsonthedevelopmentofvalidityinPMAR
Inadditiontodiversity,acquiringscientificknowledgeabout MA practice requires that PMAR investigates the intended phenomenon faithfully. In this context, validity refers to the
“approximatetruth”of theinferencesfromempiricalinvestiga- tions(Bisbeetal.,2007;Shadishetal.,2002,p.35).Thevalidityof inferencesisa“coreconcern”forPMAR(LuftandShields,2014,p.
551)becauseitfocusesonregularitiesamongmanyinstancesbut reliesonalimitedsetofobservationstodrawinferences(Lukka andKasanen,1995).Inthesocialsciences,evaluationsofempiri- calresearchconventionallyrelyonfourtypesofvalidity:internal validity,externalvalidity,constructvalidityandstatisticalconclu- sionvalidity(Abernethyetal.,1999;AtkinsonandShaffir,1998;
Birnbergetal.,1990;LuftandShields,2014;Modell,2005;Modell, 2009;Shields,2015;VanderStedeetal.,2005).
Internal validity refers to whether the observed covariation betweenadependentandindependentvariablereflectsacausal relationship (Birnberg et al.,1990; Shadish et al., 2002, p. 53).
Therefore,internalvalidityisakeyissuefortheory-testingstud- iesthatseektoexplainMAphenomena(SnowandThomas,1994;
VanderStedeetal.,2005).IfPMARgraduallymovesfromdescrib- ingtoexplainingMApractices,asarguedinSection2.1,increasing emphasisshouldbeplacedoninternalvalidity.Consequently,we expectthattheinternalvalidityofPMARincreasesovertime.Exter- nalvalidityreferstowhethercausalrelationsidentifiedinonestudy canbegeneralized to widersettings and populations(Birnberg etal.,1990;Shadishetal.,2002,p.83).Modelsofscientificenquiry suggest that explanatoryresearchfirst investigates correlations amongvariablesunderspecificconditionsandthenmovesonby buildingtheorieswithmoregeneralpropositionsandwiderappli- cability(Kaplan,1986).Forthisreason,wearguethattheexternal validityofPMARincreasesovertimeasresearchprogressivelyaims attestingtheoriesthatareapplicabletoavarietyofsettings.
Constructvalidityreferstowhetheranoperationaldefinition ofaconstructisanadequatemeasureoftheunderlyingtheoret- icalconcept(Abernethyetal.,1999;Shadishetal.,2002,p.p.65).
Constructsplayakeyroleintheorybuildingandtesting(Hamann et al.,2013; Shadishet al.,2002,p. 100). Therefore,we expect thatresearchersdevoteincreasingattentiontoconstructvalidityas PMARbecomesmoreexplanatory.Aprogressingcomplexityofthe relationsinvestigatedreinforcesthisassumption(LuftandShields, 2003).Whereasearlyexplanatorystudiesregularlyfocusoncore propositions,subsequentresearchconsidersadditionalvariables, thusresulting inmore complexempirical models(Colquittand Zapata-Phelan,2007;SnowandThomas,1994).Statisticalconclu- sionvalidityreferstotheuseofappropriatestatisticsforinferences aboutthecovariationbetweendependentandindependentvari- ables(Shadishetal.,2002,p.37).Associationsbetweenvariables maybepoorlyestimatedinthepresenceofthreatssuchasendo- geneityormulticollinearity(Ittner,2014;NikolaevandvanLent, 2005).Inparticular,investigatingmorecomplexrelationshipsis likelytogiverisetosuchissues(ChenhallandMoers,2007).There- fore,weexpectthatresearchersincreasinglytestfortheseeffects toavoidbiasedestimationsofassociations,implyingthatstatistical conclusionvalidityincreasesovertime.
In summary, weargue that theassumed tendency of PMAR tomovefromprimarilydescriptivetomoreexplanatoryresearch affectsthefourdimensionsofvaliditythat playpivotalrolesin buildingandtestinggenerallyapplicablepropositions.Incontrast tothediversitydimensions,theliteraturedoesnotincludeargu- mentsthatputourexpectationsintoperspective.Forthisreason, wespecifythefollowingpropositions:
Proposition4.
InternalvalidityofPMARincreasesovertimeperiods.
Proposition5.
ExternalvalidityofPMARincreasesovertimeperiods.
Proposition6.
ConstructvalidityofPMARincreasesovertimeperiods.
Proposition7.
StatisticalconclusionvalidityofPMARincreasesovertimeperiods.
3. Method
3.1. Selectionofjournalsandpapers
OurjournalselectionfollowsthatofHesfordetal.(2007),who focuson“outletsinwhichmanagementaccountingresearchhas beenprominently published” (p.5). Thisselection alsoreflects theleadingjournalsaccordingtoaccountingfacultysurveysand journalrankings(e.g.,BallasandTheoharakis,2003;Bonneretal., 2006). However, we modified the selection in three respects.
Sincewefocusonpositivistresearch,weexcludedtheJournalof AccountingLiteratureduetoitspreviouslyexclusivefocusonlit- eraturereviews.For the samereason, we excludedtheReview of Accounting Studies, which primarily publishes MA research thatdrawsonmathematicalmodelling(Hesfordetal.,2007).We enhancedthejournalselectionbyaddingtheEuropeanAccount- ing Review becauseof its significant reputationas the leading journal of theEuropean AccountingAssociation and itsimpor- tant role in disseminatinga diverseset of MAresearch papers (Bhimani,2002;Raffournierand Schatt,2010;Van Campenhout and Van Caneghem, 2010).Thus, our final selection comprises thefollowing nine journals(inalphabetical order):Accounting, OrganizationsandSociety(AOS),BehavioralResearchinAccount- ing(BRIA), ContemporaryAccountingResearch (CAR), European AccountingReview(EAR),JournalofAccountingand Economics (JAE),JournalofAccountingResearch(JAR),JournalofManagement AccountingResearch(JMAR), ManagementAccountingResearch (MAR)and The Accounting Review(TAR). For thereasons out- linedinSection1,weselectedpaperspublishedinfourperiods, eachconsistingofthreeyears(1980–1982,1990–1992,2000–2002, 2010–2012).5
DuetotheirexclusivefocusonMA,weincludedallfullpapers published in JMAR and MAR during the aforementioned peri- odsthatemployapositivistresearchapproach.6 Concerningthe remainingsevengeneralaccountingjournals,twoauthorsinde- pendentlyread theabstractsofallfullpaperstoidentifyPMAR papers. In this context, we reliedonthe definitionof MApro- posedbyFosterandYoung (1997).7Thisapproachresultedina highdegreeofconsensus.Incaseofdisagreement,allthreeauthors reviewedthepaperandmadeajointdecision.Theselectionpro- cessesidentified375PMARpapers.Thesepapersarealmostevenly
5Weconcedethatthenon-randomsamplingdoesnotallowforinferencesregard- ingotheryears.Instead,ourstudyprovidesacomparisonoffourdistinctiveperiods.
Wealsoacknowledgethattheperiodsmaysynchronizewiththetenuresofindi- vidualeditors.Therefore,differencesamongtheperiodsmaybedrivenbyvarying editorialpreferences.Forthisreason,weidentifiedtheeditorsinchargeofthe respectivejournalsforeachyearandcheckedwhetherrotationsoccurred.Formost journalswithlimitededitortenure,rotationsoccurredduringtheindividualperiods.
Therefore,weexpectthisissuetohavelimitedimpactonourfindings.
6Theresearchmethodsconsideredcorrespondwiththecategoriesdistinguished forthecontentanalysisasexplainedinAppendixA(Code2).
7AccordingtoFosterandYoung(1997,p.64),MAconstitutesa“valueadding, continuousimprovementprocessofplanning,designing,measuring,andoperat- ingnonfinancialandfinancialinformationsystemsthatguidesmanagementaction, motivatesbehaviour,andsupportsandcreatestheculturalvaluesnecessaryto achieveanorganization’sstrategic,tacticalandoperativeobjectives”.
dividedbetweenNorthAmerican(52.8%)andEuropeanjournals (47.2%).8
3.2. Contentanalysis
The papers were subject to a content analysis. The corre- spondingcategoryschemeprimarilydrawsontheframeworkby ScanduraandWilliams(2000),whoapplyitinacomprehensive assessmentoforganizationalstudies.Werefertothisframework asitcomprisesvariousdimensionsthatreflectourresearchinter- estindiversityandvalidity.Table1providesanoverviewofthe correspondingcodingdimensions.Eachcodingdimensionconsists ofcategoriestowhichweassignedthestudies.Whereastheselec- tionofcodingdimensionsisderivedfromScanduraandWilliams (2000),wedevelopedtherespectivecategoryschemesbasedon therelatedMAliterature(e.g.,Hesfordet al.,2007;Lillis,1999;
Shields,1997;VanderStedeetal.,2005).Therationalesforthecod- ingdimensionsandthedescriptionofthecorrespondingcategories aredisclosedinfulldetailinAppendixA.
Twooftheauthorsperformedthecodingproceduresindepen- dentlybasedoncodingguidelinesthatincludedexplanationsof thecategoriesandexamples.First,bothauthorsconductedapre- codingbasedonarandomizedsampleof40papers,comparedtheir codings and discusseddeviations. This review ledtofew clari- ficationsregardingthecategorydefinitions.Subsequenttothese modifications, the authors performed thefinal coding. Codings werereviewed and compared after50 papers were completed.
Anydeviationwassubjecttodiscussionuntiltheauthorsreached anagreementthatwasconsistentwiththetreatmentoftheother papers.
3.3. Dataanalysis
In line withScandura and Williams (2000), our dataanaly- sisreliesona seriesofdescriptivestatisticsandstatisticaltests.
Fornominaldata,weperformedchi-squareteststoidentifyasso- ciations between particular coding dimensions. We performed multi-samplemediantestsandKruskal-Wallisteststodetermine whetherdifferencesamongordinaldatafromthefourperiodswere significant.Inaddition,weusedlinearregressionanalysestodeter- minewhetherchangesamongperiodsreflectcontinuity(Scandura andWilliams,2000).
Thedegreeofcontent,methodandtheorydiversitywasmea- suredbycalculating“heterogeneityindices”(HarrisonandKlein, 2007;ScanduraandWilliams,2000)thatreflecttherangeoftopics, methodsandtheoreticalperspectives.Theseindicescomplement ourdescriptiveanalyses.Wecalculatedtheseindicesusingthefol- lowingformula(Blau,1977;p.78):
hj,k=1−
ni=1
p2i
wherehj,krepresentstheheterogeneityindexforcodejinperiod k.pi isthefractionof thecontent-analysed studiesassigned to categoryiofcodejwithnasthenumberofcategoriesincodej.
Thevaluesoftheindexrangefromzero(minimumheterogeneity) to(n−1)/n(maximumheterogeneity).Thelatterimpliesthatthe studiesareevenlydividedamongthecategoriesofaparticularcode
8Ourjournalselectionfacilitates comparisonsbetweenEuropeanandNorth Americanjournals,whichmayaddressclaimsthatthelattercontributeconsiderably toanincreasingnarrownessofMAresearch(e.g.,Merchant,2010).Assupplemen- taryanalyses,wethereforeperformedeachanalysisseparatelyforEuropeanand NorthAmericanjournals.Ifconsiderabledifferencesemerged,wereportthemin Section4.
Table1
ContentanalysiscodingdimensionsderivedfromtheScanduraandWilliams(2000)framework.
Dimensionof analysis
Typeof diversity/validity
Codingdimension(Codein brackets)
Descriptiona
Diversity Contentdiversity Content(1) Referstothesubjectareasstudiedinthepapers.
Methoddiversity Researchmethod(2) Referstothemethodsemployedinthestudies.
Theorydiversity Theoreticalperspective(3) Referstothetheoreticalperspectivesemployedinthestudies.
Supplementarydimensionofanalysis Levelofanalysis(4) Referstothelevelatwhichtheissueunderstudyoccurs.
Validity Internalvalidity Timeframe(5) Referstowhetherdatawerecollectedcross-sectionallyor longitudinally.
Externalvalidity Typeofsample(6) Referstothesettingofthestudies.
Primaryoccupation(7) Referstotheprofessionalbackgroundofstudyparticipants.
Constructvalidity Numberofmeasuresfor constructvalidation(8)
Referstothemeasurestakentoaddressconstructvalidityas disclosedinthestudies.
Numberofreliabilitymeasures (9)
Referstothemeasurestakentoaddressreliabilityofresearch designsasdisclosedinthestudies.
Typeofdependentvariables (10)
Referstothetypesofdependentvariablesemployedinthestudies.
Numberofdatasources(11) Referstothenumberofdatasourcesusedinthestudies.
Statistical conclusionvalidity
Considerationofthreatsto statisticalconclusionvalidity (12)
Referstothemeasurestakentoaddressthreatsthatmayleadto incorrectstatisticalconclusionsasdisclosedinthestudies.
aExplanationsofcodingdimensionsandcorrespondingcategoriesaredisclosedinfulldetailinAppendixA.
(HarrisonandKlein, 2007).Wedrawprimarilyontheseindices fortimecomparisons.Anincreasingheterogeneityindexsuggests increasingdiversity,whichmayenhanceourunderstandingofMA practices.
4. Findings
Intheremainderofthissection,wefirstaddresstheissueof diversityandpresenttheresultsforPropositions1–3thatreferto content,methodandtheorydiversity(Section4.1).Inthesecond step,wefocusonvalidityandprovidefindingsforthePropositions 4–7(Section4.2).Inthesub-sectionsthatrefertotherespective propositions,weprovideaseriesofanalysesaspreviouslyoutlined.
4.1. FindingsonthedevelopmentofdiversityinPMAR 4.1.1. Contentdiversity(Proposition1)
Proposition1referstotherangeoftopicsstudiedandsuggests anincreaseincontentdiversityamongthefirstthreedecadesand adecreasebetweenthe2000sand2010s.Table2reportstheclas- sification ofpapers according totheircontent (Code1). In this context,werelyonaslightlymodifiedcategoryschemereportedby Hesfordetal.(2007).Inthe1980s,PMARinvestigatesalmostevenly decision-facilitating(category“Costandplanning”; 45.8%ofthe papers)aswellasdecision-influencing(category“Control”;54.2%) issues.Inthetwofollowingperiods,thesharesofpapersfromboth categoriesdecrease,whereastheshareofpapersonothertopics grows.However,thesetendenciesreverse inthe2010s ascon- trolissuesareincreasinglyaddressed,withaparticularemphasis placedonperformancemeasurement(43.8%ofallpaperspublished inthisperiod).FollowingScanduraandWilliams(2000),wecon- ductedtime-basedregressionsforallcodingdimensionstodetect whetherchangesovertimerepresentalinearpatternthatindi- catessomecontinuityinthedevelopments.For this reason,we performedsimplelinearregressionanalysesforeachcategory.The yearofpublicationisthedependentvariableandadummyforthe respectivesub-categoryistheindependentvariable.9Theregres-
9 Forinstance,theregressionforthecategory “Control”employsadummy variableastheindependentvariablethatequals1forpapersassignedtoanysub- categoryof“Control”and0otherwise.Increasesordecreasesrefertothenumber ofstudies.Weconductedequivalentanalysesforeachofthecodesexceptforcodes 8and9.Wereportsignificantfindingsinthetables(column“Overalltrend”).Ifwe donotreportatendencyforacategory,findingswereinsignificant.
sionresultsunderlinetheaforementioneddevelopmentaswefind thatthedecreaseincost andplanningstudiesandtheincrease incontrolstudiesaresignificantatthe1%and5%levels,respec- tively.Inaddition,wedetectsignificantdevelopmentsforvarious sub-categories(e.g.,budgetingorperformancemeasurementand evaluation),asshowninTable2(column“Overalltrend”).Corre- spondingly,theheterogeneityindexshowninTable2increases duringthefirstthreeperiods(1980–1982:0.75;1990–1992:0.83;
2000–2002:0.86)anddeclinesinthelastperiod(2010–2012:0.76).
Overall,thesefindingssupportProposition1.10
4.1.2. Methoddiversity(Proposition2)
AccordingtoProposition2,methoddiversityincreasesoverthe firstthreetimeperiodsbutdeclinesinthefourthperiod.Inref- erencetothecategoriesofHesfordetal.(2007),Table3(Panel A)showsthespreadsofresearchmethods(Code2)intherespec- tiveperiods.Thetablerevealsthatexperimentalstudiesrepresent 60.0%ofPMARinthe1980s,whichcorrespondswithacompara- tivelylowheterogeneityindexof0.57.However,sincethe1990s, methoddiversity has grown substantially due to anincreasing relianceon archival, survey, and field studies. From the 1990s onwards,we finda balanced spectrum ofresearch approaches, whichisalsoreflectedbyhighandstableheterogeneityindices (1990–1992:0.76;2000–2002and2010–2012:0.77).Time-based regressionanalysesindicateasignificantincreaseinthereliance onarchivalstudies(p<0.01)andasignificantdecreaseinexperi- mentalresearch(p<0.1).
In Section2.2,we arguedthat methoddiversity impliesthe potentialtoovercomethelimitationsofotherresearchapproaches.
Therefore,itseemsimportantthatnotonlyPMARingeneralbut alsospecificsubjectareasarediversified.Forthisreason,wecross- tabulatethesubjectareas(i.e.,costandplanning,controlandother) andtheresearchmethodsemployed.In thisway,weilluminate whethertheoverallmethoddiversityismirroredinthemainsub- jectareas.Thefindings reportedinTable3(PanelB) indicatea comparativelystrongrelianceonarchival,surveyandexperimental researchforcostandplanningaswellascontrolissues.Incontrast, researchonothertopicstendstorelymorestronglyoncaseand
10SupplementaryanalysesofpaperspublishedinNorthAmericanandEuro- peanjournalsrevealthattheoveralltendencyisreflectedbybothjournalgroups.
Moreover,theEuropeanjournals(averageheterogeneityindex(AHI):0.58)arenot considerablymorediversifiedthantheNorthAmericanones(AHI:0.55).
Table2 Contents.
Content(Code1)† 1980–1982#(%) 1990–1992#(%) 2000–2002#(%) 2010–2012#(%) Total Overalltrend Costandplanning
Costallocation 0a(0.0%)b 3(3.8%) 12(10.7%) 12(7.5%) 27
Othercostaccountingtopics 2(8.3%) 6(7.6%) 6(5.4%) 1(0.6%) 15 ***(−)
Costpractices 0(0.0%) 4(5.1%) 3(2.7%) 10(6.3%) 17
Budgeting 7(29.2%) 12(15.2%) 13(11.6%) 8(5.0%) 40 ***(−)
Capitalbudgeting 2(8.3%) 2(2.5%) 2(1.8%) 6(3.8%) 12
Total“Costandplanning” 11(45.8%) 27(34.2%) 36(32.1%) 37(23.1%) 111 ***(−)
Control
Performancemeasurementandevaluation 8(33.3%) 27(34.2%) 27(24.1%) 70(43.8%) 132 *(+)
Organizationalcontrol 5(20.8%) 4(5.1%) 17(15.2%) 25(15.6%) 51
Internationalcontrol 0(0.0%) 5(6.3%) 6(5.4%) 0(0.0%) 11 *(−)
Multiple 0(0.0%) 2(2.5%) 2(1.8%) 6(3.8%) 10
Total“Control” 13(54.2%) 38(48.1%) 52(46.4%) 101(63.1%) 204 **(+)
Other
Quality(TQM) 0(0.0%) 0(0.0%) 1(0.9%) 0(0.0%) 1
Just-in-time 0(0.0%) 1(1.3%) 2(1.8%) 0(0.0%) 3
Strategicmanagement 0(0.0%) 9(11.4%) 16(14.3%) 18(11.3%) 43
TransferPricing 0(0.0%) 2(2.5%) 3(2.7%) 0(0.0%) 5
Multiple 0(0.0%) 2(2.5%) 2(1.8%) 4(2.5%) 8
Total“Other” 0(0.0%) 14(17.7%) 24(21.4%) 22(13.8%) 60
Heterogeneityindices 0.75 0.83 0.86 0.76
Referstothenumberofpapers(n=375).†Sub-categories(6),(11)and(14)arenottabulatedsincenopapershavebeenassignedtothem.aAbsolutenumberofpapers.b Percentageofpapersfromtherespectivecategoryoutofthetotalnumberofpaperspublishedwithintheperiod.Findingsfromtime-basedregressionanalyses:***—significant decrease(−)(p<0.01);**—significantincrease(+)(p<0.05);*—significantincrease(+)ordecrease(−)(p<0.1).
Table3
Researchmethods.
PanelA:Researchmethod(Code2)perperiod
1980–1982#(%) 1990–1992#(%) 2000–2002#(%) 2010–2012#(%) Total Overalltrend
Archival 0a(0.0%)b 13(15.9%) 24(19.5%) 46(26.9%) 83 ***(+)
Survey 6(24.0%) 27(32.9%) 39(31.7%) 38(22.2%) 110
Experimentalc 15(60.0%) 23(28.0%) 27(22.0%) 50(29.2%) 115 *(−)
Casestudy 2(8.0%) 8(9.8%) 10(8.1%) 18(10.5%) 38
Fieldstudy 2(8.0%) 11(13.4%) 23(18.7%) 19(11.1%) 55
Heterogeneityindices 0.57 0.76 0.77 0.77
PanelB:Researchmethod(Code2)percontentgroup(Code1)
Costandplanning#(%) Control#(%) Other#(%) Total
Archival 27d(23.5%)e 50(22.8%) 6(9.0%) 83
Survey 39(33.9%) 56(25.6%) 15(22.4%) 110
Experimental 30(26.1%) 65(29.7%) 20(29.9%) 115
Casestudy 10(8.7%) 16(7.3%) 12(17.9%) 38
Fieldstudy 9(7.8%) 32(14.6%) 14(20.9%) 55
Heterogeneityindices 0.75 0.77 0.77
Referstothenumberofsamples(n=401).PanelA:aAbsolutenumberofsamples.bPercentageofsamplesfromtherespectivecategoryoutofthetotalnumberofsamples publishedwithintheperiod.Findingsfromtime-basedregressionanalyses:***—significantincrease(+)(p<0.01);*—significantdecrease(−)(p<0.1).cIncludestwofield experiments.PanelB:dAbsolutenumberofsamplesemployingtherespectiveresearchmethodpercontentgroup.ePercentageofsamplesfromtherespectiveresearch methodoutofthetotalnumberofsamplesassignedtotherespectivecontentgroup.Chi-squaretestforcross-tabulationofmethodsandcontentgroups:2=20.158;p=0.01;
n=401.
fieldstudies.Nonetheless, theheterogeneityindices persubject area(rangingfrom0.75to0.77)suggestaconsiderablediversityin allofthem.Inaddition,untabulatedfindingsindicatethatmethod diversityincreasesovertimeorisatleaststableinallthreesubject areas.Therefore,weconcludethatthemainsubjectareasshowa highmethoddiversity.11
Insummary,ourfindingscorroborateProposition2onlypar- tially.Wefindevidenceforanincreasingmethoddiversityfrom
11 UntabulatedcomparisonsbetweenNorthAmericanandEuropeanaccount- ingjournalsrevealthattheresearchmethodsmostfrequentlyappliedinstudies publishedinNorthAmericanjournalsarearchivalandexperimentalapproaches, whereasthemethodsmostfrequentlypublishedinEuropeanjournalsaresurveys, caseandfieldstudies.However,contrarytotheviewsexpressedintheliterature (e.g.,Callen,2015;Merchant,2010),Europeanjournals(AHI:0.64)donotappearto beconsiderablymorediversifiedthanNorthAmericanjournals(AHI:0.61).
the1980stothe1990sandalmostconstantmethoddiversityin thefollowingtimeperiods.Accordingly,ourfindingsdonotindi- catetheanticipateddeclineinmethoddiversityfromthe2000sto the2010s.
4.1.3. Theorydiversity(Proposition3)
Proposition3suggeststhattheorydiversitywillfollowasimi- larpathasmethoddiversity.Table4summarisesourclassification ofstudieswithregardtotheirtheoreticalperspectives(Code3) basedonthecategoryschemereportedbyShields(1997).Panel Ashowsthatstudiespublishedinthe1980sdrawmostlyonpsy- chologicaltheoriestoexplaintheirfindings,whereas aplethora oftheoreticalperspectivesisusedinthefollowingdecades.More precisely,overthelastthreedecadesPMARmostfrequentlydraws oneconomic(between15.9%and 22.8%)andpsychologicalthe- ories(between 15.4% and22.8%) andtheoriesof organizational
behaviour (between 23.2% and 27.5%). Time-based regressions reveal significant increases regarding organizational behaviour (p<0.05)andproductionandoperationsmanagement(p<0.1),as wellasasignificantdecreaseinstudieswithoutanexplicitthe- oreticalperspective(p<0.01).Thehighandstableheterogeneity indicesfromthe1990s onwards(1990–1992:0.82;2000–2002:
0.81;2010–2012:0.79)indicatestrongandpersistenttheorydiver- sity.Inotherwords,theydonotindicateanincreasingnarrowness intermsoftheoreticalperspectives.
InSection2.2,wearguedthattherelianceondifferenttheoret- icalperspectivesmayreinforceourunderstandingofMApractice.
Wethusanalysewhetherthemainsubjectareasshowasimilar degreeoftheorydiversityasPMARingeneral.Forthisreason,we cross-tabulatethetheoreticalperspectivesandthesubjectareas asshowninTable4(PanelB).Theheterogeneityindicespersub- jectareadifferonlyslightly(rangingfrom0.78to0.83)andsuggest thatmultipletheoreticalperspectivesareusedineachofthesesub- jectareas.Nevertheless,theanalysisrevealsarea-specificpatterns, suchastheinfrequentreferenceofstudiesonothertopicstoeco- nomictheoriesorthelowshareofcontroltopicsthatdonotrelyon anexplicittheoreticalperspective.Untabulatedtimecomparisons showthattheorydiversityincreasesintheareasofcostandplan- ningandothertopics,whereasitslightlydeclinesinthecontrolarea sincethe1990s.12Similarlytothecaseofmethoddiversity,wefind partialsupportforProposition3.Fromthe1990sonwards,theory diversityishighandlargelystable.Hence,wedonotfindevidence forasubstantialdecreaseintheorydiversityinthe2010s.
4.1.4. Supplementaryanalysisonlevelsofanalysis
InSection2.2,wearguedthatdifferenttheoreticalperspectives focusondifferentlevelsofanalysis(Covaleskietal.,2003;Luftand Shields,2003).ForafurtherassessmentofPMAR,weilluminate whetherthedevelopmentsconcerningthetheoreticalperspectives implychangesin thelevelsofanalysis.For thisreason, wefol- lowScanduraandWilliams(2000)andsupplementouranalysis ondiversitywithaninvestigationoftheselevels.InlinewithLuft andShields(2003),werefer tothedependentandindependent variablesinvestigatedanddefinethelevelofanalysis(Code4)as theone“atwhichthevariationofinterestoccurs”(p.175).Ifno variableswereemployed,weidentifiedthelevelofanalysisbased ontherespectiveresearchquestions.Ifatleasttwodifferentlevels wereaddressed,weassignedthestudytothecross-levelcategory.
Table5(PanelA)reportsdevelopmentsovertimeandshows thatPMARprimarilyfocusesontheindividuallevelinthe1980s.
Inthesubsequentdecades,ithasshiftedmoretowardstheorgani- zationalandthebeyond-organizationallevels.Moststudiesrefer toone levelofanalysis,whereascross-levelstudiesaccountfor roughly10%ofthestudies.Time-basedregressionsrevealsignif- icantincreasesfortheorganizationalandbeyond-organizational levels(p<0.1andp<0.05respectively)andasignificantdecrease instudiesattheindividuallevel(p<0.01).
Our illumination of the levels of analysis per subject area (Table5,PanelB)revealstwonotablepatterns.First,PMARoncon- trolissuesfocusesmorestronglyontheindividuallevelthanthe othertwosubjectareas.Second,wefindthatovertime,PMARon costandplanningissuesrefersincreasinglytothebeyond-orga- nizationallevel.Overall, thissupplementaryanalysissuggests a
12 AsupplementarycomparisonofNorthAmericanandEuropeanjournalsshows thatstudiespublished inNorthAmerican(European)journalsrelymainlyon economicandpsychological(sociologicalandorganizationalbehaviour)theories.
Inbothgroups,theorydiversityincreasesfromthe1980sto1990sandslightly decreasesthereafter.Moreover,theorydiversityappearsslightlyhigherinNorth Americanjournals(AHI:0.76)thaninEuropeanjournals(AHI:0.72).
gradualshiftinthefocusofPMARfromtheindividualtotheorga- nizationallevelandbeyond.
4.2. FindingsonthedevelopmentofvalidityinPMAR 4.2.1. Internalvalidity(Proposition4)
Proposition4suggeststhatinternalvalidityincreasesovertime.
FollowingScandura and Williams (2000) , we address internal validitybyexaminingthetimeframes(Code5)ofthestudies.Lon- gitudinalstudiestendtoimplyahigherlevelofinternalvalidity thancross-sectionalonesbecausetheyallowresearcherstodirectly identifycausalrelationshipsinwhichcausesprecedeeffects(Van der Stede,2014).Table 6 (Panel A) indicatesthat the share of longitudinalstudiesdecreasesamongthefirstthreeperiodsand increasesfromthe2000stothe2010s.Thisfindingthuscorre- spondsonlypartiallywithProposition4.
In a second step, we illuminate whether this development isdrivenbyparticularresearchmethods.Caseand fieldstudies (AhrensandDent,1998),archival(Moers,2007)andexperimen- talresearch arefrequently longitudinal(Maines et al.,2006).13 Bycontrast,VanderStedeetal.(2005)findthatMAsurveysare usuallycross-sectional,althoughmost“aimtotest theoriesthat specifycausalrelationshipsamongvariables”(p.665).Forthisrea- son,cross-tabulatingtimeframesandresearchmethodsmayshed lightonthequestionregardingwhetherPMARhasovercomethe lattershortcoming.ThefindingspresentedinTable6(PanelB)con- firmthat archival,experimentaland case studiesmostlygather longitudinaldata.Theyaccountfor88.2%ofthelongitudinalstud- ies,whereassurveysthatcollectdataattwoormorepointsintime remainarareexceptionoveralltimeperiods.Insummary,wefind limitedsupportforProposition4.Moreover,weconcludethata considerablelimitationwithregardtosurveyresearchprevails.
4.2.2. Externalvalidity(Proposition5)
According to Proposition 5, external validity increases over time. In linewithScanduraand Williams (2000),we focus on thetypesofsamplesandtheprimaryoccupationofparticipants ascodingdimensionsthatshedlightonthepotentialtogeneral- izefindings(Dahlstrometal.,2008).Table7reportsourfindings onthetypes ofsample(Code6)based onthecategory scheme byScapensandBromwich(2010b).“Generic”referstolaboratory experiments,whereastheothercategoriesrepresenttheremaining researchmethods.Thefindingssuggestthattheshareofstudieson manufacturingdecreasesinfavourofstudiesthatfocusoneither specificindustries, theservice sector orboth (subsumed under othertypesofsamples).Thesetendenciesareemphasizedbytime- basedregressionsthatrevealsignificantincreasesforstudiesinthe publicsectororothersettings(p<0.05andp<0.01respectively) andsignificantdecreasesforstudiesinagenericormanufacturing setting(p<0.1andp<0.01respectively).Overall,PMARseemsto refertoanincreasingvarietyofsettings,whichmayfacilitatethe identificationofdifferencesandcommonalitiesandincreasethe generalizabilityoffindingsamongdifferentsectors.
Becauseexternal validity alsorefers togeneralizingfindings amongdifferentindividuals,groupsandoccupations(Brutusetal., 2013;ScanduraandWilliams,2000),wecapturetheprimaryoccu- pation(Code7)oftheparticipants.Forthisreason,werefertothe categoriesreportedbyScanduraandWilliams(2000),whichwe adaptedtotheMAcontext.Notethatthisanalysisexcludesarchival studiesthatdonotinvolveparticipants.Table8(PanelA)reports
13Notethatweconsiderexperimentalstudieslongitudinalevenifdatawerecol- lectedatonepointintimebecausethemanipulationofindependentvariables precedestheobservationofthedependentvariables.Therefore,causalitycanbe established.
