Outcome eect, controllability and performance evaluation of
managers: some ®eld evidence from multi-outlet businesses
Dipankar Ghosh*, Robert F. Lusch
University of Oklahoma, Price College of Business, 397W Brooker, Room 200, Norman OK, 73109, USA
Abstract
This ®eld study provides evidence of the outcome eect in performance evaluations of managers in an organization. Speci®cally, in a retail chain, subjective evaluations of store managers by their supervisors were negatively impacted by unfavorable outcome knowledge. As expected, outcome determinants over which the managers have control in¯uence their performance evaluations and environmental determinants of outcome over which they have no control do not in¯uence their evaluations. However, unexpectedly, central management determinants of outcome over which the managers have no control also in¯uence their evaluations. After these outcome determinants are considered, we ®nd evidence of an outcome eect since failure of the store to meet its target outcome results in a more negative perfor-mance evaluation of the managers. Also, the extent to which store managers' evaluations are prone to the outcome eect is not contingent on the measure of the outcome used.#2000 Elsevier Science Ltd. All rights reserved.
Evaluating managerial performance is an impor-tant function of control in organizations. Both sub-jective (e.g. ratings of subordinates' performance by their superior) and objective (e.g. pro®tability) criteria are used in these evaluations. However, it is well known that the subjective evaluations of managers are correlated with objective measures of outcomes. For example, a meta-analysis by Bommer, Johnson, Rich, Podsako and Mack-enzie (1995) of studies containing both objective and subjective ratings of employee performance resulted in a mean correlation of 0.39. Clearly the manager should be given some credit for the objective performance of the organization that a manager helps to achieve. However, when sub-jective measures of managerial performance are
in¯uenced by objective measures of performance that the manager does not control, then the per-formance evaluation system is less than ideal and therefore the control system is suboptimal.
One of the main reasons for this weakness in performance evaluation and/or control systems is the ``outcome eect'' or bias in which evaluators assess manager's performance based on outcomes (Baron & Hershey, 1988; Hawkins & Hastie, 1990). Thus, when an outcome is positive (nega-tive), evaluators tend to evaluate the manager positively (negatively) regardless of the actual appropriateness of the decision resulting in that outcome. By taking outcomes into account that do not re¯ect managerial performance will aect not only the quality of the evaluation decision but will also incorrectly reward/penalize managers.
As much of the research studying the outcome eect has been undertaken in a laboratory setting,
0361-3682/00/$ - see front matter#2000 Elsevier Science Ltd. All rights reserved. P I I : S 0 3 6 1 - 3 6 8 2 ( 9 9 ) 0 0 0 4 5 - 8
www.elsevier.com/locate/aos
the primary objective of this research is to assess if that eect can be shown to exist in an actual organization. In addressing this problem it was necessary to develop a methodology for studying the outcome eect in the ®eld. Importantly, the development of this methodology was a secondary objective of this research. The development of this methodology is more complex than developing a ®eld study questionnaire. It was necessary to gather data from four distinct sources and the data had to be integrated into a two-step mathe-matical model that was used to isolate the pre-sence or abpre-sence of the outcome eect. The methodology is applicable to a multi-unit business and, thus, can be used as the basis for future stu-dies interested in addressing how a host of orga-nizational factors could alleviate the outcome eect. In general, the methodology can be used to obtain a better understanding of the outcome eect and whether its in¯uence can be mitigated during the manager's performance evaluation.
The research speci®cally addresses two ques-tions: (i) are subjective performance evaluations of segment managers in¯uenced by outcomes after controllability of these outcomes are taken into consideration? (ii) Are outcome eects robust across alternative measures of segment economic outcome, namely, sales and gross pro®t?
The research questions are motivated by several reasons. First, the outcome eect has been docu-mented by a large number of laboratory studies (Baron & Hershey, 1988; Brown & Solomon, 1987,1993; Lipe, 1993; Lipshitz, 1989; Mitchell & Kalb, 1981) which with their high internal validity provides the theory the best chance of being sup-ported. But when a research design emphasizes one type of validity, it necessarily weakens the other (Abdel-khalik & Ajinkya, 1979). Since the two primary criteria for evaluating the quality of research are internal and external validity, the next step, therefore, is to test the theory in a ®eld set-ting with its high external validity. The triangula-tion of research, testing the outcome eect in both the laboratory and a ®eld setting, can demonstrate the robustness of the theory.
Second, evaluating managers in organizations is more complicated than the outcome eect literature implies since the laboratory context is substantially
dierent from the situation faced by an evaluator in an organization (Borman, 1992). For example, in experiments the decision maker is usually eval-uated on a single decision where the connection between the decision and the outcome is rather transparent and the duration between the two is relatively short. In contrast, managers in organi-zations are evaluated for their overall performance which involves a host of interrelated decision variables collectively impacting on outcomes over varying time periods. Furthermore, since the eva-luator is often not completely aware about the manager's decision environment, how the deci-sions impact the outcomes is not transparent. For example, outcomes are commonly delayed and not easily attributable to a particular action and variability in the environment degrades the relia-bility of outcome feedback (Tversky & Kahne-man, 1986). Also, evaluative decisions are rarely context-free. Rather, a broader sample of evidence with various degrees of objectivity and consistency is available, some of which may not pertain to the manager's performance because the manager has no control over them.
Finally, prior research assessing the outome eect did not control for ``uncontrolled'' factors. Decision researchers (and most other people) would argue that good decisions tend to lead to good outcomes and bad decisions lead to bad outcomes. Thus, in the absence of other informa-tion about the initial decision, using outcomes to evaluate the decision is reasonable and, therefore, an outcome eect is not established by merely showing that evaluators take outcomes into account (Hershey & Baron, 1992). However, this is not the case when controllability of outcomes is considered. When an outcome is controllable by the manager then the manager should be held respon-sible for the consequences of his or her decisions (Kelley & Michela, 1980; Tan & Lipe, 1997). Simi-larly, a manager should not be held responsible for the consequences of his or her decisions when an outcome is not controllable by them.
measure (knowledge of whether the target sales and gross pro®t for the period was met or not met). Furthermore, a methodology was developed to partition the continuous economic measures into outcome under the control of store managers and outcome not under their control. If we look at our continuous measure of outcome (sales and gross pro®t), as expected, the results show that sales and gross pro®t over which managers have control in¯uence how they are evauated by their superiors. Also as expected the sales and gross pro®t outcomes determined by the environment over which the managers have no control do not in¯uence their evaluations. However, unexpect-edly the results also show that sales and gross pro®t outcomes controlled by the central manage-ment and over which the managers have no direct control also in¯uences manager's evaluations. This is an unexpected eect; however, we provide a plausible alternative explanation in the discus-sion section. When we study the in¯uence of the dichotomous measure of outcome (i.e. knowledge of target sales and gross pro®t achieved) we ®nd that subjective evaluations of store managers were negatively impacted. Given that we have statisti-cally accounted for the outcomes managers have control or no control over this ®nding provides strong evidence of an outcome eect. Finally, this study found that the outcome eect is generally not contingent on the two dierent economic measures of segment outcome.
The remainder of the paper is organized as fol-lows. The next section reviews the literature and develops the hypotheses. Section three discusses the research method, section four presents the data analyses, and section ®ve reviews the results and examines the implications of this study.
1. Development of research hypotheses
1.1. Performance evaluation and outcome eect
As stated above, the outcome eect has been empirically demonstrated in several experimental studies. Mitchell and Kalb (1981) investigated the outcome eect in supervisor's evaluations of sub-ordinates in a health care setting. The study found
that those supervisors with outcome knowledge, especially in the case of a negative outcome, rated the outcome as more probable, held the subordinate more responsible for the outcome, and made more internal attributions for the outcome than subjects did with no outcome knowledge. Baron and Her-shey (1988), in a series of ®ve studies, gave subjects a set of 12±16 medical and gambling decisions to evaluate and the outcome of each of the decisions. The results were very consistent: the outcome of the decision (good or bad) systematically in¯uenced subjects' evaluations of the quality of the decision.
Accounting research has tried to isolate factors that can eliminate the outcome eect by making the evaluator familiar with the manager's decision environment. This familiarity can facilitate evi-dence recall (other than the outcome) during the manager's performance evaluation (Fischho, 1975). For example, Brown and Solomon (1987, 1993) tried to enhance the involvement of the evaluator in the manager's (i.e., the evaluatee's) decision by acquainting the evaluator with the manager's decision variables. In their ®rst study, evaluators were asked some questions about the manager's task prior to making the performance evaluation. In a subsequent study, the evaluators were made ``advisors'' to the managers and had to have a decision consensus between themselves and the managers. In Fisher and Selling (1993), the eva-luator observed the decision making process of the managers. Finally, in Lipe (1993), expenditures on variance investigation were framed for the evaluator as either a cost or a loss, depending on whether the investigation did or did not ®nd a problem.
objective criteria as well (Borman, 1992). Thus, subjective measures, in spite of their limitations, are most frequently used in manager's performance evaluation (Borman, 1992).
The reason why we observe the outcome eect in manager's evaluations is because outcome knowledge in¯uences the evidence recalled by the evaluator when attempting to assess the perfor-mance of the manager (Slovic & Fischho, 1977). Evidence recall is a deliberate task and since humans have limited information processing capacity (Simon, 1957), ``knowing what happened facilitates knowing where to look for and what to accept as reasons'' (Fischho, 1975, p. 297). Thus, as observed in the experimental studies, the out-come eect should persist in organizations although, as we discuss next, using outcomes to evaluate performance is not always reasonable.
1.2. Outcome eect and outcome controllability
In organizations, managers have varying levels of control over outcomes. In which case it may be reasonable to use outcome knowledge for evalua-tions since they provide us with information about manager's decisions (Baron & Hershey, 1988). But to make the outcome more informative, one must separately model how much the outcomes of the segment are due to actions and decisions of segment managers. Hence, we next identify the decision fac-tors aecting a segment's outcomes and examine whether those factors, grouped by their controll-ability, in¯uences the segment manager's subjective performance evaluation. If after accounting for these factors the manager's subjective performance eva-luation continues to be aected by knowledge of whether the target outcomes were achieved, we will have evidence of an outcome eect.
When is it reasonable to use outcome knowl-edge? The distinction between a good decision and a good outcome is a basic one in all decision ana-lysis because they are not the same (Edwards, 1984). Some argue that using outcome knowledge may be justi®ed and appropriate in some contexts while remaining erroneous in others. In general, when the manager has, or ought to have, more information than the evaluator, outcome is diag-nostic of the quality of the manager's decisions
since it is reasonable to assume that bad (good) outcomes results from poor (good) decisions (Baron & Hershey, 1988). However, if there is no information asymmetry problem, an evaluator should only use the information about potential outcomes and the probabilities and utilities of those outcomes that existed at the time of the decision in evaluating decision quality (Hershey & Baron, 1992).1 Hershey and Baron (1995) speci®cally
addressed the issue of when is it reasonable (or ``normative,'' using their term) to use outcome knowledge and illustrate that this knowledge is sometimes informative about decision quality. For example, using their terminology, assume the eva-luator is trying to determine if the manager made a good decision (i.e. G) or a bad decision (i.e. B) and the only knowledge available to the evaluator is the outcome resulting from the decision, namely, suc-cess (i.e. S) or failure (i.e. F). The outcome knowl-edge can then be used to infer the manager's decision quality provided the following relations among the conditional probabilities hold.
P SjG>P SjBand P FjB>P FjG
Hershey and Baron (1995) concludes ``. . .that the
likelihood of a good decision is increased by the occurrence of a success, and the likelihood of a bad decision is increased by the occurrence of a failure'' (p. 127). Thus, outcome knowledge can be very informative about the decision quality if that deci-sion signi®cantly in¯uences outcome probability.
Further, the extent to which outcome knowledge is informative about the decision is directly related to the degree by which its probabilities are increased by the decision. Thus, when P(SjG) is considerably higher than P(SjB), and P(FjB) is considerably higher than P(FjG), outcome knowledge is more informative regarding decision quality. Such would be the case if the manager has high control over the factors in¯uencing outcome since controllability
increases the responsibility attached to a manager for the consequences of his or her decisions (Kelley & Michela, 1980). Speci®cally, when a manager has high control over the outcome, then P(SjG) will be high with P(SjB) low due to the perceived direct and causal (i.e. controllable) relation between the deci-sion and the outcome. Likewise, P(FjB) will be high with P(FjG) low. Thus, in these situations, the out-come knowledge is more informative regarding the quality of the manager's initial decision. On the other hand, with a more uncontrollable outcome, these direct causal relations are absent and it is di-cult to assess the relative size of the relevant prob-abilities. In that case, outcome knowledge is much less informative regarding the manager's initial decision. Thus, outcome controllability can be a cri-terion not only to establish the reasonableness of the use of outcome knowledge in evaluating decision quality but also indicative of when use of outcome knowledge may be observed in evaluations.
Before proceeding further, we need to better understand the term controllability. Controllable decision factors are those over which the segment manager has considerable latitude. For example, often segment managers can control such things as marketing, production schedules, and pricing. Obviously the actual research setting will determine which factors are controllable. Uncontrollable fac-tors are the environmental facfac-tors and the central oce factors aecting a segment's outcome which a manager cannot control. These factors can include both supply and demand factors that are in the ®rm's external environment and resource support from central management such as facilities and equipment. Therefore, the above discussion suggests that when outcome factors are controllable by the seg-ment manager, it should be reasonable for the eva-luator to consider these factors in evaluating the segment manager's performance. This is consistent with Hershey and Baron's (1995) mathematical results that the conditional probability that is expected under conditions of controllability, leads correctly to use of the outcome knowledge. On the other hand, with regard to factors uncontrollable by the segment manager, considering these factors for evaluation should not be considered as rea-sonable. Thus, once the in¯uence of the outcome factors grouped by their controllability are
accoun-ted for, theoretically, the outcome knowledge should be inconsequential to manager's subjective performance evaluation. However, as discussed earlier, experiments have consistently demonstrated that subjective performance evaluations are biased by the outcome knowledge. For example, even when the evaluators had all the relevant information known to the evaluatee, plus the outcome knowl-edge, the evaluators took outcome into considera-tion in rating the quality of the decision of the evaluatee (Baron & Hershey, 1988). The preceding discussion allows us to state our hypotheses.
H1A: Segment manager's subjective performance evaluation will be aected by sales and gross pro®t outcomes they control and will not be aected by sales and gross pro®t outcomes they do not control.
H1B: After accounting for the controllability of sales and gross pro®t outcomes, the segment manager's subjective performance evaluation will be aected by knowledge of whether the target sales and gross pro®t goal was achieved (i.e. target outcome knowledge).
With this discussion of theory, hypotheses, and de®nitions we now move to a more concrete dis-cussion of the speci®c research setting and method for hypotheses testing.
2. Research method
2.1. Overview of the research setting
can isolate the outcomes (sales and gross pro®t) controllable by the store manager. There are two reasons why the retail chain in this study provides an appropriate ®eld setting to detect if outcome eects occur in the more complex setting of an actual organization. First, each store is a decen-tralized segment of a retail chain as well as a distinct and stand alone economic unit. Thus, store-spe-ci®c accounting information is less noisy and fac-tors aecting each store's outcome are more easily identi®ed. Second, the regional managers who evaluate the performance of the store managers were once store managers themselves. Further, they very closely monitor each store under their supervision, including frequent store visits. Thus, it can be assumed that by virtue of their experi-ence, the regional managers were familiar with the factors that signi®cantly in¯uence store outcomes (Davis, 1982; Choo, 1989).
2.2. Data sources and data collection
The data were collected from a chain store retailer with almost 250 stores in the United States. The statistical analysis was based on 204 stores for which complete data were available for all variables. The retail units are located in ®ve major metropolitan statistical areas (MSA) and rural towns in nearly equal numbers. Most stores have 3000±10,000 square feet of selling area while more recent units are larger, ranging from 15,000 to 25,000 square feet of selling space. Each store oers a wide range of general merchandise items, catering to a relatively narrow trading area with a tight customer focus. Most stores are fairly uni-form in terms of their merchandise mix and store layouts. The stores are staed by a store manager as well as one or more assistant managers. A regional manager monitors the operations of 10± 30 retail units by regularly visiting the stores and reviewing reports on the ®nancial performance of the units. In addition, each regional manager has previously served as a store manager in this orga-nization and, thus, is familiar with the decisions and control the manager has over operations.
The data were collected from four sources. First, the relevant economic and operating character-istics of each store were obtained from the internal
accounting records. For each store, there were income statements, selected balance sheet data and information about other resources. For this study, the following store-speci®c data items were obtained: annual sales, gross pro®t, average inventory investment, annual advertising expendi-tures, gross pro®t percentage, number of full time employees, total square feet of space for the store and total square feet of selling space for the store, and if the store's target annual sales and gross pro®t were met. Second, we were able to obtain the internal records data on the regional man-ager's annual rating of the store manman-ager's eec-tiveness or performance. Third, supply related environmental factors, namely, the size of the market, the size in square feet of the store's num-ber one competitor, the numnum-ber of competing stores in the trade area or store saturation, and the economic health of the trade area, were obtained. These data were obtained from a measurement instrument designed jointly by the researchers and the retail organization to provide information on the nature and scope of competition in each trade area as well as the overall economic health of the trade area in which the store was located.2A form
was sent to each store manager requesting infor-mation on competitors. This form was provided through the normal organizational channels to each manager with a cover letter and a set of instructions. Each returned form was reviewed by the researchers and the regional manager for missing or miscoded items. If required, the form was returned to the store manager to ensure com-plete and accurate responses. Finally, the demand related environmental factor, namely, the market match which depended on the population mix of the trade area, was purchased from National Planning Data Corporation. This ®rm commer-cially supplies demographics, based on U.S. Cen-sus of Population and Housing, for trade areas, as de®ned by the client.
2.3. Segment (i.e. store) outcomes
Store outcome was gauged using two measures: annual net sales per square foot of selling space (hereafter, referred to as SALES) and annual gross pro®t per square foot of selling space (here-after, referred to as GPROFIT). A review of measures of store performance revealed that they are both widely used measures of store perfor-mance (Lusch, 1986) and are frequently reported in retail trade publications and operating result studies by retail trade associations (National Retail Hardware Association, 1992). Each store also has target outcomes for annual sales and gross pro®t. For analysis purpose, knowledge of target outcome achievement was coded as `0' for target sucess and `1' for target failure.
2.4. Determinants of segment outcomes
2.4.1. Environmental factors
We identi®ed four uncontrollable environmental factors which in¯uence segment outcomes. These factors represent both supply and demand related factors. Market matching is the extent to which the demographics of the trade area match the desired target market. Thus, one would expect that a higher degree of market matching leads to higher economic performance of a store. The market matching concept is very similar to the concept of environment-strategy coalignment in the strategic management literature (Venkatraman & Prescott, 1990). In this study, market matching was indicated by the percentage of Hispanics (E1)
in the trade area population since Hispanics were the primary target market of the stores. Relative store size of No. 1 competitor (E2) was measured
by dividing the square footage of the largest trade area competitor by the square footage of our sample store. Hu's (1964) retail patronage and trade area model prominently recognizes the importance of the relative size of a retail store or center in attracting patrons. The higher the rela-tive store size of the No. 1 competitor, then the lower the economic performance of the store. Store saturation (E3) is another major environmental
variable in¯uencing store performance and it is measured by the number of stores in the retailer's
trade area per thousand population in the trade area (Ingene & Lusch, 1981). The economic per-formance of the store is negatively correlated with store saturation. Finally the economic health of the trade area (E4) is measured on a seven-point
scale, measuring whether it is a good time to open a store in the trade area. Responses could range from ``de®nitely not''(1) to ``de®nitely yes''(7). If it is believed that opening an addi-tional store would be a good decision, then the economic health of the trade area is likely to be good. One would expect that the better the eco-nomic health of the trade area, the higher the economic performance of the store.
2.4.2. Central management factors
For the retail chain organization under study, the ®rst central management factor is store size (C1), measured by the total square footage of the
store. The second factor, size of market (C2), is
indicated by whether the store was located in a metropolitan statistical area (MSA) or non-metropolitan environment (i.e. rural town). If a store was located in a nonmetropolitan market, then it was coded a ``1'' and ``0'' otherwise. Size of market was important because the chain stores in this retail organization tended to operate better in nonmetropolitan environments where fewer com-petitors are located. Both store size as well as the location is decided by the central management and hence are uncontrollable by the store manager.
2.4.3. Store manager factors
The store manager has control on four impor-tant decision factors Ð inventory, advertising, customer service capacity and pricing. Inventory intensity (D1) is measured by inventory investment
per square foot of selling area. This is a frequently mentioned determinant of store performance (Lusch, Dunne & Gebhardt, 1993). The higher the inventory intensity, the higher the economic per-formance of the store. Advertising intensity (D2) is
measured as annual advertising expenditures per square foot of selling area. The higher the adver-tising intensity, the higher the economic perfor-mance of the store. Customer service capacity (D3) is
customer service capacity (Ingene & Lusch, 1981). However, it should be noted that a high value on this measure (i.e. a high number of square feet of selling space per employee) is an indicator of low service capacity while a low value (i.e. relatively few square feet of selling space per employee) is a measure of high service capacity. The impact of customer service capacity on the economic perfor-mance of the store probably depends on the type of store and line of retail trade. For the retail chain organization under study, stores are posi-tioned as neither self service nor full service but somewhere between these extremes. Thus, some level of customer service capacity is important. Consequently, one would expect the economic performance of the store to be positively in¯u-enced by customer service capacity. Finally, store performance is aected by the gross margin per-cent (D4) which Lusch and Moon (1984) point out
to be a critical decision variable in retailing. Since all store managers are able to order merchandise as they see ®t from the chain's distribution center, they can determine the appropriate mix of mer-chandise and the initial storewide markup. The store managers, by ordering the ``right'' quantity and mix of merchandise, also in¯uence the need for markdowns and determine the timing of markdowns. As a consequence, store managers have a signi®cant in¯uence on the realized gross margins for the store. The eect of gross margin on store economic performance could be either positive or negative, depending on the price elas-ticity of demand.
2.5. Subjective performance evaluation of segment manager
The measure of store manager performance was operationalized by summing the subjective rat-ings [R] on eight questions intended to appraise managerial performance (both decision quality and eort). The eight items re¯ect the supervisor's satisfaction with the manager's job performance on the following attributes: managerial skills, taking responsibility, level of motivation, decision making ability, knowledge of trade area, tolerance for pressure, relations with supervisors, and manage-rial potential. The district supervisor rated each
manager on each attribute on a seven-point Likert type scale (where 1=``very dissatis®ed'' and 7=``very satis®ed''). We summed the eight ratings to obtain an index with higher scores representing a higher evaluation of the store manager's overall performance by the supervisor (the scores could range from 8 to 56). Factor analysis of the eight items provides support for this approach. It revealed that they all highly loaded on one factor with an eigenvalue of 5.74 and which explained 71.8% of the standardized co-variances among the eight items.
3. Statistical analysis
Table 1 shows the descriptive statistics and the correlation matrix of all the research variables.
3.1. Hypotheses testing of H1A and H1B
To provide a statistical test for the two hypoth-eses requires several steps. To begin with, we model the outcome determinants of both SALES and GPROFIT. We propose the following regres-sion model for operationalizing the determinants for both SALES and GPROFIT, the two measures of a store's economic outcome:
SorGPA1A2 E1 A3 E2 A4 E3
A5 E4 A6 C1 A7 C2 A8 D1
A9 D2 A10 D3 A11 D4 e
1
where:
S = SALES
GP = GPROFIT
E1 = market matching No.
E2 = relative size of 1 competitor
E3 = store saturation
E4 = economic health of trade area
C1 = store size in square feet
C2 = size of market
D1 = inventory intensity
D2 = advertising intensity
D3 = customer service level
e = error term which captures unmeasured or omitted variables and random eects
or uncertainty
A1ÿA11 = regression parameters to be estimated.
In Eq. (1), the E variables are uncontrollable environmental variables, the C variables are uncontrollable central management factors, and
the D variables are controllable store manager's decision variables. The error term includes the eect of forcing a linear relation between the variables as well as the eect of all omitted and unmeasured variables and random eects.
Regression equations were estimated separately for SALES and GPROFIT. Multicollinearity was not considered a signi®cant problem since the variance in¯ation factors (VIFs) for all variables were low (i.e. they ranged from 1.15 to 2.28 for Table 1
Descriptive statistics of the research variablesa
Panel A: Descriptive statistics
(S) (GP) (E1) (E2) (E3) (E4) (C1) (C2) (D1) (D2) (D3) (D4) (R) TS TGP Mean 62.83 7.91 32.67 3.65 3.05 3.06 11606 0.52 13.86 0.80 1098.5 3751.2 41.9 0.18 0.21 SD 23.65 9.63 29.50 3.32 5.12 2.32 4681 0.50 3.20 0.41 340 339 8.7 0.39 0.41
Panel B: Correlation coecients (p-values) [Note: Pearson, except Spearman for (C3) (Ts) and (TGP)]
(S) 1.00 0.98 0.29 0.07 ÿ0.05 ÿ0.36 ÿ0.11 ÿ0.13 0.48 0.47 ÿ0.69 0.25 0.33 ÿ0.24 ÿ0.18 (0.00) (0.00) (0.34) (0.44) (0.00) (0.12) (0.07) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00)
(GP) 1.00 0.30 ÿ0.06 ÿ0.04 ÿ0.36 ÿ0.14 0.15 0.43 0.42 ÿ0.69 0.43 0.37 ÿ0.28 ÿ0.24 (0.00) (0.43) (0.56) (0.00) (0.04) (0.03) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00)
(E1) 1.00 ÿ0.03 ÿ0.22 ÿ0.03 0.16 ÿ0.15 0.15 ÿ0.12 ÿ0.20 0.16 0.19 ÿ0.11 ÿ0.11 (0.63) (0.00) (0.70) (0.02) (0.05) (0.03) (0.10) (0.00) (0.02) (0.01) (0.02) (0.10) (E2) 1.00 ÿ0.02 0.13 ÿ0.26 ÿ0.25 0.09 0.15 ÿ0.11 ÿ0.05 ÿ0.15 0.04 0.006 (0.79) (0.05) (0.00) (0.00) (0.20) (0.03) (0.11) (0.48) (0.03) (0.33) (0.30) (E3) 1.00 ÿ0.15 0.03 ÿ0.20 0.03 ÿ0.06 0.13 0.04 ÿ0.05 ÿ0.04 ÿ0.02
(0.03) (0.71) (0.00) (0.68) (0.42) (0.07) (0.55) (0.49) (0.45) (0.68) (E4) 1.00 ÿ0.18 0.13 0.22 0.11 ÿ0.35 0.13 0.11 ÿ0.17 ÿ0.16
(0.00) (0.05) (0.00) (0.11) (0.00) (0.06) (0.13) (0.05) (0.00)
(C1) 1.00 ÿ0.22 ÿ0.35 0.08 0.46 ÿ0.18 0.20 ÿ0.01 ÿ0.02
(0.00) (0.00) (0.28) (0.00) (0.01) (0.00) (0.96) (0.84)
(C2) 1.00 0.15 0.25 ÿ0.07 0.08 ÿ0.03 ÿ0.01 ÿ0.01
(0.03) (0.00) (0.27) (0.24) (0.58) (0.88) (0.88)
(D1) 1.00 0.35 0.53 ÿ0.07 ÿ0.05 ÿ0.02 0.02
(0.00) (0.00) (0.29) (0.49) (0.66) (0.81) (D2) 1.00 ÿ0.37 ÿ0.05 0.05 ÿ0.03 0.06
(0.00) (0.50) (0.43) (0.63) (0.39)
(D3) 1.00 ÿ0.15 ÿ0.11 0.16 0.14
(0.03) (0.13) (0.10) (0.08)
(D4) 1.00 0.31 ÿ0.24 ÿ0.33
(0.00) (0.00) (0.00)
(R) 1.00 ÿ0.56 ÿ0.52
(0.00) (0.00)
(TS) 1.00 0.69
(0.00)
(TGP) 1.00
both SALES and for GPROFIT).3 Table 2
pre-sents the results separately for SALES and GPROFIT; columns 3±5 pertains to SALES and 6±8 pertains to GPROFIT. Overall, models for both SALES (F=40.63,p=0.0001,R2=O.66) and
GPROFIT (F=46.16, p=0.0001, R2=0.69) are
statistically signi®cant. An examination of the predictor variables for SALES indicates that, with
the exception of store saturation (E3), all of the
variables are statistically signi®cant (p40.05) with the expected signs on the coecients.4 For
GPROFIT, store saturation (E3) also is not
sig-ni®cant at the customary 0.05 level.
In the above step we identi®ed variables which signi®cantly aected SALES and GPROFIT. Next, we separately examine whether these variables
Table 2
Regression results for factors aecting store outcomea
S=GPA1A2 E1 A3 E2 A4 E3 A5 E4 A6 C1 A7 C2 A8 D1
A9 D2 A10 D3 A11 D4 e
T2:1
Independant variable Predicted sign S=dependant variable GP=dependant variable
Coecient T-stat. P-valueb Coecient T-stat. P-valueb
Intercept 0 ÿ6.723 ÿ0.43 0.6771 ÿ22.248 ÿ3.61 0.0004
E1 + 0.131 3.43 0.0004 0.053 3.59 0.0002
E2 ÿ ÿ0.839 ÿ2.56 0.0057 ÿ0.322 ÿ2.52 0.0063
E3 ÿ 0.362 1.75 0.0810 0.142 1.77 0.0790
E4 + 1.419 3.10 0.0011 0.548 3.08 0.0012
C1 + 0.001 3.36 0.0005 0.001 3.20 0.0008
C2 + 5.480 2.36 0.0097 1.999 2.21 0.0142
D1 + 1.614 4.02 0.0001 0.619 3.96 0.0001
D2 + 13.650 4.75 0.0001 4.701 4.20 0.0001
D3c ÿ ÿ0.032 ÿ7.54 0.0001 ÿ0.012 ÿ7.34 0.0001
D4 + 0.014 4.46 0.0001 0.010 8.79 0.0001
AdjustedR2 0.661 0.690
ModelF-value 40.627 46.155
ModelP-value 0.0001 0.0001
a S=annual sales per square foot of selling space or SALES;GP=annual pro®t per square foot of selling space or GPROFIT;
E1=market matching;E2=relative size of No. 1 competitor;E2=store stauration;E4=economic health of trade area;C1=store size in square feet;C2=size of market;D1=inventory intensity;D2=advertising intensity;D3=customer service level;D4=gross margin percentage.
b All tests are one-tail tests except for gross margin percent (D4) and store saturation (E3).
c Customer service is measured by square feet of selling area per fulltime equivalent employee. Therefore a high value represents low customer serice and a low value represents high customer service levels.
3 The variance in¯ation factor is the reciprocal of 1ÿR2when a particular independent variable is regressed on all other independent variables See Belsley, Kuh and Welsch (1980) for a discussion of this diagnostic approach.
4 The coecient of the store saturation (or E3) variable, although not statistically signi®cant, is unexpectedly positive. We expected that as the market becomes more saturated, sales per square foot would decline. The president of the retail chain was, therefore, consulted about this ®nding. His explanation was that in highly saturated markets the stores tended to be viewed as more convenient and accessible since they were relatively small
which aected SALES and GPROFIT also aect the subjective performance evaluation of store managers. To do this we must estimate how much in SALES and GPROFIT is due to the four broad categories of determinants of store outcomes which, as discussed previously, were: (1) environ-mental factors, (2) central management factors, (3) store manager factors, and (4) all other unmea-sured or omitted variables and random eects. If one takes the information incorporated in the regression equations presented in Table 2, one can decompose the sources of SALES or GPROFIT outputs into four components as follows:PE =pre-dicted outcomes due to uncontrollable environ-mental factors. This value is obtained by considering only the signi®cant factors from separate regressions for SALES and GPROFIT in Table 2. Thus, for example, from Table 2 for SALES:PE=0.131(E1or
market matching)ÿ0.839 (E2or relative size of No.
1 competitor)+1.419 (E4 or economic health of
trade area).
For GPROFIT, since the same environmental variables were also signi®cant, we have the fol-lowing from Table 2:
PE=0.053(E1 or market matching) ÿ0.322(E2
or relative size of No. 1 competitor) +0.548(E4or
economic health of trade area).
PC=the predicted outcomes due to uncontrol-lable central management factors. For SALES, this value is obtained from regression for SALES in Table 2:
PC=0.001(C1or store size) +5.480(C2or size of
market).
A similar procedure was also conducted for GPROFIT to obtain:
PC=0.001(C1or store size)+1.999(C2or size of
market).
PD=the predicted outcomes due to decisions the store manager controls. For SALES, this value is obtained from regression equation in Table 2:
PD=1.614(D1or inventory intensity)+13.650(D2
or advertising intensity)ÿ0.032(D3or customer
ser-vice level)+0.014(D4or gross margin percentage).
A similar procedure was also conducted for GPROFIT.
PD=0.619(D1or inventory intensity)+4.701(D2
or advertising intensity)ÿ0.012(D3or customer
ser-vice level) +0.010(D4or gross margin percentage).
PU=outcomes due to all omitted and unmea-sured determinants including random eects; this is estimated by e, the residual term from each of the equations in Table 2. This residual term (e) is the predicted store outcomes less the actual store outcomes based on the regression equations pre-sented in Table 2.
We can now proceed to suggest a test of H1A and H1B. Consider the following regression equation:
RBoB1 PE B2 PC B3 PD B4 PU
B5Te
2
where:
R = the supervisor's subjective evaluation of the store manager's performance;
PE, PC,
PD, PU= the decomposed outcome (SALES or
GPROFIT) due to dierent sources as discussed above;
T = achievement of target outcome for either SALES or GPRORFIT (coded as dummy variable; ``1'' for failure to meet the target and ``0'' otherwise);
e = error term;
B1ÿB5 = regression parameters to be estimated.
In Table 3 the statistical results for this regres-sion equation are provided separately for both SALES and GPROFIT. Multicollinearity is not considered a signi®cant problem (the VIFs ranged from 1.03 to 1.20 for SALES and 1.01 to 1.31 for GPROFIT). Columns 2±4 present the results for the SALES equation and columns 5±7 present the results for the GPROFIT equation.
decision factors rise or fall by one dollar the man-ager's performance rating rises or falls by 0.06 (0.20) points.
Hypothesis 1A also predicted that the manager's performance will not be aected by central man-agement factors since they are uncontrollable at the store manager's level. In Table 3 we do not ®nd support for this prediction for either SALES and GPROFIT output. For example, when SALES attributable to central management fac-tors (PC) rises or falls by one dollar the store manager gains or loses 0.41 rating points, as the case may be, which is signi®cant at p=0.0003. Similarly, when GPROFIT attributable to central management factors (PC) changes by one dollar the store manager's evaluation also changes by 1.26, which is signi®cant atp=0.0001.
Although we did not formally state any hypotheses about the impact on SALES or GPROFIT due to unmeasured, omitted, or ran-dom eects on the store manager's evaluation, it is interesting to examine these results. Examining these results we see that for SALES and GPRO-FIT, the manager receives a higher rating as these outputs rise (PU), and vice-versa. For example, as SALES due to unmeasured, omitted, and random
eects rise by one dollar the manager receives 0.09 rating points and when GPROFIT rises by one dollar due to these eects the manager receives 0.30 rating points.
As hypothesized in H1B, target outcome knowledge or whether or not SALES or GPRO-FIT has been achieved signi®cantly in¯uences the manager's evaluation. This illustrates the outcome eect is stubborn. For failure to meet the target sales the manager is penalized 11.56 points (p=0.0001) and for failure to meet the gross pro®t target the penalty is 10.04 points (p=0.0001).
Discussion
This ®eld study provides evidence of the out-come eect in performance evaluations. Speci®-cally, for the retail chain studied, we found that SALES and GPROFIT under the control of the manager in¯uences the performance evaluation of that manager as expected, and that SALES and GPROFIT due to uncontrollable environmental factors do not in¯uence the manager's perfor-mance evaluation, also as expected. However, inconsistent with our expectations, the SALES Table 3
Regression results for performance evaluation as a function of controllability of output factors and target outcome knowledgea
RB0B1 PE B2 PC B3 PD B4 PU B5Te T3:1
Independant variable Outcome measure=SALES Outcome measure=GPROFIT
Coecient T-stat. P-value Coecient T-stat. P-value
Intercept 34.98 14.25 0.0001 29.42 8.12 0.0001
PE 0.12 1.24 0.2154 0.26 1.12 0.2629
PC 0.41 3.65 0.0003 1.26 4.10 0.0001
PD 0.06 1.69 0.0501 0.20 2.78 0.0060
PU 0.09 2.47 0.0145 0.30 3.25 0.0014
T ÿ11.56 ÿ8.91 0.0001 ÿ10.04 ÿ8.16 0.0001
AdjustedR2 0.39 0.38
ModelF-value 26.98 25.81
ModelP-value 0.0001 0.0001
a Variable de®nitions are as follows:R=the supervisor's valuation of the store manager's performance,PE=predicted economic outcomes due to environmental or uncontrollable factors,PC=predicted economic outcome due to conditionally controllable factors,
AND GPRPOFIT due to uncontrollable central management factors do in¯uence the manager's evaluation. At ®rst glance one would tend to con-sider this result as an anomaly since the store manager does not decide either the store location or the size of a store but is assigned to a store location and/or store size facility. However, as one might expect, managers are not assigned to stores randomly but based on obtaining a good ®t between the store and the manager. When central management decides which manager to assign to which outlet, it is logical to assume that the assignment is conditional on the manager meeting some criteria (see e.g. Libby & Frederick, 1990), such as performance or experience, or else the manager would not be in charge of that particular store to begin with. Whatever that criteria may be, the key point is the store assignments are not ran-dom. These two evaluation factors (i.e. location and store size) have some information content about the manager's prior ability to positively in¯uence the probability distribution of store out-comes. Each of these evaluation factors is a source of information. What matters is the information content of the factors and whether or not the store manager can in¯uence or control the information content of these factors, as opposed to the factors per se. Intuitively, store managers do exercise such an in¯uence, which is why location and store size are, perhaps, signi®cant explanatory variables of managers' subjective performance evaluation. Incidentally, when the information content of a factor can be controlled or in¯uenced, as opposed to the factor per se, it is the same as saying the factor is conditionally controllable since informa-tion content is linked to condiinforma-tional controllability (Antle & Demski, 1988; Demski, 1994). Thus, store location and size may be considered as con-ditionally controllable factors by the store manager which may explain why they were given credit in the evaluation of these factors.
Despite the preceding ®ndings this study did ®nd strong evidence of an outcome eect when target outcome knowledge was taken into con-sideration as well as the controllable and uncontrollable determinants of segment outcome. Since this ®nding is consistent with prior results from experimental studies, the robustness of the
outcome eect in subjective performance evalua-tions is evident.
In general, this study empirically demonstrated that the outcome eect is prevalent in an industry setting and is robust across the measure of out-comes. Kinney (1969) suggests that dierences in performance of multi-unit businesses can be due to dierences in the environmental factors con-fronting the multi-unit businesses and dierences in the performance of the outlet managers (i.e. their managerial decision making ability). He sug-gests ``the eects of these environmental factors should be extracted before evaluating the perfor-mance of the manager'' (p. 44) because if the per-formance rating of the manager is in¯uenced by uncontrollable environmental factors then the performance measurement and evaluation system is ¯awed. Further, the manager should be only held responsible for factors that are controllable by him or her. This research found that the multi-unit retail chain studied performed well in this regard. It was neither rewarding nor penalizing its managers for environmental factors that in¯uence store performance but took into consideration the controllable factors. Also, it was giving the man-agers credit for unmeasured, random, or omitted variables. Hershey and Baron (1992) argue that while in an experiment the probability of various outcomes and other relevant information aecting outcome can be fully conveyed to the judge, ``in more realistic cases, it is likely that the decision maker would have information relevant to the probability judgement that the judge cannot have''(p. 91). Some of the omitted variables could represent controllable factors we did not measure and are hard for the supervisors (judge) to have complete knowledge about (i.e. day to day opera-tions such as cleanliness of store, employee rela-tions, etc.) or would be dicult for the supervisor to know precisely. In other words, it is indeed possible the store manager has some in¯uence over outcomes but the evaluator does not know the nature or the extent of that in¯uence. In such situations, it is reasonable to give some credit to the segment manager for these outcomes.
serious weakness of a ®eld study is its ex post facto character. Thus, statements of relations are weaker than they are in experimental research (Kerlinger, 1986). To complicate matters further, the ®eld situation almost always has a plethora of variables. In an experiment, these variables can be largely controlled, but in a ®eld study, they cannot be strictly controlled. However, they could be partially controlled if data on possible covariates is collected. Although this would still not represent the strict internal control of an experiment, it would be a step in the direction of greater control of extraneous factors. Another methodological weakness is the problem of precision in the mea-surement of ®eld variables. In ®eld studies, this problem is of course more acute than in experi-ments due to the greater complexity of ®eld set-tings. In addition to the general problems associated with ®eld study there are at least two which are more relevant for the current study. The research setting was in a retail chain. Therefore, identi®cation of the controllable/uncontrollable factors in this study was industry-speci®c which limits the generalizability of the results. However, the research site is representative of other multi-unit organizations. The second limitation pertains to the data availability. The measures were con-structed based on the data available and therefore an opportunity exists to develop better measures.
Future research should persevere in its endeavor to further investigate the subtleties of outcome eect and performance evaluation. Four possible avenues of research are suggested. First, research is needed on whether communicating the decision process, as well as the decision outcome, to the evaluator mitigates the outcome eect. For exam-ple, in addition to reporting the material price variance, management accountants may also report the assumptions that the DM had to make in order to estimate the initial budgeted price Ð qual-ity of raw material, consumption patterns, market demands, supply sources, etc. This information would allow the evaluator to assess the validity of the critical assumptions and the uncertainty faced by the DM in budgeting the materials price. Sec-ond, one might also examine whether an educa-tion in debiasing strategies during training programs for supervisors reduces the outcome
eects. Such an education would describe or warn the supervisors (see Fischho, 1982) of this potential problem in their evaluations of sub-ordinates and may help to overcome, at least to some extent, their biased judgments. Third, one could assess if conducting the subjective perfor-mance of a manager either before or at various time lags (e.g. 1, 2, 4 or 6 weeks) after the objective target outcome knowledge on segment perfor-mance becomes available mitigates the outcome eect. Finally, since this research developed a ®eld methodology for isolating outcome eects, we believe it would be prudent to study how various organizational factors or industry factors might in¯uence the level of outcome eect.
This ®rst attempt at uncovering outcome eects in a ®eld study may encourage others to explore some of the preceding questions and also open the way for studies in other industry settings. This seems a natural step for building on the long tra-dition of high quality laboratory research that has been conducted which has addressed many issues in accounting and managerial decision making.
Acknowledgements
We would like to thank Margaret Boldt, Steve Butler, Uday Chandra, the two anonymous review-ers, and the workshop participants at the University of Oklahoma and Michigan State University for their comments on earlier versions of the paper.
References
Abdel-khalik, R. A., & Ajinkya, B. B. (1979). Empirical research in accounting (Vol. 4). Sarasota, FL: American Accounting Association.
Antle, R., & Demski, J.S. The controllability principle in responsibility accounting(Vol. 58, pp. 700±718).
Baron, J., & Hershey, J. C. (1988). Outcome bias in decision evaluation.Journal of Personality and Social Psychology, 54, 569±579.
Belsley, D. A., Kuh, E., & Welsch, R. E. (1980).Regression diagnostics. New York: John Wiley and Sons.
Borman, W.C. Job behavior, performance, and eectiveness. In M.D. Dunnette & L.M. Hough,Handbook of industrial and organizational psychology. Palo Alto, CA: Consulting Psy-chologists Press.
Brown, C. E., & Solomon, I. (1987). Eects of outcome infor-mation on evaluations of managerial decisions. The Accounting Review, 62, 564±577.
Brown, Solomon (1993). An experimental investigation of expla-nations for outcome eects on appraisals of capital-budgeting decisions.Contemporary Accounting Research, 10, 83±111. Choo, F. (1989). Expert±novice dierences in judgement/decision
making research.Journal of Accounting Literature, 8, 106±136. Davis, R. (1982). Expert systems: where are we, and where are
we going from here?The AI Magazine, 3±22.
Demski, J. (1994).Managerial uses of accounting information. Norwell, MA: Kluwer Academic Publishers.
Edwards, W. (1984). How to make good decisions. Selected Pro-ceedings of the 9th research conference on subjective prob-ability, utility and decision making.Acta Psychologica, 56, 5±27. Fischo, B. (1975). Hindsight6foresight: the eect of outcome knowledge on judgment under uncertainty. Journal of Experimental Psychology: Human Perception and Perfor-mance, 1, 288±299.
Fischho, B. (1982). Debiasing. In D. Kahneman, P. Slovic, & A. Tversky,Judgment under uncertainty: heuristic and biases
(pp. 422±444). Cambridge: Cambridge University Press. Fisher, J., & Selling, T. I. (1993). The outcome eect in
per-formance evaluation: decision process observability and consensus.Behavioral Research in Accounting, 5, 58±77. Guion, R.M.,Personnel testing. New York: McGraw-Hill, 1965. Hawkins, S. A., & Hastie, R. (1990). Hindsight: biased
judg-ments of past events after the outcomes are known. Psycho-logical Bulletin, 107, 311±327.
Hershey, J. C., & Baron, J. (1992). Judgment by outcomes: when is it justi®ed? Organizational Behavior and Human Decision Processes, 53, 89±93.
Hershey, J. C., & Baron, J. (1995). Judgement by outcomes: when is it warranted? Organizational Behavior and Human Decision Processes, 62, 127.
Holmstrom, B. (1979). Moral hazard and observability. Bell Journal of Economics, 10, 74±91.
Horngren, C. T., Foster, G., & Datar, S. M. (1997). Cost Accounting(9th ed.). Englewood, NJ: Prentice-Hall. Hu, D. L. (1964). De®ning and estimating a trade area.
Jour-nal of Marketing, 28, 34±38.
Ingene, C. A., & Lusch, R. F. (1981). A Model of Retail Structure. Research in Marketing, 5,101±164.
Kelley, H. H., & Michela, J. L. (1980). Attribution theory and research.Annual Review of Psychology, 31, 457±501. Kerlinger, F. N. (1986). Foundations of behavioral research.
Fort Worth, TX: Holt, Rinehart and Winston.
Kinney, W.R. An environmental model for performance mea-surement in multi-outlet businesses Journal of Accounting Research(Spring, 1969) pp. 44-52.
Libby, R., & Frederick, D. M. (1990). Experience and the ability to explain audit ®ndings. Journal of Accounting Research, 28, 348±367.
Lipe, M. G. (1993). Analyzing the variance investigation deci-sion: the eects of outcomes, mental accounting, and fram-ing.The Accounting Review, 68, 748±764.
Lipshitz, R. (1989). Either a medal or corporal: the eects of success and failure on the evaluation of decision making and decision makers.Organizational Behavior and Human Deci-sion Processes, 44, 380±395.
Lusch, R. F., Dunne, P., & Gebhardt, R. (1993).Retail mar-keting. Cincinnati, OH: South-Western Pub. Co.
Lusch, R. F., & Moon, S. Y. (1984). An exploratory analysis of the correlates of labor productivity in retailing. Journal of Retailing, 60, 37±61.
Lusch, R. L. (1986). The new algebra of high performance retail management.Retail Control, 54, 15±35.
Mitchell, T. R., & Kalb, L. S. (1981). Eects of outcome knowledge and outcome valence on supervisors' evaluations.
Journal of Applied Psychology, 66, 604±612.
National Retail Hardware Association (1992). Management report: detailed cost-of-doing business data for hardware stores. Indianapolis, IN: National Retail Hardware Associa-tion.
Simon, H. A. (1957).Models of man. New York: Wiley. Slovic, P., & Fischho, B. (1977). On the psychology of
experimental surprises.Journal of Experimental Psychology: Human Perception and Performance, 3, 544±551.
Tan, H., & Lipe, M. G. (1997). Outcome eects: the impact of decision process and outcome controllability. Journal of Behavioral Decision Making, 10, 315±325.
Tversky, A., & Kahneman, D. (1986). Rational choice and the framing of decisions.Journal of Business, 59, 251±294. Venkatraman, N., & Prescott, J. E. (1990).
