Accountability as a Tool for Improving Judgmental Forecasting? A Cognitive Dual-Process Approach
09 / 2006 - 09 / 2010
Although traditionally most research on forecasting has focused on developingquantitative models as decision aids, surveys indicate that judgmental forecasting isubiquitous (Armstrong, Brodie, & McIntyre, 1987; Kahn, 2002) and its significant rolehas now become generally acknowledged (Lawrence, Goodwin, O'Connor, & Onkal,2006). In an influential article, Blattberg and Hoch (1990) showed that a combination of astatistical model and managerial judgment outperformed either decision input in isolationin two forecasting situations. Especially in areas where there is a lack of data for modeldevelopment, a general uncertainty surrounding the marketplace and limited decisiontime, judgmental input is significant. The new product forecasting function provides acompelling example where judgmental forecasting plays a key role.Marketing departments are primarily responsible for the new product forecastingfunction. Indeed, marketing managers and product managers are charged with theresponsibility of identifying which new products and projects are worthwhile to pursueand which should be buried. Moreover, new product forecasts drive a variety ofmultifunctional decisions, such as marketing decisions on marketing budgets andpromotion schedules, finance decisions on corporate budgets and financial expectationsfor the new products and sales decisions on support materials and salespeople training.Given this breadth of decisions, company management is very interested in finding waysto improve the new product forecasting effort, and thereby minimize forecasting error(Kahn, 2002).Judgments can be based on intuition or analysis. It used to be commonplace towarn against judgments based on intuition. Indeed, a substantial stream of research hasindicated that superficial information search and processing biases can cause gross errorsin decision making (e.g., Kahneman, Slovic, & Tversky, 1982; Kirkpatrick & Epstein,1992; Schoemaker & Russo, 1993; Slovic, Fischoff, & Lichtenstein, 1977). One type ofjudgment error is strategy-based and occurs when a judge uses a suboptimal informationprocessing strategy (Arkes, 1991).The identification of judgment errors and biases initiated a research programexploring ways to improve suboptimal judgment behavior (i.e. debiasing techniques).3Making managers accountable for their decisions has been proposed as a valuabledebiasing technique since it moves the judgment or forecasting process away from anintuitive process and toward an analytical process (Stewart, 2001). Tetlock and Kim(1987), for example, found that making participants justify their judgment strategyresulted in more accurate forecasts and less overconfidence in a personality predictiontask.While most research in decision theory has focused on the inadequacies ofintuitive judgments, some research in the domain of social psychology (e.g., Dijksterhuis,2004; Wilson & LaFleur, 1995; Wilson & Schooler, 1991) and cognitive psychology(e.g., Berry & Broadbent, 1988; Reber, Kassin, Lewis, & Cantor, 1980) has demonstratedthe inadequacies of analytical judgments. More specifically, an analytical informationprocessing strategy can be detrimental for decision making in more complexenvironments where the key variables are not salient.In this paper, we reconsider accountability as a potential debiasing technique inthe context of a multiple-cue learning paradigm, where participants have to learn topredict a criterion on the basis of information about specific cues. We will propose twostudies. In a first study, we try to show that the commitment to justify one s decisionstrategy (i.e. procedural accountability), relative to a mere evaluation of the quality ofone s decision outcomes (i.e. outcome accountability), improves judgmental accuracy ina multiple-cue learning task where linear cue-criterion relations have to be learned.However, in a nonlinear task we expect that procedural accountability impairs judgmentalaccuracy, relative to outcome accountability. Thus, we hypothesize that the relativeeffects of procedural and outcome accountability on judgmental accuracy are dependenton specific task characteristics, such as cue-criterion relations. In a second study, we tryto show by using cognitive models that procedural accountability elicits an analytical,rule-based processing strategy while outcome accountability triggers an intuitive,exemplar-based processing strategy.This research is important in several ways. First, it adds to theorizing in currentforecasting literature where it is assumed that justification of decision procedures alwaysresults in improved forecasting accuracy. Indeed, this pioneering research indicatesspecific conditions under which procedural accountability might result in worse4judgmental accuracy than outcome accountability. Second, this paper extends previousinquiries in psychological literature investigating the consequences of specificaccountability types in terms of cognitive processes. It would be the very first researchdirectly assessing information processing strategies triggered by accountabilitymanipulations, based on cognitive models.