| Abstract: | There have been two dominant paradigms in understanding and modeling the expert's decision-making behavior: output analysis and process-tracing. While the two paradigms are complementary, they have not been used as of yet in a combined manner. The purpose of this study was to combine and extend previous research work in the two paradigms to expert systems research by 1) analyzing the expert's decision strategies in a financial judgment situation, 2) comparing performance of three popular inductive modeling methods, and 3) matching their performance against the type of decision strategy used. An important element of this study was consideration of the properties of modeling situation as a contingent factor in comparing model performance. By considering the contingent factor, a more adequate evaluation of decision models can be achieved. Model performance was compared within the same task with the same data, but across three different modeling situations. Dimensions considered here included the predictive accuracy of experts, linear versus nonlinear processing strategies, validity of the strategies, and model performance. Analysis of decision strategies indicates that decision behaviors in a financial judgment situation is contingent upon the risk level of the task faced by expert decision makers. The results showed significant matches between linear strategies and linear modeling method, and between nonlinear strategies and a machine learning method. This study also found that simulation capability of machine learning methods did not contribute to improving the validity of models in predicting the environmental outcomes. This finding not only supports the need for considering the properties of modeling situation in evaluating performance of decision models, but also provides a plausible explanation for the good performance of linear models reported in numerous output analysis studies... |
|---|