A Comparative Analysis of the Empirical Validity of Two Rule-Based Belief Languages

Rule-based expert systems deal with inexact reasoning through a variety of quasi-probabilistic methods, including the widely used subjective Bayesian (SB) and certainty factors (CF) models, versions of which are implemented in many commercial expert system shells. Previous research established that under certain independence assumptions, SB and CF are ordinally compatible : when used to compute the beliefs in several hypotheses of interest under the same set of circumstances, the hypothesis that will attain the highest posterior probability will also attain the highest certainty factor, etc. This monotonicity is important in the context of expert systems, where most inference-engines and explanation facilities are designed to utilize relative scales of posterior beliefs, making little or no use of their absolute magnitudes. This research extends the comparative analysis of SB and CF to the field, where subjective degrees of belief and different elicitation procedures are likely to complicate their analytic similarity and impact their actual validity. In particular, we describe an experiment in which CF was shown to dominate SB in terms of several validity criteria, a finding which we attribute to parsimony and robustness considerations. The paper is relevant to (i) practitioners who use belief languages in rule-based systems, and (ii) researchers who seek a methodology to investigate the validity of other belief languages in controlled experiments.