Abstract
In this paper we propose a framework to quantitatively evaluate user awareness and the level of support that visual analytics decision support systems (VADS) provide. For the framework, which has a theoretical underpinning from the field of judgement analysis, we propose a model for VADS system. The framework bridges the gap between judgment analysis and VADS evaluation by conceptually connecting judgment analysis concepts to visual analytic. The proposed approach offers an insights based evaluation to measure the importance and the utility of the insights. We propose to model insights and user findings as random variables that parametrize user decisions. The mixed methodology used in our framework has the potential to study user decision process in real situations while producing results that can be generalized. Our contributions in this work appear in the modeling of VADS system and the evaluation framework we propose which quantifies situation awareness. Other advantages include evaluating collaboration and analyzing joint decisions. Some limitations of the framework are also discussed including the requirement of large testing data.
Keywords
Evaluation, Quantitative, Idiographic statistics, Judgment Analysis, Lens Model, Situation
Date of this Version
10-16-2017