Analyzing Thresholds and Efficiency with Hierarchical Bayesian Logistic Regression

Keywords

Ideal observer analysis; Bayesian data analysis

Abstract

Ideal observer analysis is a fundamental tool for analyzing the efficiency with which a cognitive or perceptual system uses available information using classification accuracy. The ideal observer performance is a formal measure of the amount of information in a given experiment. The ratio of ideal observer performance to human performance, i.e. efficiency, can be used to compare across experimental conditions while controlling for the differences due merely to the stimuli or task specific demands. In previous research using ideal observer analysis, the effects of varying experimental conditions on efficiency have been tested using ANOVAs and pair-wise comparisons. In this work, we present a model that combines Bayesian estimates of psychometric functions with hierarchical logistic regression for inference about both unadjusted human performance metrics and efficiencies. Our approach improves upon the existing methods by constraining the statistical analysis using a standard model connecting stimulus intensity to human observer accuracy and by accounting for variability in the estimates of individual and ideal observer performance scores. This allows for both individual and group level inferences giving a deeper understanding of the efficiency data.

Start Date

18-5-2017 11:38 AM

End Date

18-5-2017 12:00 PM

This document is currently not available here.

Share

COinS
 
May 18th, 11:38 AM May 18th, 12:00 PM

Analyzing Thresholds and Efficiency with Hierarchical Bayesian Logistic Regression

Ideal observer analysis is a fundamental tool for analyzing the efficiency with which a cognitive or perceptual system uses available information using classification accuracy. The ideal observer performance is a formal measure of the amount of information in a given experiment. The ratio of ideal observer performance to human performance, i.e. efficiency, can be used to compare across experimental conditions while controlling for the differences due merely to the stimuli or task specific demands. In previous research using ideal observer analysis, the effects of varying experimental conditions on efficiency have been tested using ANOVAs and pair-wise comparisons. In this work, we present a model that combines Bayesian estimates of psychometric functions with hierarchical logistic regression for inference about both unadjusted human performance metrics and efficiencies. Our approach improves upon the existing methods by constraining the statistical analysis using a standard model connecting stimulus intensity to human observer accuracy and by accounting for variability in the estimates of individual and ideal observer performance scores. This allows for both individual and group level inferences giving a deeper understanding of the efficiency data.