Modelling pairwise comparisons for Thurstonian scaling and Kendall rank correlation

Pairwise comparisons are a simple and effective way to measure the relative ordering of two samples. For more than two samples, a global ordering can be constructed and with sufficient data it is possible to construct a metric scale, for example by using Thurstonian scaling. We will discuss the concept of Number of Distinguishable Levels (NDLs) that emerges from a Thurstonian scaling procedure. The NDL is the attribute range in terms of Just Noticeable Differences (JNDs), for example the number of grayscale values. The NDL is either limited by the visual system or by the range of stimuli. The latter case is interesting as it can be used to characterise and compare image collections, e.g. from different periods in history. We have used this in experiments on the translucency in sea depictions, or fabric softness in paintings of the holy Mary.

A related but different case is when pairwise comparisons are used to quantify how well images represent reality, for example in product photos. We ran various material perception experiments in reality and online and compared their similarity by using Kendall rank correlations. To assess robustness we combined Thurstonian scaling and Kendall rank correlations using synthetic data from Monte Carlo simulations that allow for an estimation of confidence intervals.