TARP Benchmarks for Detection Problems
We propose the use of one-dimensional random projections to build benchmarks from the receiver-operating characteristic curves of detection algorithms. For our benchmarks, we extend the concept of "thresholding after random projection", originally introduced in the context of classification, to the detection framework. The benchmarks measure the inherent simplicity or difficulty of the detection problems under consideration and provide a criterion against which other methods can be compared. Our benchmarks are curves in the two-dimensional space parameterized by the area above the receiver-operating characteristic curve and by the computational cost. Each benchmark curve divides the plane into a positive gain and a negative gain region, with the benchmark curve itself representing zero gain. The positive gain region is further split into a structural gain region and a computational gain region. Other methods can be characterized based upon which region they lie in. Methods which lie on the benchmark curve perform comparably to a series of random projections. Methods which lie in the structural and computational gain regions outperform the benchmark and methods in the negative gain region underperform the benchmark; the methods with negative gain have poorer performance in a detection task than a series of random projections and thus are considered ill-suited for the given problem. Our numerical experiments illustrate each case using real data.
Boutin, Purdue University.
Off-Campus Purdue Users:
To access this dissertation, please log in to our