Column replacement algorithms and under-determined linear systems

Zhengbo Zhu, Purdue University

Abstract

In this dissertation, we look at the under-determined systems of linear equations with sparse solutions. Our motivation comes from a field called compressed sensing that has seen an unprecedented expansion in recent years. A rather wide span of possible applications of compressed sensing has been considered in the literature during the last decade. They include image reconstruction, single-pixel camera design, the decoding of linear codes, channel estimation in wireless communications to machine learning, DNA microarrays, magneto-encephalography, and many others that can be found in, e.g. [1-7]. The field seems to be developing at a fairly fast pace and, as mentioned above, in this dissertation, we look at several interesting problems derived from it. It is by now fairly well known that the core mathematical problem of compressed sensing reduces to solving large under-determined systems of linear equations. Of course, such systems typically would have a large number of solutions and would not really be exactly (uniquely) solvable. If, on other hand, one additionally assumes that the solution vectors are sparse, then matters become completely different. Depending on the systems' dimensions and the underlying sparsity, the systems may in fact be solvable. Not only that, but if elements of the system matrix (typically in the compressed sensing literature called the measurement matrix) are chosen according to some appropriate probability distributions and if the solution vector (signal) is sparse enough, then a certain polynomial technique called ℓ1-minimization can recover it with overwhelming probability (see, e.g. [3, 8, 9]). In fact, [3, 8, 9] established (in a statistical context) that, if the number of measurements is proportional to the length of the signal, then there is a sparsity of the unknown signal also proportional to its length for which the success of ℓ 1-minimization is guaranteed with overwhelming probability. This result singlehandedly skyrocketed the popularity of the compressed sensing and established it as one of the main research arenas in various branches of mathematics and engineering over the last decade. Since the appearance of pioneering work [3, 8, 9], many powerful results have been developed within the compressed sensing arena as well as in various applications outside of it. In addition to the aforementioned linear under-determined systems, their various alternatives eventually emerged as both practically and theoretically important research problems. Some of these alternatives will be the subject of our attention in this thesis as well. As a short preview, we will mention the two probably the most outstanding ones. The first one relates to the so-called noisy systems. Namely, the linear measurements (mathematically, effectively the equations of the system) may be not only incomplete but also imprecise/corrupted. Moreover, the ideal solution sparseness (which is essentially the core idea behind the entire compressed sensing concept) may not be there, either. If the deviations of the noisy systems from the ideal (non-noisy) ones are not substantial, one would expect that small refinements of the standard ℓ1 would produce results comparable to those, one typically obtains when employing the standard ℓ1 itself in the ideal systems. This indeed turns out to be the case and effectively implies what is typically called in the compressed sensing literature the ℓ1's stability (see, e.g. [8, 10]). The second alternative relates to the existence of systems solutions with special structures that go beyond the standard sparsity one. Of course, such existence is typically only useful if one is aware of it before attempting to solve the systems. While at first glance, it may appear artificial, as in numerous applications of compressed sensing, it actually does happen that one is a priori aware of the special structure that goes beyond the sparsity. Typical examples include block and binary structures (see, e.g. [11-16]). As addressed above, one would then expect that appropriate adjustments to the ℓ1 would be possible so that one could conceivably incorporate the a priori available knowledge of the signals structure. This again does turn out to be the case, and given that one has more information about the unknown solution of the system that is a priori available, the recovery abilities of the adapted algorithms substantially supersede the standard ℓ 1 (see, e.g. [14-16]). In this dissertation, we will first consider a couple of algorithms that are conceptually different from the ℓ1 benchmark and make several observations related to their performances when used for solving under-determined linear systems. We will first focus on the ideal scenarios and then switch to noisy systems and systems with structured solutions. We will present a thorough discussion related to the implementations, complexities, and various other features of these algorithms. Possible applications of these algorithms in other areas that are to some extent also related to sparse signal recovery will be discussed as well.

Degree

Ph.D.

Advisors

Stojnic, Purdue University.

Subject Area

Applied Mathematics|Electrical engineering|Operations research

COinS