The Journal of Problem Solving

Perspectives on Problem Solving in Educational Assessment: Analytical, Interactive, and Collaborative Problem Solving

Samuel Greiff et al. — Thu, 18 Apr 2013 13:12:49 PDT

Problem solving has received broad public interest as an important competency in modern societies. In educational large-scale assessments paper-pencil based analytical problem solving was included first (e.g., Programme for International Student Assessment, PISA 2003). With growing interest in more complex situations, the focus has shifted to interactive problem solving (e.g., PISA 2012) requiring identification and control of complex systems. In the future, collaborative problem solving represents the next step in assessing problem solving ability (e.g., PISA 2015). This paper describes these different approaches to assessing problem solving ability in large-scale assessments considering theoretical questions as well as assessment issues. For each of the three types of problem solving, the definition and understanding of the construct is explained, items examples are shown together with some empirical results, and limitations of the respective approach are discussed. A final discussion centers on the connection of cognitive and differential psychology within educational research and assessment.

On Evaluating Human Problem Solving of Computationally Hard Problems

Sarah Carruthers et al. — Thu, 18 Apr 2013 13:12:45 PDT

This article is concerned with how computer science, and more exactly computational complexity theory, can inform cognitive science. In particular, we suggest factors to be taken into account when investigating how people deal with computational hardness. This discussion will address the two upper levels of Marr’s Level Theory: the computational level and the algorithmic level. Our reasons for believing that humans indeed deal with hard cognitive functions are threefold: (1) Several computationally hard functions are suggested in the literature, e.g., in the areas of visual search, visual perception and analogical reasoning, linguistic processing, and decision making. (2) People appear to be attracted to computationally hard recreational puzzles and games. Examples of hard puzzles include Sudoku, Minesweeper, and the 15-Puzzle. (3) A number of research articles in the area of human problem solving suggest that humans are capable of solving hard computational problems, like the Euclidean Traveling Salesperson Problem, quickly and near-optimally.

This article gives a brief introduction to some theories and foundations of complexity theory and motivates the use of computationally hard problems in human problem solving with a short survey of known results of human performance, a review of some computationally hard games and puzzles, and the connection between complexity theory and models of cognitive functions. We aim to illuminate the role that computer science, in particular complexity theory, can play in the study of human problem solving. Theoretical computer science can provide a wealth of interesting problems for human study, but it can also help to provide deep insight into these problems. In particular, we discuss the role that computer science can play when choosing computational problems for study and designing experiments to investigate human performance. Finally, we enumerate issues and pitfalls that can arise when choosing computationally hard problems as the subject of study, in turn motivating some interesting potential future lines of study. The pitfalls addressed include: choice of presentation and representation of problem instances, evaluation of problem comprehension, and the role of cognitive support in experiments. Our goal is not to exhaustively list all the ways in which these choices may impact experimental studies, but rather to provide a few simple examples in order to highlight possible pitfalls.

Effects of Cluster Location on Human Performance on the Traveling Salesperson Problem

James N. MacGregor — Thu, 18 Apr 2013 13:12:42 PDT

Most models of human performance on the traveling salesperson problem involve clustering of nodes, but few empirical studies have examined effects of clustering in the stimulus array. A recent exception varied degree of clustering and concluded that the more clustered a stimulus array, the easier a TSP is to solve (Dry, Preiss, & Wagemans, 2012). However, a limitation to this conclusion arises because degree of clustering may have been partially confounded with cluster location. An experiment was conducted to test the effects of cluster location while holding degree of clustering constant. Stimuli with a cluster near a boundary were solved more quickly and accurately than stimuli with the same cluster located more centrally. The results support and extend the previous findings of MacGregor, Ormerod, & Chronicle (1999). They also qualify the results of Dry et al. (2012). To the extent that degree of clustering may have been confounded with the location of clusters in their stimuli, it is unclear to what extent each factor may have affected performance.

A Comparison of Reinforcement Learning Models for the Iowa Gambling Task Using Parameter Space Partitioning

Helen Steingroever et al. — Thu, 18 Apr 2013 13:12:39 PDT

The Iowa gambling task (IGT) is one of the most popular tasks used to study decisionmaking deficits in clinical populations. In order to decompose performance on the IGT in its constituent psychological processes, several cognitive models have been proposed (e.g., the Expectancy Valence (EV) and Prospect Valence Learning (PVL) models). Here we present a comparison of three models—the EV and PVL models, and a combination of these models (EV-PU)—based on the method of parameter space partitioning. This method allows us to assess the choice patterns predicted by the models across their entire parameter space. Our results show that the EV model is unable to account for a frequency-of-losses effect, whereas the PVL and EV-PU models are unable to account for a pronounced preference for the bad decks with many switches. All three models underrepresent pronounced choice patterns that are frequently seen in experiments. Overall, our results suggest that the search of an appropriate IGT model has not yet come to an end.

Front Matter

Thu, 18 Apr 2013 13:12:36 PDT

The Problems with Problem Solving: Reflections on the Rise, Current Status, and Possible Future of a Cognitive Research Paradigm

Stellan Ohlsson — Wed, 17 Oct 2012 12:46:47 PDT

The research paradigm invented by Allen Newell and Herbert A. Simon in the late 1950s dominated the study of problem solving for more than three decades. But in the early 1990s, problem solving ceased to drive research on complex cognition. As part of this decline, Newell and Simon’s most innovative research practices – especially their method for inducing subjects’ strategies from verbal protocols - were abandoned. In this essay, I summarize Newell and Simon’s theoretical and methodological innovations and explain why their strategy identification method did not become a standard research tool. I argue that the method lacked a systematic way to aggregate data, and that Newell and Simon’s search for general problem solving strategies failed. Paradoxically, the theoretical vision that led them to search elsewhere for general principles led researchers away from studies of complex problem solving. Newell and Simon’s main enduring contribution is the theory that people solve problems via heuristic search through a problem space. This theory remains the centerpiece of our understanding of how people solve unfamiliar problems, but it is seriously incomplete. In the early 1970s, Newell and Simon suggested that the field should focus on the question where problem spaces and search strategies come from. I propose a breakdown of this overarching question into five specific research questions. Principled answers to those questions would expand the theory of heuristic search into a more complete theory of human problem solving.

Insight Problem Solving: A Critical Examination of the Possibility of Formal Theory

William H. Batchelder et al. — Wed, 17 Oct 2012 12:46:45 PDT

This paper provides a critical examination of the current state and future possibility of formal cognitive theory for insight problem solving and its associated “aha!” experience. Insight problems are contrasted with move problems, which have been formally defined and studied extensively by cognitive psychologists since the pioneering work of Alan Newell and Herbert Simon. To facilitate our discussion, a number of classical brainteasers are presented along with their solutions and some conclusions derived from observing the behavior of many students trying to solve them. Some of these problems are interesting in their own right, and many of them have not been discussed before in the psychological literature. The main purpose of presenting the brainteasers is to assist in discussing the status of formal cognitive theory for insight problem solving, which is argued to be considerably weaker than that found in other areas of higher cognition such as human memory, decision-making, categorization, and perception. We discuss theoretical barriers that have plagued the development of successful formal theory for insight problem solving. A few suggestions are made that might serve to advance the field.

Human Performance on Hard Non-Euclidean Graph Problems: Vertex Cover

Sarah Carruthers et al. — Wed, 17 Oct 2012 12:46:43 PDT

Recent studies on a computationally hard visual optimization problem, the Traveling Salesperson Problem (TSP), indicate that humans are capable of finding close to optimal solutions in near-linear time. The current study is a preliminary step in investigating human performance on another hard problem, the Minimum Vertex Cover Problem, in which solvers attempt to find a smallest set of vertices that ensures that every edge in an undirected graph is incident with at least one of the selected vertices. We identify appropriate measures of performance, examine features of problem instances that impact performance, and describe strategies typically employed by participants to solve instances of the Vertex Cover problem.

Relevancy in Problem Solving: A Computational Framework

Johan Kwisthout — Wed, 17 Oct 2012 12:46:41 PDT

When computer scientists discuss the computational complexity of, for example, finding the shortest path from building A to building B in some town or city, their starting point typically is a formal description of the problem at hand, e.g., a graph with weights on every edge where buildings correspond to vertices, routes between buildings to edges, and route-distances to edge-weights. Given such a formal description, either tractability or intractability of the problem is established, by proving that the problem either enjoys a polynomial time algorithm or is NP-hard. However, this problem description is in fact an abstraction of the actual problem of being in A and desiring to go to B: it focuses on the relevant aspects of the problem (e.g., distances between landmarks and crossings) and leaves out a lot of irrelevant details.

This abstraction step is often overlooked, but may well contribute to the overall complexity of solving the problem at hand. For example, it appears that “going from A to B” is rather easy to abstract: it is fairly clear that the distance between A and the next crossing is relevant, and that the color of the roof of B is typically not. However, when the problem to be solved is “make X love me”, where the current state is (assumed to be) “X doesn’t love me”, it is hard to agree on all the relevant aspects of this problem.

In this paper a computational framework is presented in order to formally investigate the notion of relevance in finding a suitable problem representation. It is shown that it is in itself intractable in general to find a minimal relevant subset of all problem dimensions that might or might not be relevant to the problem. Starting from a computational complexity stance, this paper aims to contribute a computational framework of ‘relevancy’ in problem solving, in order to be able to separate ‘easy to abstract’ from ‘hard to abstract’ problems. This framework is then used to discuss results in the literature on representation, (insight) problem solving and individual differences in the abstraction task, e.g., when experts in a particular domain are compared with novice problem solvers.

Indentations and Starting Points in Traveling Sales Tour Problems: Implications for Theory

James N. MacGregor — Wed, 17 Oct 2012 12:46:40 PDT

A complete, non-trivial, traveling sales tour problem contains at least one “indentation”, where nodes in the interior of the point set are connected between two adjacent nodes on the boundary. Early research reported that human tours exhibited fewer such indentations than expected. A subsequent explanation proposed that this was because the observed human tours were close to the optimal, and the optimal tours happened to have few indentations. The present article reports two experiments. The first was designed to test the “few indentations” hypothesis under more stringent conditions than previously, by including point sets with two (near) optimal solutions that had a different number of indentations. For these critical point sets, participants produced the optimal solution with fewer indentations significantly more often than the alternative optimal solution. In addition, participants’ solutions started on boundary points significantly more often than by chance. A second experiment tested whether the preference for fewer indentations is the result of a conscious strategy, or the product of the processes that generate a solution. The results supported the latter conclusion. The implications for theories of human tour generation are discussed.

Editor's Introduction

Iris van Rooij — Wed, 17 Oct 2012 12:46:38 PDT

Wed, 17 Oct 2012 12:46:37 PDT

Intuitive Tip of the Tongue Judgments Predict Subsequent Problem Solving One Day Later

Azurii K. Collier et al. — Fri, 11 May 2012 20:25:54 PDT

Often when failing to solve problems, individuals report some idea of the solution, but cannot explicitly access the idea. We investigated whether such intuition would relate to improvements in solving and to the manner in which a problem was solved after a 24- hour delay. On Day 1, participants attempted to solve Compound Remote Associate problems, for which they viewed three problem words (crab, sauce, pine) and tried to generate one solution word (apple) that could form a compound word with each problem word (crabapple, applesauce, pineapple). For problems they failed to solve, participants reported whether they had an intuitive sense that they might have solution related processing in the back of their mind, similar to a Tip-of-the-Tongue (TOT) experience. After an overnight delay, on Day 2 participants attempted to solve unsolved Old problems from Day 1 (mixed among New problems). Participants solved more Old problems for which they reported a TOT on Day 1 than Old problems without a TOT, demonstrating a TOT specific incubation effect. Interestingly, participants reported solving a marginally higher proportion of these TOT problems, compared to No TOT problems, with insight. Results suggest that intuitive TOT judgments are indicative of subthreshold solution related activation that can facilitate eventual problem solving, especially with insight.

Is Insight Always the Same? A Protocol Analysis of Insight in Compound Remote Associate Problems

Edward A. Cranford et al. — Fri, 11 May 2012 20:25:53 PDT

Compound Remote Associate (CRA) problems have been used to investigate insight problem solving using both behavioral and neuroimaging techniques. However, it is unclear to what extent CRA problems exhibit characteristics of insight such as impasses and restructuring. CRA problem-solving characteristics were examined in a study in which participants solved CRA problems while providing concurrent verbal protocols. The results show that solutions subjectively judged as insight by participants do exhibit some characteristics of insight. However, the results also show that there are at least two different ways in which people experience insight when solving CRA problems. Sometimes problems are solved and judged as insight when the solution is the first thing considered, but these solutions do not exhibit any characteristics of insight aside from the “Aha!” experience. In other cases, the solution is derived after a longer period of problem solving, and the solution process more closely resembles insight as it is has been traditionally defined in the literature. The results show that separating these two types of solution processes may provide a better understanding of the behavioral and neuroanatomical correlates of insight solutions.

Firing the Executive: When an Analytic Approach to Problem Solving Helps and Hurts

Daniel A. Aiello et al. — Fri, 11 May 2012 20:23:10 PDT

There is a general assumption that a more controlled or more focused attentional state is beneficial for most cognitive tasks. However, there has been a growing realization that creative problem solving tasks, such as the Remote Associates Task (RAT), may benefit from a less controlled solution approach. To test this hypothesis, in a 2x2 design, we manipulated whether solvers were given the RAT before or after an implicit learning task. We also varied whether they were told to “use their gut” as part of either initial task. The results suggest that a less analytic approach engendered by a “use your gut” instruction benefits performance on the RAT for monolingual solvers. The same benefit was not found for bilingual speakers suggesting that more controlled solution processes may be needed when speakers with multiple lexicons perform this task, which relies heavily on accessing common phrases in a particular language.

Visual Attention Modulates Insight Versus Analytic Solving of Verbal Problems

Ezra Wegbreit et al. — Fri, 11 May 2012 20:23:09 PDT

Behavioral and neuroimaging findings indicate that distinct cognitive and neural processes underlie solving problems with sudden insight. Moreover, people with less focused attention sometimes perform better on tests of insight and creative problem solving. However, it remains unclear whether different states of attention, within individuals, influence the likelihood of solving problems with insight or with analysis. In this experiment, participants (N = 40) performed a baseline block of verbal problems, then performed one of two visual tasks, each emphasizing a distinct aspect of visual attention, followed by a second block of verbal problems to assess change in performance. After participants engaged in a center-focused flanker task requiring relatively focused visual attention, they reported solving more verbal problems with analytic processing. In contrast, after participants engaged in a rapid object identification task requiring attention to broad space and weak associations, they reported solving more verbal problems with insight. These results suggest that general attention mechanisms influence both visual attention task performance and verbal problem solving.

Remote Associates Test and Alpha Brain Waves

Henk J. Haarmann et al. — Fri, 11 May 2012 20:23:07 PDT

Previous studies found that performance on the remote associates test (RAT) improves after a period of incubation and that increased alpha brain waves over the right posterior brain predict the emergence of RAT insight solutions. We report an experiment that tested whether increased alpha brain waves during incubation improve RAT performance. Participants received two blocks of RAT items (RAT1 and RAT2), with the second block consisting of items that were not solved during the first block. Participants were randomly assigned to three groups, which were matched for their number of RAT1 solutions. Participants in an alpha-up neurofeedback group aimed to increase their alpha brain waves over the right posterior brain in between the two blocks, whereas participants in an alpha-down neurofeedback group aimed to decrease these same brain waves. A third group of participants did not perform neurofeedback and proceeded immediately from the first to the second block of RAT items. We found evidence for more RAT2 solutions in participants who interrupted their RAT performance with neurofeedback, especially in ones who showed high alpha brain waves during neurofeedback. These results are consistent with the notion that an alert but relaxed mental state, indexed by alpha brain waves, may aid the read out of an implicitly activated memory network of weak associates.

Testing the Cue Dependence of Problem-Solving-Induced Forgetting

Benjamin C. Storm et al. — Fri, 11 May 2012 20:23:03 PDT

Thinking and remembering can cause forgetting. In the context of remembering, retrieving one item can cause the forgetting of other items (Anderson, Bjork, & Bjork, 1994). A similar phenomenon has been observed in the context of creative problem solving—attempting to generate a target associate in the Remote Associates Test (RAT) can cause the forgetting of inappropriate associates (Storm, Angello, & Bjork, 2011). Experiment 1 examined whether this problem-solving-induced forgetting is cue dependent or cue independent by manipulating the cues used at final test. Whereas some participants were tested on the inappropriate associates using the same cues that were used during problem solving, other participants were tested using new, or independent, cues. Problem-solving-induced forgetting was observed in the same-cue condition, but not in the new-cue condition. Experiment 2 replicated the overall absence of problem-solvinginduced forgetting in the new-cue condition and found that individual differences in cue-independent forgetting did not predict problem-solving performance on a separate set of RAT problems.

Clue Insensitivity in Remote Associates Test Problem Solving

Steven M. Smith et al. — Fri, 11 May 2012 20:23:02 PDT

Does spreading activation from incidentally encountered hints cause incubation effects? We used Remote Associates Test (RAT) problems to examine effects of incidental clues on impasse resolution. When solution words were seen incidentally 3-sec before initially unsolved problems were retested, more problems were resolved (Experiment 1). When strong semantic associates of solutions were used as incidental clues, however, it did not improve resolution (Experiments 2 and 4). The semantic associates we used as incidental clues primed our RAT solution words in a lexical decision task, but they did not facilitate impasse resolution unless participants were explicitly instructed to use the associates as hints to the retested problems (Experiment 4). The results do not support the theory that spreading activation is a sufficient cause of incubation effects, and suggest that serendipitously encountered clues (i.e., words that are semantically related to RAT solutions) have no automatic benefit on impasse resolution in RAT problem solving.

Investigating Insight as Sudden Learning

Ivan K. Ash et al. — Fri, 11 May 2012 20:22:59 PDT

Gestalt psychologists proposed two distinct learning mechanisms. Associative learning occurs gradually through the repeated co-occurrence of external stimuli or memories. Insight learning occurs suddenly when people discover new relationships within their prior knowledge as a result of reasoning or problem solving processes that re-organize or restructure that knowledge. While there has been a considerable amount of research on the type of problem solving processes described by the Gestalt psychologists, less has focused on the learning that results from these processes. This paper begins with a historical review of the Gestalt theory of insight learning. Next, the core assumptions of Gestalt insight learning theory are empirically tested with a study that investigated the relationships among problem difficulty, impasse, initial problem representations, and resolution effects. Finally, Gestalt insight learning theory is discussed in relation to modern information processing theories of comprehension and memory formation.