Assistant Professor of Neural Science and Psychology
Cognition & Perception , Center for Neural Science & Center for Brain Imaging

Research

I study how people and animals learn from trial and error (and from rewards and punishments) to make decisions, combining computational, neural, and behavioral perspectives. I focus on understanding how subjects cope with computationally demanding decision situations, notably choice under uncertainty and in tasks (such as mazes or chess) requiring many decisions to be made sequentially. In engineering, these are the key problems motivating reinforcement learning and Bayesian decision theory. I am particularly interested in using these computational frameworks as a basis for analyzing and understanding biological decision making. Some ongoing projects include:

Computational models in neuroscientific experiments.

Computational models (such as reinforcement learning algorithms) are more than cartoons: they can provide exquisitely detailed trial-by-trial hypotheses about how subjects might approach tasks such as decision making. By fitting such models to behavioral and neural data, and comparing different candidates, we can understand in detail the processes underlying subjects' choices. Such models can also quantitatively characterize hitherto subjective phenomena (such as the anticipation of reward or punishment), allowing the principled study of their neural representations. Methodologically, I am interested in developing experimental designs and analytical techniques for such issues as how to use models to pool heterogeneous data sources (such as simultaneously obtained choice behavior, eye monitoring, and BOLD signals from multiple brain areas). Practically, I apply these methods in behavioral and functional imaging experiments to study human decision making, including some of the issues discussed below.

Interactions between multiple decision-making systems.

The idea that the brain contains multiple, separate decision systems is as ubiquitous (in psychology, neuroscience, and even economics) as it is bizarre. For instance, much evidence points to competition between a reflective or cognitive planning system centered in prefrontal cortex, and a more stimulus-bound 'habitual' controller associated with dopamine and the basal ganglia. Such competition has often been implicated in self-control issues such as dieting or drug addiction. But (as these examples suggest) having multiple solutions to the problem of making decisions actually compounds the decision problem, by requiring the brain to choose between the systems. The computational underpinnings and neural substrates for this sort of arbitration are poorly understood. I have pursued computational models of multiple decision making systems and their interactions; armed with such a detailed characterization, we are beginning to search for the fingerprints of these interactions in
human functional imaging data.

Learning and neuromodulation.

Much evidence has now amassed for the idea that the neuromodulator dopamine serves as a particular sort of teaching signal for reinforcement learning in appetitive tasks. This relatively good characterization can now provide a foothold for extending this understanding in a number of exciting new directions. These include computational (e.g., how can this system balance the need to explore unfamiliar options versus exploit old favorites), behavioral (how is dopaminergically mediated learning manifest; how is it deficient in pathologies such as drug addiction or Parkinson's disease), and neural (what is the contribution of systems that interact with dopamine, such as serotonin and the prefrontal cortex). One recent example that crosscuts these categories is the interaction of appetitive and aversive learning. Psychologists have long suggested that the brain contains parallel, opponent motivational systems for reward and punishment; the identification of the former with dopamine allowed us to suggest an account of serotonin as its opponent for aversive learning. We are presently investigating these ideas with imaging and pharmacological studies of decision making under reward and punishment.

back to the top

Biography

Awards:

• McKnight Scholar Award (2009)
• NARSAD Young Investigator Award (2009)

Education

• Ph.D. (computer science, cognitive neuroscience) Carnegie Mellon University, Pittsburgh, PA, 2003
• M.S. (computer science) Carnegie Mellon University, Pittsburgh, PA, 2000
• B.A. (philosophy of science) Columbia University, New York, NY, 1996

Fellowships

• Royal Society USA Research Fellowship (2003-2006)
• National Science Foundation Graduate Research Fellowship (1998-2001)
  

back to the top

Selected Publications

Simon, D.A., and Daw, N.D. (2011, in press) “Neural correlates of forward planning in a spatial decision task in humans.” Journal of Neuroscience.

Daw, N.D., Gershman, S.J., Seymour, B., Dayan, P., and Dolan, R.J. (2011, in press) “Model-based influences on humans’ choices and striatal prediction errors.” Neuron.

Cools, R., Nakamura, K., and Daw, N.D., (2011) “Serotonin and dopamine: Unifying affective, activational, and decision functions,” Neuropsychopharmacology 36:98-113.

Gershman, S.J., Pesaran, B., and Daw, N.D. (2009), “Human reinforcement learning subdivides structured action spaces by learning effector-specific values,” Journal of Neuroscience 29:13524-31.

Dayan, P., and Daw, N.D., (2008) “Decision theory, reinforcement learning, and the brain,”  Cognitive, Affective, and Behavioral Neuroscience 8:429-453.

Daw, N.D.*, O’Doherty, J.P.*, Dayan, P., Seymour, B., and Dolan, R.J. (2006) “Cortical substrates for exploratory decisions in humans,” Nature 441:876-879.

Courville, A.C.*, Daw, N.D.*, and Touretzky, D.S. (2006) “Bayesian theories of conditioning in a changing world,” Trends in Cognitive Sciences: 10:294-300.

Daw, N.D., Niv, Y., and Dayan, P. (2005) “Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control,” Nature Neuroscience 8:1704-1711.

Daw, N.D., Kakade, S., and Dayan, P. (2002) “Opponent interactions between serotonin and dopamine,” Neural Networks 15:603-616.
back to the top

Address

Nathaniel Daw
Assistant Professor of Neural Science and Psychology
Department of Psychology
New York University
6 Washington Place, room 873
New York, NY 10003
(212) 998-2104

back to the top