The

Journal

of

Neuroscience

May

-LRB-

-RRB-

Behavioral/Systems/Cognitive

Time

Discounting

for

Primary

Rewards

Samuel

M.

McClure

Keith

M.

Ericson

David

I.

Laibson

George

Loewenstein

and

Jonathan

D.

Center

for

the

Study

of

Brain

Mind

and

Behavior

and

Department

of

Psychology

Princeton

University

Princeton

New

Jersey

Bureau

for

Economic

Research

Cambridge

Massachusetts

of

Economics

Harvard

University

Cambridge

Massachusetts

of

Social

and

Decision

Science

Carnegie

Mellon

University

Pittsburgh

Pennsylvania

and

of

Psychiatry

University

of

Pittsburgh

Pittsburgh

Pennsylvania

Previous

research

involving

monetary

rewards

found

that

limbic

reward-related

areas

show

greater

activity

when

an

intertemporal

choice

includes

an

immediate

reward

than

when

the

options

include

only

delayed

rewards

In

contrast

the

lateral

prefrontal

and

parietal

cortex

-LRB-

areas

commonly

associated

with

deliberative

cognitive

processes

including

future

planning

-RRB-

respond

to

intertemporal

choices

in

general

but

do

not

exhibit

sensitivity

to

immediacy

-LRB-

McClure

et

al.

-RRB-

The

current

experiments

extend

these

findings

to

primary

rewards

-LRB-

fruit

juice

or

water

-RRB-

and

time

delays

of

minutes

instead

of

weeks

Thirsty

subjects

choose

between

small

volumes

of

drinks

delivered

at

precise

times

during

the

experiment

-LRB-

e.g.

ml

now

vs

ml

in

min

-RRB-

Consistent

with

previous

findings

limbic

activation

was

greater

for

choices

between

an

immediate

reward

and

a

delayed

reward

than

for

choices

between

two

delayed

rewards

whereas

the

lateral

prefrontal

cortex

and

posterior

parietal

cortex

responded

similarly

whether

choices

were

between

an

immediate

and

a

delayed

reward

or

between

two

delayed

rewards

Moreover

relative

activation

of

the

two

sets

of

brain

regions

predicts

actual

choice

behavior

A

second

experiment

finds

that

when

the

delivery

of

all

rewards

is

offset

by

min

-LRB-

so

that

the

earliest

available

juice

reward

in

any

choice

is

min

-RRB-

no

differential

activity

is

observed

in

limbic

reward-related

areas

for

choices

involving

the

earliest

versus

only

more

delayed

rewards

We

discuss

implications

of

this

finding

for

differences

between

primary

and

secondary

rewards

Key

words

discounting

primary

reward

ventral

striatum

medial

prefrontal

cortex

DLPFC

ACC

Introduction

Empirical

studies

of

intertemporal

choice

find

that

value

declines

rapidly

over

the

short

run

but

at

a

slower

rate

over

the

long

run

-LRB-

Ainslie

Thaler

Loewenstein

and

Prelec

Kirby

-RRB-

A

convenient

functional

form

for

representing

such

time

preferences

is

the

quasihyperbolic

discount

function

-LRB-

Laibson

-RRB-

according

to

which

the

present

value

of

a

stream

of

consumptions

-LRB-

-RRB-

is

as

follows

Vt

u

ct

t

t

u

ct

-LRB-

-RRB-

where

u

is

the

utility

function

and

discount

parameters

and

are

bounded

between

and

A

multiplicatively

scaled

transformation

of

V

can

be

decomposed

into

two

distinct

processes

Wt

u

ct

system

u

ct

system

-LRB-

-RRB-

Received

Sept.

revised

March

accepted

April

This

work

was

supported

by

National

Institutes

of

Health

Grants

-LRB-

J.D.C.

-RRB-

-LRB-

J.D.C.

-RRB-

-LRB-

S.M.M.

-RRB-

-LRB-

D.I.L.

-RRB-

and

-LRB-

D.I.L.

-RRB-

K.

D'Ardenne

contributed

significantly

to

the

task

design

and

was

critical

in

data

collection

P.

Dayan

provided

valuable

comments

on

this

manuscript

Correspondence

should

be

addressed

to

Samuel

M.

McClure

Center

for

the

Study

of

Brain

Mind

and

Behavior

Princeton

University

Princeton

NJ

E-mail

smcclure@princeton.edu

DOI

Copyright

Society

for

Neuroscience

The

term

labeled

system

exhibits

exponential

discounting

with

discount

factor

That

labeled

system

captures

the

extra

weight

given

to

immediate

rewards

Based

on

previous

research

-LRB-

McClure

et

al.

Tanaka

et

al.

-RRB-

we

expected

the

impatient

system

to

be

related

to

the

mesolimbic

dopamine

system

and

associated

structures

-LRB-

henceforth

limbic

reward

areas

-RRB-

Humans

share

this

limbic

reward

system

with

many

other

mammalian

species

of

which

respond

to

costs

and

benefits

delayed

more

than

a

few

minutes

except

in

a

rigid

preprogrammed

manner

Moreover

recent

research

finds

that

people

with

greater

activation

in

these

limbic

reward

regions

in

response

to

gaining

or

losing

money

also

place

greater

weight

on

immediate

rewards

relative

to

delayed

rewards

-LRB-

Hariri

et

al.

-RRB-

Additionally

we

anticipated

the

component

to

be

associated

with

prefrontal

and

parietal

cortex

These

regions

have

been

implicated

in

planning

and

deliberation

Damage

to

these

areas

in

humans

produces

a

pattern

of

short-sighted

nondeliberative

behavior

termed

reward-dependency

syndrome

-LRB-

Lhermitte

-RRB-

Previously

we

studied

intertemporal

choices

between

gift

certificates

-LRB-

McClure

et

al.

-RRB-

We

found

that

activity

in

limbic

reward

areas

-LRB-

ventral

striatum

medial

prefrontal

cortex

and

orbitofrontal

cortex

-RRB-

is

associated

with

choices

involving

immediate

rewards

in

contrast

prefrontal

and

parietal

regions

are

active

in

all

intertemporal

choices

-LRB-

not

just

the

choices

with

an

immediate

option

-RRB-

One

purpose

of

the

current

study

was

to

test

whether

these

results

replicate

with

primary

rewards

and

shorter

delays

Thirsty

subjects

chose

between

volumes

of

juice

or

water

available

at

different

times

within

the

experiment

Unlike

gift

certificates

McClure

et

al.

Time

Discounting

J.

Neurosci.

May

-LRB-

-RRB-

larger

option

on

every

choice

-LRB-

which

seemed

to

indicate

either

lack

of

engagement

with

the

task

or

inappropriate

calibration

of

the

task

for

this

subject

's

intertemporal

preferences

-RRB-

The

timing

structure

of

both

experiments

is

shown

in

Figure

Each

s

period

was

broken

into

a

decision

period

and

a

fluid

delivery

period

During

each

decision

period

subjects

were

shown

a

choice

-LRB-

example

display

shown

in

Fig.

B

-RRB-

and

given

up

to

s

to

indicate

their

preference

via

a

button

box

During

the

s

fluid

delivery

period

a

total

of

four

squirts

of

fluid

could

be

delivered

depending

on

the

subjects

choices

in

previous

decision

periods

The

experiment

was

separated

into

four

min

blocks

with

a

min

rest

period

between

blocks

The

ordering

of

the

choices

is

discussed

in

section

of

the

supplemental

material

-LRB-

available

at

-RRB-

Figure

Experiment

design

A

Each

s

of

the

experiment

was

split

into

a

response

period

and

a

liquid

delivery

period

Experiment

choices

Subjects

answered

one

Subjects

were

allowed

a

maximum

of

s

to

respond

during

the

response

period

B

Choices

were

displayed

with

juice

or

water

control

question

-LRB-

one

squirt

of

juice

at

min

indicated

by

the

color

of

a

circle

icon

-LRB-

red

juice

blue

water

-RRB-

Responses

were

indicated

with

a

button

box

To

avoid

startling

delay

or

one

squirt

of

juice

at

min

delay

-RRB-

at

subject

a

button

press

was

required

to

initiate

liquid

delivery

Max

Maximum

the

beginning

of

the

experiment

to

acquaint

them

with

the

task

This

was

followed

by

an

drink

rewards

allow

us

to

control

both

when

reward

is

delivered

intermixed

set

of

experimental

choices

and

control

trials

-LRB-

to

be

described

below

-RRB-

and

when

it

is

consumed

Such

control

is

absent

from

most

preThe

intertemporal

characteristics

of

the

experimental

choices

can

be

vious

studies

of

time

discounting

in

people

-LRB-

Ainslie

and

divided

into

six

delay

categories

We

represent

each

delay

category

as

a

Monterosso

-RRB-

which

has

generated

debate

over

the

validity

pair

of

alternatives

D

versus

D

where

D

is

the

delay

to

the

early

reward

of

time

discounting

studies

-LRB-

Cubitt

and

Read

-RRB-

and

D

is

the

delay

to

the

alternative

later

reward

In

a

given

trial

the

We

extend

our

previous

statistical

analysis

by

estimating

a

subject

chose

one

of

these

two

options

early

or

late

The

six

delay

catemore

flexible

functional

form

for

discounting

We

generalize

the

gories

follow

versus

min

versus

min

versus

min

versus

idea

that

discounting

can

be

decomposed

into

present-oriented

min

versus

min

and

versus

min

Given

this

structure

it

and

more

patient

processes

Let

D

-LRB-

-RRB-

represent

an

aggregate

disfollows

that

the

time

difference

between

the

late

reward

and

the

early

count

function

where

is

the

delay

until

consumption

and

D

-LRB-

-RRB-

reward

D

D

was

either

or

min

and

the

time

of

the

earliest

reward

represents

the

current

value

of

one

unit

of

delayed

utility

We

D

was

or

min

The

reward

magnitudes

can

be

divided

into

three

magnitude

categohypothesize

that

D

-LRB-

-RRB-

is

composed

of

weighted

subcomponents

ries

We

represent

each

magnitude

category

as

a

pair

of

alternatives

R

associated

with

different

brain

systems

with

one

component

and

R

where

R

is

the

magnitude

of

the

early

reward

-LRB-

in

ml

squirts

of

D

-LRB-

-RRB-

that

discounts

at

a

faster

rate

than

another

D

-LRB-

-RRB-

The

fluid

-RRB-

and

R

is

the

magnitude

of

the

mutually

exclusive

later

reward

-LRB-

in

overall

utility

function

is

as

follows

squirts

of

fluid

-RRB-

The

three

magnitude

categories

were

versus

versus

and

versus

Over

the

duration

of

the

experiment

subjects

were

presented

with

V

t

D

u

ct

D

u

ct

each

possible

combination

of

-LRB-

R

D

-RRB-

versus

-LRB-

R

D

-RRB-

twice

once

with

juice

-LRB-

-RRB-

and

once

with

water

Hence

the

experimental

choices

had

a

system

system

structure

-LRB-

six

delay

categories

by

three

magnitude

categories

by

two

fluid

categories

-RRB-

or

experimental

choices

in

total

The

type

of

fluid

-LRB-

juice

or

We

estimate

this

function

for

choice

behavior

and

neural

activity

water

-RRB-

was

indicated

visually

as

depicted

in

Figure

B

red

circles

indiMaterials

and

Methods

cated

juice

and

blue

circles

indicated

water

The

control

trials

were

introduced

to

control

for

the

motor

and

visual

Subjects

made

a

series

of

binary

choices

of

the

following

form

choose

requirements

associated

with

task

performance

On

these

trials

subjects

either

R

squirts

of

fluid

delivered

at

delay

D

min

or

R

squirts

of

fluid

were

simply

required

to

press

a

button

to

receive

a

specified

number

of

delivered

at

delay

D

min

Some

choices

were

between

early

and

late

squirts

after

a

given

delay

Perhaps

in

part

because

of

low

power

as

a

result

delivery

of

juice

and

other

choices

were

between

early

and

late

delivery

of

of

fitting

only

events

we

were

unable

to

interpret

the

pattern

of

brain

water

The

use

of

juice

and

water

allowed

us

to

double

the

number

of

activity

during

control

events

and

do

not

discuss

these

choices

further

choices

subjects

responded

to

and

reduce

the

risk

that

subjects

would

Experiment

choices

Subjects

responded

to

a

total

of

intertemporal

experience

the

task

as

repetitive

and/or

adopt

a

fixed

rote

decision

stratchoices

in

experiment

Delays

to

reward

delivery

corresponded

to

one

egy

No

trials

were

mixed

in

other

words

the

choices

between

early

of

three

values

min

-LRB-

labeled

as

early

-RRB-

min

-LRB-

middle

-RRB-

and

delivery

and

late

delivery

were

always

for

the

same

type

of

fluid

Hence

min

-LRB-

late

-RRB-

Delay

categories

were

early

versus

middle

middle

versus

each

choice

can

be

summarized

by

a

quintuple

-LRB-

R

D

R

D

fluid

type

-RRB-

late

and

early

versus

late

Subjects

were

told

the

meaning

of

the

different

where

the

fluid

type

was

either

juice

or

water

delay

period

labels

before

entering

the

scanner

Reward

values

were

as

in

We

scanned

a

total

of

-LRB-

female

-RRB-

Princeton

graduate

and

underexperiment

with

R

and

R

corresponding

to

versus

versus

and

graduate

students

with

a

Tesla

Siemens

-LRB-

Munich

Germany

-RRB-

Allegra

versus

squirts

of

juice

or

water

Subjects

were

given

two

choices

for

scanner

-LRB-

female

-RRB-

for

experiment

and

-LRB-

female

-RRB-

for

experieach

pairing

of

delay

and

reward

value

and

one

each

for

juice

and

water

ment

Subjects

were

prescreened

to

exclude

history

of

psychiatric

disJuice

and

water

delivery

Squirts

of

juice

and

water

were

delivered

by

a

order

or

drug

use

Subjects

were

instructed

to

abstain

from

drinking

any

computer-controlled

syringe

pump

and

customized

software

Individual

fluids

for

h

before

participating

squirts

were

delivered

at

a

rate

of

ml/s

for

s

giving

a

squirt

size

of

ml

Three

subjects

were

excluded

from

data

analysis

-LRB-

all

from

experiment

Before

receiving

a

squirt

subjects

were

required

to

press

a

button

This

-RRB-

one

for

failing

to

abstain

from

drinking

before

the

experiment

one

because

of

excessive

head

movement

and

a

third

for

selecting

the

later

ensured

that

subjects

would

have

control

over

and

not

be

surprised

by

J.

Neurosci.

May

-LRB-

-RRB-

McClure

et

al.

Time

Discounting

the

fluid

delivery

This

plus

the

small

squirt

size

was

sufficient

to

keep

swallowing-related

head

movement

to

a

minimum

-LRB-

all

subjects

included

in

the

data

analysis

had

total

head

movement

mm

in

any

direction

during

the

experiment

-RRB-

Functional

magnetic

resonance

imaging

data

analysis

Functional

magnetic

resonance

imaging

-LRB-

fMRI

-RRB-

data

analysis

was

performed

using

AFNI

-LRB-

Cox

-RRB-

-LRB-

Wellcome

Department

of

Cognitive

Neurology

London

UK

-RRB-

and

custom-written

programs

in

Matlab

Whole-brain

blood

oxygenation

level-dependent

-LRB-

BOLD

-RRB-

weighted

echo-planar

images

were

acquired

parallel

to

the

anterior

commissure

posterior

commissure

line

with

a

repetition

time

of

s

and

echo

time

of

ms.

Functional

images

were

first

aligned

to

correct

for

head

movement

during

the

scan

Slice

timing

correction

was

then

performed

using

sinc

interpolation

Images

were

subsequently

normalized

to

Montreal

Neurological

Institute

coordinates

and

resampled

at

mm

resolution

A

Gaussian

smoothing

kernel

of

mm

full-width

at

half-maximum

-LRB-

FWHM

-RRB-

was

applied

to

improve

comparison

across

subjects

Head

movement

estimates

derived

from

the

first

realignment

step

were

included

as

regressors

in

all

analyses

to

help

diminish

the

impact

of

any

movement-related

effects

on

the

results

Analysis

of

a

general

linear

model

-LRB-

GLM

-RRB-

was

performed

using

In

our

previous

study

-LRB-

McClure

et

al.

-RRB-

we

conducted

such

an

analysis

with

a

regressor

that

was

constructed

by

weighting

decision

periods

with

an

immediate

reward

in

the

choice

set

by

weighting

decision

periods

with

only

delayed

options

in

the

choice

set

by

and

using

a

zero

weighting

for

periods

in

which

no

decision

is

made

We

called

this

the

regressor

In

contrast

the

regressor

used

in

our

current

analysis

selectively

weights

decision

periods

with

an

immediate

option

in

the

choice

set

by

and

all

other

periods

by

This

new

regressor

ensures

that

areas

showing

increased

activity

only

for

decisions

involving

an

immediate

reward

will

only

load

on

the

new

regressor

The

differences

produced

by

the

use

of

the

new

regressor

on

the

data

from

our

previous

study

are

subtle

and

do

not

qualitatively

change

any

of

the

effects

originally

reported

In

comparing

the

results

between

studies

-LRB-

see

Fig.

-RRB-

we

have

reanalyzed

our

previous

data

with

the

new

regressor

structure

In

our

previous

study

we

also

used

a

regressor

that

was

constructed

by

weighting

all

decision

periods

by

and

weighting

all

other

periods

by

We

called

this

the

regressor

and

we

use

the

same

regressor

in

the

current

study

In

addition

to

the

and

regressors

regressors

were

included

for

head

movement

-LRB-

six

regressors

for

movements

along

the

x

y

and

z-axes

as

well

as

rotations

around

the

x

y

and

z-axes

-RRB-

Baseline

drift

in

the

magnetic

resonance

signal

was

accounted

for

with

three

regressors

to

estimate

a

quadratic

dependence

between

signal

amplitude

and

time

Finally

additional

regressors

were

included

to

account

for

response

time

and

time

on

task

-LRB-

i.e.

linear

dependence

on

choice

number

-RRB-

All

regressors

were

fit

simultaneously

in

a

single

GLM

analysis

GLMs

were

fit

to

each

subject

individually

Between-subjects

t

tests

were

then

performed

to

identify

voxels

that

loaded

significantly

on

the

and/or

regressors

The

AFNI

program

AlphaSim

was

used

to

determine

our

significance

criterion

The

smoothness

of

our

data

were

estimated

using

Using

these

tools

we

determined

that

a

corrected

-LRB-

familywise

-RRB-

p

is

achieved

with

a

minimum

of

nine

contiguous

voxels

each

significant

at

p

Individual

trial

estimates

of

brain

response

We

estimated

the

amplitude

of

brain

response

for

each

identified

brain

region

separately

for

each

trial

to

correlate

with

choices

and

to

fit

discount

functions

To

do

so

BOLD

data

for

each

voxel

identified

in

the

initial

GLM

were

normalized

by

the

mean

across

the

experiment

to

give

percentage

signal

change

relative

to

baseline

and

then

averaged

across

voxels

in

each

identified

brain

region.We

then

fit

gamma

functions

during

each

choice

epoch

with

the

following

functional

form

Our

fitting

procedure

was

developed

to

overcome

two

main

challenges

First

we

did

not

know

at

which

exact

time

choice-related

brain

responses

occurred

This

is

because

subjects

had

s

to

make

their

choice

on

each

trial

and

took

several

seconds

on

average

to

render

their

choice

We

therefore

included

the

temporal

offset

parameter

which

was

allowed

to

vary

to

give

the

best

fit

to

the

BOLD

signal

-LRB-

where

is

at

most

the

subject

's

response

time

-RRB-

The

second

challenge

arose

in

estimating

the

baseline

from

which

the

brain

response

occurred

The

quadratic

fit

to

the

BOLD

signal

drift

during

the

experiment

was

adequate

for

the

full

GLM

analysis

-LRB-

above

-RRB-

but

did

not

account

for

higher-frequency

signal

drift

that

was

apparent

during

the

period

of

each

choice

We

therefore

fit

the

baseline

signal

individually

for

each

choice

by

computing

a

weighted

average

of

the

BOLD

signal

amplitude

around

the

time

at

which

the

choice

was

visually

displayed

We

gave

maximal

weight

to

the

BOLD

signal

at

the

time

when

the

choice

was

first

displayed

with

linearly

decreasing

weight

for

the

subsequent

s

-LRB-

to

a

weight

of

at

s

-RRB-

because

changes

in

BOLD

response

are

known

to

lag

neural

events

by

at

least

s

A

linearly

decreasing

weight

was

also

given

to

the

BOLD

signal

for

the

s

before

choice

presentation

-LRB-

weight

at

s

-RRB-

We

did

not

weight

BOLD

signals

beyond

s

because

of

the

proximity

of

preceding

juice

and

water

delivery

events

After

correcting

for

baseline

BOLD

amplitude

a

simplex

fitting

procedure

was

performed

in

Matlab

to

find

values

of

a

and

that

minimized

the

sum-squared

error

from

actual

BOLD

signal

measures

Brain

activity

and

behavior

To

estimate

the

relationship

between

brain

activity

and

behavior

-LRB-

including

fits

of

discount

functions

to

brain

activity

-RRB-

we

first

mean

normalized

the

BOLD

response

amplitudes

by

dividing

by

the

mean

amplitude

across

all

voxels

in

each

brain

region

for

each

subject

and

across

all

trials

We

then

averaged

these

mean

normalized

responses

over

all

and

areas

respectively

In

fitting

to

behavior

probit

models

were

used

to

predict

from

brain

activity

the

probability

of

choosing

the

earlier

of

the

two

rewards

in

a

given

choice

In

addition

to

the

normed

brain

activity

variables

we

entered

the

following

set

of

basic

control

variables

indicator

variables

for

each

subject

choice

number

and

the

number

of

juice

squirts

the

subject

received

in

the

delivery

period

before

the

choice

Where

noted

we

also

control

for

specific

choice

type

-LRB-

i.e.

an

indicator

variable

for

each

combination

of

reward

size

and

delay

-RRB-

Details

for

the

procedure

used

to

fit

discount

functions

to

brain

activity

are

given

in

section

of

the

supplemental

material

-LRB-

available

at

-RRB-

Results

Experiment

Each

subject

was

presented

with

binary

intertemporal

choices

-LRB-

the

timing

and

sequence

of

events

in

the

experiment

is

depicted

in

Fig.

-RRB-

For

each

choice

subjects

decided

between

a

reward

of

size

R

at

delay

D

and

a

second

reward

of

size

R

at

delay

D

-LRB-

where

R

R

and

D

D

-RRB-

We

varied

the

size

of

the

two

rewards

-LRB-

R

and

R

-RRB-

the

time

to

the

earliest

reward

-LRB-

D

-RRB-

and

the

lag

between

rewards

-LRB-

D

D

-RRB-

as

detailed

above

Each

choice

was

repeated

for

juice

squirts

and

water

squirts

Choice

behavior

Subjects

showed

clear

evidence

of

nonexponential

discounting

in

their

behavioral

preferences

Figure

provides

a

nonparametric

test

of

this

property

In

Figure

we

bin

the

data

by

the

value

of

D

the

time

delay

to

the

early

reward

in

a

binary

choice

in

our

experiment

D

is

or

min

The

probability

of

selecting

the

earlier

reward

for

the

data

labeled

D

min

in

Figure

A

is

calculated

as

the

percentage

of

trials

that

subjects

select

the

early

option

from

the

six

sets

of

trials

listed

in

Table

-LRB-

each

row

represents

a

trial

which

is

a

choice

between

an

early

option

and

a

later

option

-RRB-

Likewise

the

probability

of

selecting

the

early

reward

for

the

data

labeled

D

min

is

calculated

as

the

percentage

of

trials

that

subjects

select

the

early

option

from

the

trials

detailed

in

Table

-LRB-

again

each

row

represents

a

choice/trial

-RRB-

Analogous

calculations

are

conducted

h

t

a

cb

be

b

t

b

et

c

-LRB-

-RRB-

This

is

a

normalized

version

of

the

gamma

function

developed

by

Cohen

-LRB-

-RRB-

values

for

b

and

c

were

taken

from

Cohen

-LRB-

-RRB-

-LRB-

and

respectively

-RRB-

McClure

et

al.

Time

Discounting

J.

Neurosci.

May

-LRB-

-RRB-

Figure

Impulsivity

in

choice

behavior

A

The

probability

of

selecting

the

earlier

smaller

liquid

reward

was

calculated

for

each

subject

based

on

the

time

to

the

earliest

reward

option

-LRB-

D

-RRB-

Subjects

were

significantly

more

likely

to

select

the

earlier

option

when

they

could

obtain

it

at

a

min

delay

than

when

the

earlier

reward

was

available

after

a

or

min

delay

B

Additional

inspection

reveals

that

the

enhanced

preference

for

immediate

rewards

is

driven

primarily

by

choices

between

an

immediate

reward

and

a

reward

at

min

delay

Subjects

did

not

show

evidence

for

impulsive

responding

for

choices

between

an

immediate

reward

and

one

available

in

only

min

Error

bars

represent

SEM

across

subjects

Table

Trials

with

no

delay

for

early

option

Early

option

Delay

min

min

min

min

min

min

Magnitude

squirt

squirt

squirts

squirt

squirt

squirts

Figure

Response

to

juice

and

water

delivery

A

Brain

areas

activated

by

the

delivery

of

juice

and

water

included

gustatory

cortex

in

the

insula

-LRB-

Ins

-RRB-

premotor

areas

and

regions

in

the

ventral

striatum

-LRB-

VStr

p

minimum

contiguous

voxels

-RRB-

The

VStr

is

strongly

associated

with

reward

processing

SMA

Supplementary

motor

area

B

In

the

VStr

responses

to

juice

and

water

were

of

equal

amplitude

suggesting

that

subjects

found

the

two

liquids

to

be

approximately

equally

rewarding

Furthermore

choices

for

juice

or

water

squirts

were

statistically

indistinguishable

Because

of

these

two

findings

subsequent

analyses

do

not

distinguish

between

juice

and

water

Later

option

Delay

min

min

min

min

min

min

Magnitude

squirts

squirts

squirts

squirts

squirts

squirts

Table

Trials

with

min

delay

for

early

option

Early

option

Later

option

Delay

min

min

min

min

min

min

Magnitude

squirt

squirt

squirts

squirt

squirt

squirts

Delay

min

min

min

min

min

min

Magnitude

squirts

squirts

squirts

squirts

squirts

squirts

resented

by

Equation

We

first

estimated

a

probit

model

that

predicts

choice

based

on

the

discounting

model

This

yields

-LRB-

SE

-RRB-

and

-LRB-

SE

parameter

estimates

of

-RRB-

We

can

not

reject

the

hypothesis

that

the

system

does

not

discount

at

all

-LSB-

p

likelihood

ratio

-LRB-

LR

-RRB-

test

-RSB-

However

we

can

reject

the

hypothesis

that

the

discounting

is

purely

exponential

-LRB-

p

LR

test

-RRB-

Next

we

fit

choices

with

a

double-exponential

implementation

of

Equation

where

D

-LRB-

-RRB-

and

D

-LRB-

-RRB-

This

is

a

continuous-time

generalization

of

the

model

-LRB-

Harris

and

Laibson

-RRB-

Using

a

probit

model

we

estimate

that

-LRB-

SE

-RRB-

and

-LRB-

SE

-RRB-

Once

again

we

can

not

reject

the

hypothesis

that

the

more

patient

system

does

not

discount

at

all

-LRB-

p

Wald

test

-RRB-

If

the

two

discounting

parameters

and

are

identical

then

this

model

is

equivalent

to

the

classic

model

of

single-exponential

discounting

We

can

reject

the

hypothesis

that

the

two

discounting

parameters

are

the

same

in

the

pooled

data

-LRB-

p

Wald

test

-RRB-

suggesting

that

a

two-system

model

is

consistent

with

the

behavioral

data

Brain

response

to

juice

and

water

We

identified

brain

regions

that

responded

to

juice

and

water

delivery

as

those

that

correlate

significantly

with

a

regressor

generated

by

convolving

delivery

times

with

a

hemodynamic

response

kernel

This

analysis

reveals

three

brain

regions

-LRB-

Fig.

-RRB-

The

first

is

bilateral

primary

taste

cortex

in

the

anterior

insula

The

second

is

a

region

in

the

premotor

area

-LRB-

PMA

-RRB-

most

likely

resulting

from

the

button

press

required

to

initiate

fluid

delivery

Finally

we

find

regions

bilaterally

in

the

ventral

striatum

that

are

also

activated

by

fluid

delivery

The

ventral

striatum

has

repeatedly

been

implicated

in

reward

processing

involving

both

money

-LRB-

Knutson

et

al.

-RRB-

and

primary

rewards

such

as

juice

-LRB-

McClure

et

al.

O'Doherty

et

al.

-RRB-

This

is

also

a

main

projection

site

of

midbrain

dopamine

neurons

that

are

known

to

convey

reward

information

-LRB-

Schultz

et

al.

-RRB-

To

assess

whether

reward-related

brain

responses

to

juice

and

water

are

significantly

different

we

extract

mean

event-related

activity

changes

from

the

ventral

striatum

bilaterally

-LRB-

Fig.

-RRB-

No

differences

in

the

amplitudes

of

brain

response

to

juice

and

water

are

found

in

this

analysis

-LRB-

cf.

Berns

et

al.

-RRB-

Identification

of

and

brain

systems

The

main

purpose

of

our

study

was

to

determine

whether

distinct

brain

regions

would

show

qualitatively

different

patterns

of

time

discounting

and

if

so

whether

the

division

of

regions

would

resemble

that

revealed

by

our

previous

research

To

avoid

imposing

our

hypotheses

on

the

data

we

do

not

use

a

region-of-interest

Table

Trials

with

min

delay

for

early

option

Early

option

Later

option

Delay

min

min

min

min

min

min

Magnitude

squirt

squirt

squirts

squirt

squirt

squirts

Delay

min

min

min

min

min

min

Magnitude

squirts

squirts

squirts

squirts

squirts

squirts

for

the

data

labeled

D

min

for

which

the

trials

in

Table

are

used

Figure

A

reveals

a

greater

tendency

to

select

the

earlier

reward

option

in

the

D

min

trials

compared

with

either

D

min

trials

-LRB-

paired

t

test

p

-RRB-

or

the

D

min

trials

-LRB-

p

-RRB-

Additionally

by

comparing

the

plots

for

the

and

min

differences

between

D

and

D

it

can

be

seen

that

the

immediacy

effect

is

driven

primarily

by

the

trials

with

min

differences

-LRB-

Fig.

B

-RRB-

We

next

fit

subjects

choice

data

to

both

the

discounting

model

-LRB-

Eq

-RRB-

which

assumes

that

the

system

conforms

to

a

discrete

-LRB-

-RRB-

function

and

the

more

flexible

formulation

rep

J.

Neurosci.

May

-LRB-

-RRB-

McClure

et

al.

Time

Discounting

analysis

to

identify

potential

and

brain

areas

but

instead

conduct

a

whole-brain

analysis

We

regress

voxel-level

neural

activity

-LRB-

i.e.

the

BOLD

signal

measured

in

each

voxel

-RRB-

on

control

variables

-LRB-

time

in

experiment

and

number

of

juice/water

squirts

in

previous

delivery

period

-RRB-

and

on

dummy

variables

that

represent

different

types

of

intertemporal

choices

As

described

previously

one

dummy

variable

identifies

intertemporal

choices

that

involve

a

decision

between

an

immediate

reward

and

a

delayed

reward

-LRB-

dummy

-RRB-

A

second

dummy

variable

identifies

all

intertemporal

choices

-LRB-

in

other

words

all

decision

epochs

-RRB-

-LRB-

dummy

-RRB-

Note

that

the

dummy

identifies

a

subset

of

the

decision

epochs

identified

by

the

dummy

and

that

brain

regions

load

on

the

regressor

to

the

degree

that

they

respond

preferentially

to

Figure

and

brain

areas

fMRI

data

were

fit

with

two

regressors

A

The

regressor

identified

those

brain

areas

that

are

intertemporal

choices

involving

immedi

preferentially

activated

by

choices

involving

a

reward

available

at

a

min

delay

Brain

areas

that

correlated

with

this

regressor

ate

rewards

Brain

areas

that

load

only

on

included

a

set

of

brain

areas

all

closely

linked

with

the

mesolimbic

dopamine

system

These

include

the

NAc

PCC

mOFC

and

ACC

the

regressor

do

not

respond

preferen

-LRB-

p

minimum

contiguous

voxels

-RRB-

PCu

Precuneus

B

The

second

-LRB-

-RRB-

regressor

identifies

brain

areas

that

are

activated

tially

to

the

presence

of

an

option

for

im

by

all

intertemporal

choices

This

regressor

identified

several

brain

areas

identified

with

general

cognitive

processing

including

the

PPar

bilaterally

the

Ant

Ins

and

regions

of

the

dorsolateral

prefrontal

cortex

-LSB-

Brodmann

area

-LRB-

-RRB-

and

mediate

reward

p

-RSB-

Vis

Ctx

Visual

cortex

We

identify

several

brain

regions

that

load

on

the

regressor

-LRB-

Fig.

A

-RRB-

These

identified

in

our

previous

study

of

intertemporal

choice

for

regions

include

the

nucleus

accumbens

-LRB-

NAc

-RRB-

subgenual

cingumoney

-LRB-

McClure

et

al.

-RRB-

To

directly

compare

the

responses

late

cortex

-LRB-

SGC

-RRB-

medial

orbitofrontal

cortex

-LRB-

mOFC

-RRB-

postein

this

and

our

previous

report

we

performed

a

conjunction

rior

cingulate

cortex

-LRB-

PCC

-RRB-

and

precuneus

As

demonstrated

analysis

to

identify

overlapping

voxels

-LRB-

Fig.

-RRB-

Among

the

arbelow

these

are

very

similar

to

the

areas

identified

by

a

similar

eas

every

region

of

activity

replicates

our

previous

findings

with

analysis

in

our

previous

study

-LRB-

McClure

et

al.

-RRB-

All

of

these

the

exception

of

the

additional

finding

of

a

rostral

area

of

the

regions

are

within

the

limbic

system

and

paralimbic

cortex

and

PCC

There

is

substantial

overlap

of

the

voxels

among

the

areas

have

been

directly

implicated

in

reward

processing

in

previous

at

p

and

even

greater

overlap

with

a

threshold

of

p

studies

-LRB-

Breiter

and

Rosen

Knutson

et

al.

-RRB-

The

Among

the

areas

almost

all

regions

of

activity

replicate

dummy

also

identifies

a

region

in

the

dorsal

anterior

cingulate

across

the

two

studies

but

unlike

the

regions

this

region-level

cortex

-LRB-

ACC

-RRB-

The

ACC

is

frequently

associated

with

the

presreplication

does

not

translate

into

voxel-level

replication

At

p

ence

of

response

conflict

-LRB-

Barch

et

al.

Botvinick

et

al.

only

seven

voxels

in

the

medial

prefrontal

cortex

are

-RRB-

Finding

ACC

activity

in

this

context

may

imply

that

sistent

across

the

two

studies

Two

of

these

voxels

are

in

the

SGC

choices

involving

an

immediate

reward

option

are

associated

and

the

other

five

are

in

the

dorsal

ACC

The

number

of

overlapwith

greater

conflict

Such

conflict

is

consistent

with

a

twoping

voxels

increases

somewhat

at

p

to

include

one

voxel

system

model

of

decision

making

in

the

PCC

and

one

voxel

in

the

NAc

-LRB-

Fig.

-RRB-

As

in

our

previous

study

regions

that

correlate

significantly

A

possible

interpretation

of

these

results

is

that

cognitive

-LRB-

-RRB-

with

the

dummy

included

both

visual

and

motor

areas

as

well

brain

regions

are

domain

general

and

hence

are

consistent

across

as

regions

commonly

associated

with

higher-level

cognitive

functasks

a

finding

that

has

been

reported

by

others

-LRB-

Shallice

tions

It

is

likely

that

areas

of

primary

visual

and

premotor

cortex

Duncan

Miller

and

Cohen

-RRB-

In

contrast

limbic

-LRB-

including

PMA

and

supplementary

motor

area

-RRB-

were

activated

reward-related

-LRB-

-RRB-

areas

may

be

more

stimulus

or

task

specific

because

task

performance

requires

subjects

to

look

at

a

visual

The

same

general

brain

regions

may

be

involved

in

a

variety

of

display

and

press

a

button

to

indicate

preference

The

remaining

functions

but

the

specific

subregions

involved

may

be

dependent

areas

we

find

to

correlate

with

the

dummy

include

a

region

in

on

reward

modality

time

scale

or

other

details

of

the

particular

the

PCC

bilateral

areas

in

the

posterior

parietal

cortex

-LRB-

PPar

-RRB-

task

circumstance

bilateral

areas

in

the

anterior

insula

-LRB-

Ant

Ins

-RRB-

and

several

regions

in

the

dorsolateral

prefrontal

cortex

-LRB-

DLPFC

-RRB-

-LRB-

Brodmann

areas

Neural

discounting

Our

analysis

has

shown

that

subjects

behavioral

choices

are

and

-RRB-

-LRB-

Fig.

B

-RRB-

Activity

in

DLPFC

and

PPar

is

obscribed

by

a

two-system

discounting

model

Our

analysis

of

the

served

commonly

in

tasks

involving

cognitive

processes

such

as

brain-imaging

data

has

also

identified

two

different

neural

sysworking

memory

abstract

problem

solving

and

exertion

of

contems

which

appear

to

be

associated

with

each

of

the

two

distrol

in

favor

of

long-term

goals

-LRB-

Miller

and

Cohen

-RRB-

The

counting

systems

To

test

this

relationship

more

directly

we

fit

region

in

the

PCC

lies

slightly

rostral

to

the

region

that

correlated

discount

functions

to

the

identified

and

brain

areas

respecwith

the

regressor

with

no

shared

voxels

tively

Under

the

hypothesis

that

the

and

brain

areas

generate

the

two

components

of

the

discount

function

that

we

estimated

Consistency

with

results

from

intertemporal

choice

for

money

The

areas

associated

with

the

and

regressors

henceforth

refrom

the

behavioral

data

we

may

expect

that

there

will

be

a

direct

ferred

to

as

and

areas

are

close

to

or

overlap

with

the

areas

correspondence

between

fits

to

behavior

and

fits

to

brain

activity

McClure

et

al.

Time

Discounting

J.

Neurosci.

May

-LRB-

-RRB-

-RRB-

and

implies

substantially

less

discounting

a

min

delay

is

discounted

by

only

For

both

areas

we

can

reject

the

hypothesis

that

the

discount

factor

is

equal

to

unity

-LRB-

p

-RRB-

However

the

dependence

on

time

delay

in

areas

seems

to

be

determined

by

only

one

of

the

areas

identified

in

the

brain

by

the

regressor

-LRB-

left

anterior

insula

-RRB-

When

the

discount

factors

estimated

for

each

of

the

regions

are

examined

individually

the

hypothesis

that

the

discount

factor

is

unity

can

be

rejected

only

for

the

left

insula

at

p

-LRB-

see

supplemental

material

available

at

-RRB-

that

is

all

of

the

others

fail

to

exhibit

significant

discounting

In

contrast

all

of

the

areas

exhibit

significant

discounting

Brain

activity

and

behavior

Finally

we

analyze

the

relationship

beFigure

Consistent

brain

areas

are

activated

for

intertemporal

choices

across

reward

modality

The

brain

areas

identified

in

tween

brain

activity

and

behavior

both

Figure

-LRB-

color-coded

red

-RRB-

are

nearly

the

same

as

those

identified

for

intertemporal

choices

for

monetary

reward

-LRB-

color-coded

controlling

for

and

not

controlling

for

the

green

-RRB-

A

For

areas

the

regions

are

consistent

across

studies

but

the

exact

voxels

are

not

At

p

only

seven

specific

parameters

of

choice

-LRB-

i.e.

reward

overlapping

voxels

-LRB-

yellow

-RRB-

were

found

to

be

activated

in

the

two

experiments

Even

at

a

liberal

threshold

of

p

only

one

amounts

and

time

delays

-RRB-

that

subjects

voxel

each

in

the

NAc

and

PCC

are

consistently

activated

across

reward

modality

B

In

contrast

there

is

significant

overlap

in

the

faced

in

a

particular

trial

We

use

the

averexact

voxels

found

in

the

two

studies

among

the

regions

at

p

with

even

greater

correspondence

at

p

These

age

of

the

multiplicatively

normed

activity

findings

support

the

hypothesis

that

areas

are

involved

in

largely

domain-general

cognitive

processes

areas

may

be

more

over

the

and

areas

as

done

previously

dependent

on

the

exact

nature

of

the

task

including

the

delay

to

reward

and

reward

modality

for

the

neural

discounting

reported

in

Table

and

estimate

a

probit

model

that

predicts

the

probability

that

a

subject

chooses

Table

Neural

discount

factors

the

early

option

in

a

given

trial

Discount

factor

SEa

We

first

estimate

a

model

in

which

choice

is

predicted

by

average

-LRB-

across

identified

brain

regions

-RRB-

activity

A

and

average

areas

areas

activity

A

with

basic

controls

for

subject

choice

number

-LRB-

a

-RRB-

and

recent

juice

squirts

In

this

regression

specification

SEs

are

clustered

by

subject

only

we

exclude

choices

in

which

-LRB-

D

D

-RRB-

-LRB-

-RRB-

This

is

because

our

imaging

analysis

suggests

that

the

system

responds

to

reAs

will

be

seen

the

fits

correspond

very

well

for

the

system

but

wards

at

both

of

these

delays

and

the

behavioral

data

indicate

not

for

the

system

We

discuss

possible

reasons

for

this

in

the

that

the

immediacy

effect

is

primarily

found

when

choosing

beDiscussion

tween

delays

of

and

min

We

estimated

an

exponential

decay

function

for

the

and

We

find

that

average

activity

in

both

and

brain

areas

is

areas

We

created

a

multiplicatively

normed

measure

of

activity

associated

with

choice

such

that

greater

activity

in

areas

and

for

each

brain

region

by

dividing

the

observed

amplitude

in

a

lesser

activity

in

areas

predict

a

greater

likelihood

of

choosing

given

trial

by

that

subject

's

average

amplitude

over

all

trials

for

the

sooner

lesser

option

-LRB-

Table

-RRB-

The

coefficient

-LRB-

coeff

-RRB-

on

A

the

brain

region

-LRB-

see

supplemental

materials

section

Discountis

-LRB-

p

-RRB-

indicating

that

higher

activity

in

areas

ing

in

individual

brain

areas

available

at

-RRB-

We

increases

the

probability

of

choosing

the

earlier

reward

In

addithen

averaged

these

normalized

measures

over

the

and

areas

tion

activity

in

the

areas

has

a

marginally

significant

negative

and

fit

these

activation

levels

with

exponentially

discounted

value

effect

on

choosing

early

-LRB-

coeff

p

-RRB-

However

D

D

functions

i

R

i

R

where

i

indexes

the

identified

brain

areas

average

and

area

activity

do

not

significantly

predict

choice

-LRB-

or

-RRB-

We

use

the

sum

of

the

discounted

values

because

we

once

we

control

for

the

characteristics

of

the

choice

with

the

assume

that

the

value

of

both

options

is

appraised

within

each

addition

of

controls

for

reward

size

and

delay

the

coefficient

on

trial

Table

presents

the

estimated

discount

factors

Note

that

A

also

becomes

insignificant

-LRB-

coeff

p

-RRB-

and

the

we

find

similar

results

when

we

estimate

the

discount

factors

coefficient

on

A

is

now

far

from

the

significance

threshold

D

D

D

without

reward

magnitude

-LRB-

i.e.

using

i

i

instead

of

i

R

-LRB-

coeff

p

-RRB-

This

indicates

that

although

average

D

i

R

-RRB-

as

well

as

when

we

include

task

controls

-LRB-

juice

squirts

area

activity

is

associated

with

choosing

the

earlier

reward

it

does

not

choice

number

and

response

time

-RRB-

independently

predict

which

reward

the

subject

will

choose

once

the

Although

both

discount

factors

appear

to

be

numerically

simcharacteristics

of

that

choice

are

taken

into

account

ilar

-LRB-

i.e.

both

are

-RRB-

in

fact

the

areas

demonstrate

a

substantially

greater

amount

of

discounting

Because

time

units

are

Experiment

framing

experiment

to

test

nature

measured

in

minutes

the

discount

factor

of

implies

that

a

of

immediacy

min

delay

is

discounted

by

This

represents

a

decay

in

value

relative

to

an

immediate

reward

The

area

The

brain

areas

identified

by

the

and

regressors

are

similar

to

discount

factor

is

significantly

higher

than

that

of

the

area

-LRB-

p

those

identified

in

our

previous

study

-LRB-

McClure

et

al.

-RRB-

This

J.

Neurosci.

May

-LRB-

-RRB-

McClure

et

al.

Time

Discounting

is

surprising

given

not

only

the

different

Table

Predicting

choice

based

on

brain

activity

probit

model

modality

of

rewards

in

the

two

studies

but

Basic

controls

Basic

controls

plus

reward

size

and

delay

also

the

different

time

scales

over

which

Coefficient

dF/dx

p

Coefficient

dF/dx

p

rewards

were

discounted

In

the

current

area

average

experiment

the

latest

rewards

were

delivarea

average

ered

was

at

a

min

delay

and

these

did

Basic

controls

include

indicator

variables

for

each

subject

choice

number

and

number

of

juice/water

squirts

in

the

previous

delivery

period

Coefficients

not

elicit

a

significant

response

in

the

indicate

the

effect

of

a

change

in

activity

on

the

probit

index

whereas

dF/dx

gives

the

marginal

effect

of

a

change

of

each

region

's

activity

at

its

mean

on

the

system

In

contrast

in

our

previous

study

probability

of

choosing

early

Statistical

significance

takes

into

account

clustering

of

standard

errors

by

subject

Choices

for

which

-LRB-

D

D

-RRB-

-LRB-

-RRB-

are

excluded

the

earliest

a

reward

could

be

received

was

from

these

regressions

after

the

experimental

session

had

ended

and

the

subject

had

a

chance

to

access

their

computer

-LRB-

gift

certifV

t

u

ct

icates

were

sent

by

e-mail

-RRB-

Nevertheless

this

was

adequate

to

u

ct

-LRB-

-RRB-

evoke

activity

in

regions

Thus

despite

the

fact

that

the

latest

system

system

juice/water

reward

-LRB-

in

the

present

study

-RRB-

was

available

sooner

than

the

earliest

monetary

reward

-LRB-

in

the

previous

study

-RRB-

the

where

This

function

has

the

important

property

that

former

did

not

elicit

activity

in

areas

whereas

the

latter

did

short-run

discount

rates

are

higher

than

long-run

aggregate

disThis

pattern

of

results

suggests

the

possibility

that

the

syscount

rates

because

the

relative

magnitude

of

the

-LRB-

impatient

-RRB-

tem

may

respond

not

to

absolute

time

delay

but

rather

to

the

system

is

greater

in

the

short

run

than

in

the

long

run

earliest

reward

available

in

the

overall

choice

set

and

hence

indeThere

is

a

long

history

of

research

in

psychology

suggesting

pendently

of

the

absolute

delay

until

reward

delivery

We

tested

that

decision

making

under

many

different

circumstances

reflects

this

hypothesis

in

a

second

experiment

in

which

the

delays

were

the

interaction

of

qualitatively

different

systems

Some

researchshifted

by

min

-LRB-

D

or

-RRB-

so

that

the

shortest

delay

to

ers

have

compared

emotional

-LRB-

or

affective

-RRB-

processes

and

delibreward

delivery

was

min

To

further

encourage

subjects

to

erative

-LRB-

or

analytic

-RRB-

processes

-LRB-

Loewenstein

Kahneman

consider

the

min

delay

as

the

most

immediate

option

we

-RRB-

whereas

others

have

contrasted

automatic

and

controlled

substituted

labels

for

each

delay

early

for

min

delay

options

processes

-LRB-

Posner

and

Snyder

Shiffrin

and

Schneider

and

middle

and

late

for

the

and

min

options

Cohen

et

al.

-RRB-

Both

distinctions

have

received

support

tively

All

of

these

labels

were

explained

to

subjects

before

the

from

research

using

diverse

methods

and

subjects

Such

distincexperiment

tions

have

also

begun

to

receive

support

from

neuroscientific

Contrary

to

our

hypothesis

subjects

did

not

treat

rewards

evidence

-LRB-

Sanfey

et

al.

-RRB-

For

example

LeDoux

-LRB-

-RRB-

finds

delayed

by

min

as

if

they

were

immediate

In

fact

they

were

that

activation

of

the

amygdala

in

response

to

fear

stimuli

arises

equally

likely

to

select

the

lesser

reward

option

when

it

was

availthrough

two

pathways

one

fast

stereotyped

and

driven

by

direct

able

early

-LRB-

i.e.

vs

min

-RRB-

as

when

it

was

available

after

a

sensory

projections

and

the

other

slower

more

goal-directed

medium

delay

-LRB-

i.e.

vs

min

p

-RRB-

-LRB-

data

not

shown

-RRB-

and

adaptive

Similar

findings

have

been

observed

in

human

deCorrespondingly

we

failed

to

observe

any

brain

areas

that

cision

making

using

functional

brain

imaging

-LRB-

Sanfey

et

al.

showed

a

greater

propensity

to

respond

to

early

versus

middle

Cohen

-RRB-

choices

than

for

middle

versus

late

choices

However

we

did

observe

the

same

set

of

brain

areas

responding

to

all

choices

as

Discounting

of

primary

and

secondary

rewards

in

experiment

-LRB-

data

not

shown

-RRB-

The

consistency

in

the

pattern

of

brain

activity

across

the

McClure

et

al.

-LRB-

-RRB-

study

and

the

current

study

is

intriguing

given

Discussion

the

change

in

reward

horizons

The

money

study

and

the

primary

We

use

functional

neuroimaging

to

study

the

brain

areas

inreward

study

both

identify

a

neural/behavioral

immediacy

efvolved

in

making

intertemporal

decisions

for

primary

rewards

fect

However

the

money

study

showed

this

effect

for

financial

Delays

of

just

a

few

minutes

lead

thirsty

subjects

to

substantially

rewards

that

were

received

at

the

earliest

in

h.

Moreover

the

discount

the

value

of

juice

and

water

Estimating

a

discount

funcsubjects

in

the

money

study

were

given

gift

certificates

for

tion

from

subjects

choices

we

find

that

subjects

discount

drink

Amazon.com

so

the

receipt

of

goods

took

at

least

d.

rewards

available

min

from

the

present

by

percent

relaIn

contrast

our

current

experiment

shows

that

choices

that

tive

to

rewards

available

immediately

In

contrast

subjects

disinclude

an

option

for

an

immediate

fluid

squirt

generate

a

limbic

play

little

or

no

discounting

when

both

rewards

under

considerresponse

but

that

choices

that

include

squirts

delayed

at

least

ation

are

delayed

-LRB-

e.g.

min

vs

min

-RRB-

min

fail

to

do

so

Why

should

the

system

respond

to

gift

certifIn

contrast

with

our

previous

report

-LRB-

McClure

et

al.

-RRB-

icates

delivered

in

over

h

when

it

does

not

respond

to

squirts

of

the

current

study

uses

a

different

modality

of

reward

-LRB-

fluid

vs

juice

or

water

delayed

by

or

min

money

-RRB-

and

substantially

different

time

period

of

discounting

Our

results

are

not

consistent

with

the

conjecture

which

mo

-LRB-

minutes

vs

weeks

-RRB-

Nevertheless

the

two

studies

identify

a

contivated

experiment

that

immediacy

is

a

relative

concept

framed

sistent

array

of

brain

areas

that

are

involved

in

discounting

Acby

the

circumstances

of

the

task

If

that

were

the

case

the

system

tivity

in

the

-LRB-

limbic

reward

-RRB-

system

decays

rapidly

as

opportuwould

have

responded

to

the

earliest

available

reward

in

the

nities

for

reward

are

delayed

The

-LRB-

frontal/parietal

-RRB-

system

is

framing

experiment

despite

the

min

time

delay

to

the

earliest

much

less

sensitive

to

the

timing

of

available

rewards

reward

The

findings

from

experiment

suggest

that

the

system

In

the

current

study

we

estimated

a

model

in

which

each

of

responds

to

the

absolute

rather

than

relative

value

of

delays

at

the

two

neural

systems

has

a

separate

continuous

exponential

least

for

rewards

of

juice

and

water

discount

function

-LRB-

this

is

a

generalization

of

the

quasihyperbolic

Perhaps

the

reason

that

the

system

was

unresponsive

to

even

model

-RRB-

In

this

generalization

the

integrated

discount

function

modest

delays

of

juice

and

water

is

that

these

are

primary

reinforcers

-LRB-

rewards

that

directly

satisfy

evolved

appetitive

mechatakes

the

following

form

McClure

et

al.

Time

Discounting

J.

Neurosci.

May

-LRB-

-RRB-

nisms

-RRB-

In

contrast

gift

certificates

are

secondary

reinforcers

that

may

evoke

limbic

activity

only

indirectly

mediated

by

more

abstract

symbolic

and/or

associative

processes

that

may

be

more

susceptible

to

contextual

framing

effects

That

is

limbic

activity

in

response

to

primary

reinforcers

may

follow

a

rigid

discounting

function

that

has

adapted

under

evolutionary

pressures

to

physiological

needs

as

well

as

specific

features

of

the

environment

whereas

the

discounting

function

for

secondary

reinforcers

may

be

more

context

dependent

Primary

rewards

are

likely

to

have

reasonably

well

defined

and

stable

temporal

characteristics

-LRB-

e.g.

the

rate

of

body

water

loss

or

the

deterioration

of

food

goods

-RRB-

and

therefore

are

amenable

to

discount

rates

expressed

in

absolute

time

-LRB-

although

these

may

vary

according

to

the

type

of

reward

changes

in

internal

states

such

as

satiety

and

known

features

of

the

environment

such

as

temperature

-RRB-

In

contrast

the

value

and

timing

characteristics

of

secondary

reinforcers

may

be

more

variable

and

dependent

on

context

and

therefore

better

served

by

a

more

flexible

system

for

temporal

discounting

that

is

sensitive

to

relative

value

These

possibilities

suggest

a

number

of

predictions

-LRB-

for

instance

that

money

rewards

should

be

subject

to

framing

effects

in

a

way

that

primary

rewards

are

not

-RRB-

that

remain

to

be

tested

in

future

experiments

Difference

in

fit

of

discount

function

to

behavior

and

brain

activity

Equation

above

suggests

the

simple

hypothesis

that

the

system

and

the

system

combine

additively

to

determine

the

value

of

a

delayed

reward

The

double-exponential

discount

function

would

result

from

this

additive

combination

if

areas

and

areas

are

each

characterized

by

exponential

discount

functions

with

different

discount

factors

In

our

statistical

analysis

we

fit

the

double-exponential

discount

function

to

behavior

and

single-exponential

discount

functions

to

brain

activity

in

individual

brain

areas

Because

the

behavioral

and

neural

data

sets

are

distinct

the

resulting

estimates

present

a

new

test

of

our

model

do

the

behavioral

estimates

of

the

double-exponential

discount

function

match

the

neural

activation

estimates

of

the

exponential

discount

functions

in

the

and

regions

In

the

case

of

the

system

there

is

a

good

correspondence

In

fitting

the

behavioral

data

with

a

double-exponential

discount

factor

we

were

unable

to

reject

the

hypothesis

that

there

is

no

discounting

in

the

system

-LRB-

i.e.

the

larger

discount

factor

in

the

double-exponential

function

is

statistically

indistinguishable

from

unity

-RRB-

Likewise

in

fitting

the

neural

activation

of

individual

brain

areas

only

one

brain

area

-LRB-

the

anterior

insula

-RRB-

showed

a

significant

difference

from

unity

whereas

of

the

remaining

ones

did

so

However

the

discount

factors

extracted

for

the

myopic

system

did

not

quantitatively

match

in

the

behavioral

and

neuroimaging

analyses

Fitting

to

behavioral

data

with

a

doubleexponential

discount

function

we

estimate

a

discount

factor

in

the

system

equal

to

-LRB-

i.e.

the

smaller

discount

factor

in

the

double-exponential

function

is

-RRB-

However

the

fit

to

neural

activity

in

brain

areas

produced

a

discount

factor

of

These

two

numbers

do

not

quantitatively

match

However

a

discount

factor

of

min

still

implies

a

high

rate

of

discounting

Because

value

declines

by

if

the

delay

is

only

min

Although

a

direct

correspondence

between

discounting

behavior

and

discounting

of

brain

activation

would

be

elegant

there

is

at

least

one

simple

reason

why

it

might

not

be

observed

Our

identified

brain

regions

likely

perform

functions

other

than

valuing

the

rewards

in

our

experiment

Averaging

other

func

tions

with

valuation

will

tend

to

bias

estimates

of

region

discounting

toward

-LRB-

i.e.

toward

being

uniformly

responsive

to

all

trials

-RRB-

Intertemporal

choice

and

conflict

monitoring

We

have

observed

that

choices

between

lesser

immediate

and

greater

delayed

rewards

elicit

activity

in

distinct

neural

systems

that

appear

to

favor

different

choice

outcomes

That

is

intertemporal

choice

under

these

conditions

elicits

decisional

conflict

A

growing

body

of

evidence

suggests

that

a

dorsocaudal

region

of

the

ACC

responds

to

conflicts

in

processing

-LRB-

Carter

et

al.

Botvinick

et

al.

Yeung

et

al.

-RRB-

This

is

consistent

with

findings

from

the

current

study

in

which

we

observed

activity

in

a

similar

area

of

the

ACC

that

was

greater

for

decisions

involving

choices

between

immediate

and

delayed

rewards

than

for

choices

between

only

delayed

rewards

Such

findings

have

been

taken

as

evidence

for

a

conflict-monitoring

function

of

ACC

which

serves

to

detect

conditions

requiring

the

recruitment

of

cognitive

control

mechanisms

subserved

by

prefrontal

cortex

and

associated

structures

-LRB-

Botvinick

et

al.

Kerns

et

al.

-RRB-

Our

observation

of

an

association

of

ACC

activity

with

choices

between

an

immediate

and

delayed

reward

-LRB-

consistent

with

its

role

in

conflict

monitoring

-RRB-

and

an

association

of

PFC

activity

with

choices

favoring

delayed

rewards

-LRB-

consistent

with

its

role

in

executing

cognitive

control

-RRB-

are

consistent

with

existing

theory

regarding

the

functions

of

these

neural

systems

This

in

turn

provides

a

framework

within

which

to

generate

quantitative

predictions

about

the

dynamics

of

the

neural

mechanisms

underlying

intertemporal

choice

Such

studies

-LRB-

that

join

economic

theory

with

hypotheses

about

and

measurements

of

mechanism

from

neuroscience

-RRB-

remain

a

priority

for

future

work

References

Ainslie

G

-LRB-

-RRB-

Specious

reward

a

behavioral

theory

of

impulsiveness

and

impulse

control

Psychol

Bull

Ainslie

G

Monterosso

J

-LRB-

-RRB-

Behavior

A

marketplace

in

the

brain

Science

Barch

DM

Braver

TS

Akbudak

E

Conturo

T

Ollinger

J

Snyder

A

-LRB-

-RRB-

Anterior

cingulate

cortex

and

response

conflict

effects

of

response

modality

and

processing

domain

Cereb

Cortex

Baumeister

RF

Vols

KD

-LRB-

-RRB-

Willpower

choice

and

self-control

In

Time

and

decision

-LRB-

Loewenstein

G

Read

D

Baumeister

RF

eds

-RRB-

New

York

Russell

Sage

Foundation

Berns

GS

McClure

SM

Pagnoni

G

Montague

PR

-LRB-

-RRB-

Predictability

modulates

human

brain

response

to

reward

J

Neurosci

Botvinick

MM

Braver

TS

Barch

DM

Carter

CS

Cohen

JD

-LRB-

-RRB-

Conflict

monitoring

and

cognitive

control

Psychol

Rev

Botvinick

MM

Cohen

JD

Carter

CS

-LRB-

-RRB-

Conflict

monitoring

and

anterior

cingulate

cortex

an

update

Trends

Cogn

Sci

Breiter

HC

Rosen

BR

-LRB-

-RRB-

Functional

magnetic

resonance

imaging

of

brain

reward

circuitry

in

the

human

Ann

NY

Acad

Sci

Carter

CS

Braver

TS

Barch

DM

Botvinick

MM

Noll

D

Cohen

JD

-LRB-

-RRB-

Anterior

cingulate

cortex

error

detection

and

the

online

monitoring

of

performance

Science

Cohen

JD

Dunbar

K

McClelland

JL

-LRB-

-RRB-

On

the

control

of

automatic

processes

a

parallel

distributed

processing

account

of

the

Stroop

effect

Psychol

Rev

Cohen

MS

-LRB-

-RRB-

Parametric

analysis

of

fMRI

data

using

linear

systems

methods

NeuroImage

Cox

RW

-LRB-

-RRB-

AFNI

software

for

analysis

and

visualization

of

functional

magnetic

resonance

neuroimages

Comput

Biomed

Res

Cubitt

R

Read

D

-LRB-

-RRB-

Can

intertemporal

choice

experiments

elicit

time

preferences

for

consumption

Discussion

Papers

Nottingham

UK

The

Centre

for

Decision

Research

and

Experimental

Economics

School

of

Economics

University

of

Nottingham

Duncan

J

-LRB-

-RRB-

Disorganization

of

behavior

after

frontal

lobe

damage

Cogn

Neuropsychol

Hariri

AR

Brown

SM

Williamson

DE

Flory

JD

de

Wit

H

Manuck

SB

J.

Neurosci.

May

-LRB-

-RRB-

-LRB-

-RRB-

Preference

for

immediate

over

delayed

rewards

is

associated

with

magnitude

of

ventral

striatal

activity

J

Neurosci

Harris

CJ

Laibson

DI

-LRB-

-RRB-

Instantaneous

gratification

Harvard

mimeo

Kahneman

D

-LRB-

-RRB-

Maps

of

bounded

rationality

psychology

for

behavioral

economics

Am

Econ

Rev

Kerns

JG

Cohen

JD

MacDonald

III

AW

Cho

RY

Stenger

VA

Carter

CS

-LRB-

-RRB-

Anterior

cingulate

conflict

monitoring

and

adjustments

in

control

Science

Kirby

K

-LRB-

-RRB-

Bidding

on

the

future

evidence

against

normative

discounting

of

delayed

rewards

J

Exp

Psychol

Gen

Knutson

B

Fong

GW

Adams

CM

Varner

JL

Hommer

D

-LRB-

-RRB-

Dissociation

of

reward

anticipation

and

outcome

with

event-related

fMRI

NeuroReport

Laibson

DI

-LRB-

-RRB-

Golden

eggs

and

hyperbolic

discounting

Q

J

Econ

LeDoux

JE

-LRB-

-RRB-

The

emotional

brain

the

mysterious

underpinnings

of

emotional

life

New

York

Simon

and

Schuster

Lhermitte

F

-LRB-

-RRB-

Human

autonomy

and

the

frontal

lobes

Patient

behavior

in

complex

and

social

situations

the

environmental

dependency

syndrome

Ann

Neurol

Loewenstein

G

-LRB-

-RRB-

Out

of

control

visceral

influences

on

behavior

Org

Beh

Hum

Dec

Proc

Loewenstein

G

Prelec

D

-LRB-

-RRB-

Anomalies

in

intertemporal

choice

evidence

and

an

interpretation

Q

J

Econ

McClure

SM

Berns

GS

Montague

PR

-LRB-

-RRB-

Temporal

prediction

errors

in

a

passive

learning

task

activate

human

striatum

Neuron

McClure

SM

Laibson

DI

Loewenstein

G

Cohen

JD

-LRB-

-RRB-

Separate

neural

systems

value

immediate

and

delayed

monetary

rewards

Science

McClure

et

al.

Time

Discounting

Miller

EK

Cohen

JD

-LRB-

-RRB-

An

integrative

theory

of

prefrontal

cortex

function

Annu

Rev

Neurosci

O'Doherty

JP

Dayan

P

Friston

K

Critchley

H

Dolan

RJ

-LRB-

-RRB-

Temporal

difference

models

and

reward-related

learning

in

the

human

brain

Neuron

Posner

MI

Snyder

CRR

-LRB-

-RRB-

Attention

and

cognitive

control

In

Information

processing

and

cognition

The

Loyola

Symposium

-LRB-

Solso

RL

ed

-RRB-

Hillsdale

NJ

Erlbaum

Sanfey

AG

Rilling

JK

Aronson

JA

Nystrom

LE

Cohen

JD

-LRB-

-RRB-

The

neural

basis

of

economic

decision-making

in

the

Ultimatum

Game

Science

Sanfey

AG

Loewenstein

G

McClure

SM

Cohen

JD

-LRB-

-RRB-

Neuroeconomics

cross-currents

in

research

on

decision-making

Trends

Cogn

Sci

Schultz

W

Dayan

P

Montague

PR

-LRB-

-RRB-

A

neural

substrate

of

prediction

and

reward

Science

Shallice

T

-LRB-

-RRB-

Specific

impairments

in

planning

Phil

Trans

R

Soc

Lond

B

Biol

Sci

Shiffrin

RM

Schneider

W

-LRB-

-RRB-

Controlled

and

automatic

human

information

processing

II

Perceptual

learning

automaticity

attending

and

a

general

theory

Psychol

Rev

Tanaka

SC

Doya

K

Okada

G

Ueda

K

Okamoto

Y

Yamawaki

S

-LRB-

-RRB-

Prediction

of

immediate

and

future

rewards

differentially

recruits

cortico-basal

ganglia

loops

Nat

Neurosci

Thaler

RH

-LRB-

-RRB-

Some

empirical

evidence

on

time

inconsistency

Rev

Econ

Stud

Yeung

N

Botvinick

MM

Cohen

JD

-LRB-

-RRB-

The

neural

basis

of

error

detection

conflict

monitoring

and

the

error-related

negativity

Psychol

Rev

fmri neuroeconomics discounting economics human