Published

April

JCB

FEATURE

Error

bars

in

experimental

biology

Geoff

Cumming

Fiona

Fidler

and

David

L.

School

of

Psychological

Science

and

of

Biochemistry

La

Trobe

University

Melbourne

Victoria

Australia

THE

JOURNAL

OF

CELL

BIOLOGY

Error

bars

commonly

appear

in

figures

in

publications

but

experimental

biologists

are

often

unsure

how

they

should

be

used

and

interpreted

In

this

article

we

illustrate

some

basic

features

of

error

bars

and

explain

how

they

can

help

communicate

data

and

assist

correct

interpretation

Error

bars

may

show

confidence

intervals

standard

errors

standard

deviations

or

other

quantities

Different

types

of

error

bars

give

quite

different

information

and

so

figure

legends

must

make

clear

what

error

bars

represent

We

suggest

eight

simple

rules

to

assist

with

effective

use

and

interpretation

of

error

bars

What

are

error

bars

for

error

bars

encompass

the

lowest

and

highest

values

SD

is

calculated

by

the

formula

SD

-LRB-

X

M

-RRB-

n

where

X

refers

to

the

individual

data

points

M

is

the

mean

and

-LRB-

sigma

-RRB-

means

add

to

find

the

sum

for

all

the

n

data

points

SD

is

roughly

the

average

or

typical

difference

between

the

data

points

and

their

mean

M

About

two

thirds

of

the

data

points

will

lie

within

the

region

of

mean

SD

and

of

the

data

points

will

be

within

SD

of

the

mean

Downloaded

from

on

May

Journals

that

publish

science

knowledge

gained

through

repeated

observation

or

experiment

do

n't

just

present

new

conclusions

they

also

present

evidence

so

readers

can

verify

that

the

authors

reasoning

is

correct

Figures

with

error

bars

can

if

used

properly

-LRB-

-RRB-

give

information

describing

the

data

-LRB-

descriptive

statistics

-RRB-

or

information

about

what

conclusions

or

inferences

are

justified

-LRB-

inferential

statistics

-RRB-

These

two

basic

categories

of

error

bars

are

depicted

in

exactly

the

same

way

but

are

actually

fundamentally

different

Our

aim

is

to

illustrate

basic

properties

of

figures

with

any

of

the

common

error

bars

as

summarized

in

Table

I

and

to

explain

how

they

should

be

used

What

do

error

bars

tell

you

Descriptive

error

bars

Range

and

standard

deviation

-LRB-

SD

-RRB-

are

used

for

descriptive

error

bars

because

they

show

how

the

data

are

spread

-LRB-

Fig.

-RRB-

Range

David

L.

Vaux

d.vaux@latrobe.edu.au

It

is

highly

desirable

to

use

larger

n

to

achieve

narrower

inferential

error

bars

and

more

precise

estimates

of

true

population

values

Descriptive

error

bars

can

also

be

used

to

see

whether

a

single

result

fits

within

the

normal

range

For

example

if

you

wished

to

see

if

a

red

blood

cell

count

was

normal

you

could

see

whether

it

was

within

SD

of

the

mean

of

the

population

as

a

whole

Less

than

of

all

red

blood

cell

counts

are

more

than

SD

from

the

mean

so

if

the

count

in

question

is

more

than

SD

from

the

mean

you

might

consider

it

to

be

abnormal

As

you

increase

the

size

of

your

sample

or

repeat

the

experiment

more

times

the

mean

of

your

results

-LRB-

M

-RRB-

will

tend

to

get

closer

and

closer

to

the

true

mean

or

the

mean

of

the

whole

population

We

can

use

M

as

our

best

estimate

of

the

unknown

Similarly

as

you

repeat

an

experiment

more

and

more

times

the

Figure

Descriptive

error

bars

Means

with

error

bars

for

three

cases

n

n

and

n

The

small

black

dots

are

data

points

and

the

column

denotes

the

data

mean

M

The

bars

on

the

left

of

each

column

show

range

and

the

bars

on

the

right

show

standard

deviation

-LRB-

SD

-RRB-

M

and

SD

are

the

same

for

every

case

but

notice

how

much

the

range

increases

with

n.

Note

also

that

although

the

range

error

bars

encompass

all

of

the

experimental

results

they

do

not

necessarily

cover

all

the

results

that

could

possibly

occur

SD

error

bars

include

about

two

thirds

of

the

sample

and

x

SD

error

bars

would

encompass

roughly

of

the

sample

SD

of

your

results

will

tend

to

more

and

more

closely

approximate

the

true

standard

deviation

-LRB-

-RRB-

that

you

would

get

if

the

experiment

was

performed

an

infinite

number

of

times

or

on

the

whole

population

However

the

SD

of

the

experimental

results

will

approximate

to

whether

n

is

large

or

small

Like

M

SD

does

not

change

systematically

as

n

changes

and

we

can

use

SD

as

our

best

estimate

of

the

unknown

whatever

the

value

of

n.

Inferential

error

bars

In

experimental

biology

it

is

more

common

to

be

interested

in

comparing

samples

from

two

groups

to

see

if

they

are

different

For

example

you

might

be

comparing

wildtype

mice

with

mutant

mice

or

drug

with

The

Rockefeller

University

Press

The

Journal

of

Cell

Biology

Vol

No.

April

JCB

Published

April

-LRB-

Fig.

-RRB-

The

interval

defines

the

values

that

are

most

plausible

for

Because

error

bars

can

be

descriptive

or

inferential

and

could

be

any

of

the

bars

listed

in

Table

I

or

even

something

else

they

are

meaningless

or

misleading

if

the

figure

legend

does

not

state

what

kind

they

are

This

leads

to

the

first

rule

Rule

when

showing

error

bars

always

describe

in

the

figure

legends

what

they

are

Statistical

significance

tests

and

P

values

Figure

Confidence

intervals

Means

and

CIs

for

independent

sets

of

results

each

of

size

n

from

a

population

with

mean

-LRB-

marked

by

the

dotted

line

-RRB-

In

the

long

run

we

expect

of

such

CIs

to

capture

here

do

so

-LRB-

large

black

dots

-RRB-

and

do

not

-LRB-

open

circles

-RRB-

Successive

CIs

vary

considerably

not

only

in

position

relative

to

but

also

in

length

The

variation

from

CI

to

CI

would

be

less

for

larger

sets

of

results

for

example

n

or

more

but

variation

in

position

and

in

CI

length

would

be

even

greater

for

smaller

samples

for

example

n

placebo

or

experimental

results

with

controls

To

make

inferences

from

the

data

-LRB-

i.e.

to

make

a

judgment

whether

the

groups

are

significantly

different

or

whether

the

differences

might

just

be

due

to

random

fluctuation

or

chance

-RRB-

a

different

type

of

error

bar

can

be

used

These

are

standard

error

-LRB-

SE

-RRB-

bars

and

confidence

intervals

-LRB-

CIs

-RRB-

The

mean

of

the

data

M

with

SE

or

CI

error

bars

gives

an

indication

of

the

region

where

you

can

expect

the

mean

of

the

whole

possible

set

of

results

or

the

whole

population

to

lie

Table

I.

Common

error

bars

Error

bar

Range

Standard

deviation

-LRB-

SD

-RRB-

Type

Descriptive

Descriptive

If

you

carry

out

a

statistical

significance

test

the

result

is

a

P

value

where

P

is

the

probability

that

if

there

really

is

no

difference

you

would

get

by

chance

a

difference

as

large

as

the

one

you

observed

or

even

larger

Other

things

-LRB-

e.g.

sample

size

variation

-RRB-

being

equal

a

larger

difference

in

results

gives

a

lower

P

value

which

makes

you

suspect

there

is

a

true

difference

By

convention

if

P

you

say

the

result

is

statistically

significant

and

if

P

you

say

the

result

is

highly

significant

and

you

can

be

more

confident

you

have

found

a

true

effect

As

always

with

statistical

inference

you

may

be

wrong

Perhaps

there

really

is

no

effect

and

you

had

the

bad

luck

to

get

one

of

the

-LRB-

if

P

-RRB-

or

-LRB-

if

P

-RRB-

of

sets

of

results

that

suggests

a

difference

where

there

is

Of

course

even

if

results

are

statistically

highly

significant

it

does

not

mean

they

are

necessarily

biologically

important

It

is

also

essential

to

note

that

if

P

and

you

therefore

can

not

conclude

there

is

a

statistically

significant

effect

you

may

not

conclude

that

the

effect

is

zero

There

may

be

a

real

effect

but

it

is

small

or

you

may

not

have

Figure

Inappropriate

use

of

error

bars

Enzyme

activity

for

MEFs

showing

mean

SD

from

duplicate

samples

from

one

of

three

representative

experiments

Values

for

wild-type

vs.

MEFs

were

significant

for

enzyme

activity

at

the

timepoint

-LRB-

P

-RRB-

This

figure

and

its

legend

are

typical

but

illustrate

inappropriate

and

misleading

use

of

statistics

because

n

The

very

low

variation

of

the

duplicate

samples

implies

consistency

of

pipetting

but

says

nothing

about

whether

the

differences

between

the

wildtype

and

MEFs

are

reproducible

In

this

case

the

means

and

errors

of

the

three

experiments

should

have

been

shown

Downloaded

from

on

May

repeated

your

experiment

often

enough

to

reveal

it

It

is

a

common

and

serious

error

to

conclude

no

effect

exists

just

because

P

is

greater

than

If

you

measured

the

heights

of

three

male

and

three

female

Biddelonian

basketball

players

and

did

not

see

a

significant

difference

you

could

not

conclude

that

sex

has

no

relationship

with

height

as

a

larger

sample

size

might

reveal

one

A

big

advantage

of

inferential

error

bars

is

that

their

length

gives

a

graphic

signal

of

how

much

uncertainty

there

is

in

the

data

The

true

value

of

the

mean

we

are

estimating

could

plausibly

be

anywhere

in

the

CI

Wide

inferential

bars

indicate

large

error

short

inferential

bars

indicate

high

precision

Replicates

or

independent

samples

what

is

n

Description

Amount

of

spread

between

the

extremes

of

the

data

Typical

or

-LRB-

roughly

speaking

-RRB-

average

difference

between

the

data

points

and

their

mean

A

measure

of

how

variable

the

mean

will

be

if

you

repeat

the

whole

study

many

times

A

range

of

values

you

can

be

confident

contains

the

true

mean

Formula

Highest

data

point

minus

the

lowest

SD

-LRB-

X

M

-RRB-

n

Standard

error

-LRB-

SE

-RRB-

Inferential

SE

SD/n

Confidence

interval

-LRB-

CI

-RRB-

usually

CI

Inferential

M

t

-LRB-

n

-RRB-

SE

where

t

-LRB-

n

-RRB-

is

a

critical

value

of

t

If

n

is

or

more

the

CI

is

approximately

M

SE

Science

typically

copes

with

the

wide

variation

that

occurs

in

nature

by

measuring

a

number

-LRB-

n

-RRB-

of

independently

sampled

individuals

independently

conducted

experiments

or

independent

observations

Rule

the

value

of

n

-LRB-

i.e.

the

sample

size

or

the

number

of

independently

performed

experiments

-RRB-

must

be

stated

in

the

figure

legend

It

is

essential

that

n

-LRB-

the

number

of

independent

results

-RRB-

is

carefully

distinguished

from

the

number

of

replicates

JCB

VOLUME

NUMBER

Published

April

Figure

Inferential

error

bars

Means

with

SE

and

CI

error

bars

for

three

cases

ranging

in

size

from

n

to

n

with

descriptive

SD

bars

shown

for

comparison

The

small

black

dots

are

data

points

and

the

large

dots

indicate

the

data

mean

M.

For

each

case

the

error

bars

on

the

left

show

SD

those

in

the

middle

show

CI

and

those

on

the

right

show

SE

Note

that

SD

does

not

change

whereas

the

SE

bars

and

CI

both

decrease

as

n

gets

larger

The

ratio

of

CI

to

SE

is

the

t

statistic

for

that

n

and

changes

with

n.

Values

of

t

are

shown

at

the

bottom

For

each

case

we

can

be

confident

that

the

CI

includes

the

true

mean

The

likelihood

that

the

SE

bars

capture

varies

depending

on

n

and

is

lower

for

n

-LRB-

for

such

low

values

of

n

it

is

better

to

simply

plot

the

data

points

rather

than

showing

error

bars

as

we

have

done

here

for

illustrative

purposes

-RRB-

volume

of

cells

from

the

same

stock

culture

into

adjacent

wells

of

a

tissue

culture

plate

and

subsequently

treating

them

identically

Although

it

would

be

possible

to

assay

the

plate

and

determine

the

means

and

errors

of

the

replicate

wells

the

errors

would

reflect

the

accuracy

of

pipetting

not

the

reproduciblity

of

the

differences

between

the

experimental

cells

and

the

control

cells

For

replicates

n

and

it

is

therefore

inappropriate

to

show

error

bars

or

statistics

If

an

experiment

involves

triplicate

cultures

and

is

repeated

four

independent

times

then

n

not

or

The

variation

within

each

set

of

triplicates

is

related

to

the

fidelity

with

which

the

replicates

were

created

and

is

irrelevant

to

the

hypothesis

being

tested

To

identify

the

appropriate

value

for

n

think

of

what

entire

population

is

being

sampled

or

what

the

entire

set

of

experiments

would

be

if

all

possible

ones

of

that

type

were

performed

Conclusions

can

be

drawn

only

about

that

population

so

make

sure

it

is

appropriate

to

the

question

the

research

is

intended

to

answer

In

the

example

of

replicate

cultures

from

the

one

stock

of

cells

the

population

being

sampled

is

the

stock

cell

culture

For

n

to

be

greater

than

the

experiment

would

have

to

be

performed

using

separate

stock

cultures

or

separate

cell

clones

of

the

same

type

Again

consider

the

population

you

wish

to

make

inferences

about

it

is

unlikely

to

be

just

a

single

stock

culture

Whenever

you

see

a

figure

with

very

small

error

bars

-LRB-

such

as

Fig.

-RRB-

you

should

ask

yourself

whether

the

very

small

variation

implied

by

the

error

bars

is

due

to

analysis

of

replicates

rather

than

independent

samples

If

so

the

bars

are

useless

for

making

the

inference

you

are

considering

Sometimes

a

figure

shows

only

the

data

for

a

representative

experiment

implying

that

several

other

similar

experiments

were

also

conducted

If

a

representative

experiment

is

shown

then

n

and

no

error

bars

or

P

values

should

be

shown

Instead

the

means

and

errors

of

all

the

independent

experiments

should

be

given

where

n

is

the

number

of

experiments

performed

Rule

error

bars

and

statistics

should

only

be

shown

for

independently

repeated

experiments

and

never

for

replicates

If

a

representative

experiment

is

shown

it

should

not

have

error

bars

or

P

values

because

in

such

an

experiment

n

-LRB-

Fig.

shows

what

not

to

do

-RRB-

Downloaded

from

on

May

which

refers

to

repetition

of

measurement

on

one

individual

in

a

single

condition

or

multiple

measurements

of

the

same

or

identical

samples

Consider

trying

to

determine

whether

deletion

of

a

gene

in

mice

affects

tail

length

We

could

choose

one

mutant

mouse

and

one

wild

type

and

perform

replicate

measurements

of

each

of

their

tails

We

could

calculate

the

means

SDs

and

SEs

of

the

replicate

measurements

but

these

would

not

permit

us

to

answer

the

central

question

of

whether

gene

deletion

affects

tail

length

because

n

would

equal

for

each

genotype

no

matter

how

often

each

tail

was

measured

To

address

the

question

successfully

we

must

distinguish

the

possible

effect

of

gene

deletion

from

natural

animal-toanimal

variation

and

to

do

this

we

need

to

measure

the

tail

lengths

of

a

number

of

mice

including

several

mutants

and

several

wild

types

with

n

for

each

type

Similarly

a

number

of

replicate

cell

cultures

can

be

made

by

pipetting

the

same

Figure

Estimating

statistical

significance

using

the

overlap

rule

for

SE

bars

Here

SE

bars

are

shown

on

two

separate

means

for

control

results

C

and

experimental

results

E

when

n

is

-LRB-

left

-RRB-

or

n

is

or

more

-LRB-

right

-RRB-

Gap

refers

to

the

number

of

error

bar

arms

that

would

fit

between

the

bottom

of

the

error

bars

on

the

controls

and

the

top

of

the

bars

on

the

experimental

results

i.e.

a

gap

of

means

the

distance

between

the

C

and

E

error

bars

is

equal

to

twice

the

average

of

the

SEs

for

the

two

samples

When

n

and

double

the

length

of

the

SE

error

bars

just

touch

-LRB-

i.e.

the

gap

is

SEs

-RRB-

P

is

-LRB-

we

do

n't

recommend

using

error

bars

where

n

or

some

other

very

small

value

but

we

include

rules

to

help

the

reader

interpret

such

figures

which

are

common

in

experimental

biology

-RRB-

ERROR

BARS

IN

EXPERIMENTAL

BIOLOGY

CUMMING

ET

AL.

Published

April

Figure

Estimating

statistical

significance

using

the

overlap

rule

for

CI

bars

Here

CI

bars

are

shown

on

two

separate

means

for

control

results

C

and

experimental

results

E

when

n

is

-LRB-

left

-RRB-

or

n

is

or

more

-LRB-

right

-RRB-

Overlap

refers

to

the

fraction

of

the

average

CI

error

bar

arm

i.e.

the

average

of

the

control

-LRB-

C

-RRB-

and

experimental

-LRB-

E

-RRB-

arms

When

n

if

CI

error

bars

overlap

by

half

the

average

arm

length

P

If

the

tips

of

the

error

bars

just

touch

P

What

type

of

error

bar

should

be

used

Rule

because

experimental

biologists

are

usually

trying

to

compare

experimental

results

with

controls

it

is

usually

appropriate

to

show

inferential

error

bars

such

as

SE

or

CI

rather

than

SD

However

if

n

is

very

small

-LRB-

for

example

n

-RRB-

rather

than

showing

error

bars

and

statistics

it

is

better

to

simply

plot

the

individual

data

points

What

is

the

difference

between

SE

bars

and

CIs

Standard

error

-LRB-

SE

-RRB-

Suppose

three

experiments

gave

measurements

of

and

which

are

the

data

points

in

the

n

case

at

the

left

in

Fig.

The

mean

of

the

data

is

M

and

the

SD

which

is

the

length

of

each

arm

of

the

SD

bars

M

-LRB-

in

this

case

-RRB-

is

the

best

estimate

of

the

true

mean

that

we

would

like

to

know

But

how

accurate

an

estimate

is

it

This

can

be

shown

by

inferential

error

bars

such

as

standard

error

-LRB-

SE

sometimes

referred

to

as

the

standard

error

of

the

mean

SEM

-RRB-

or

a

confidence

interval

-LRB-

CI

-RRB-

SE

is

defined

as

SE

SD/n

In

Fig.

the

large

dots

mark

the

means

of

the

same

three

samples

as

in

Fig.

For

the

n

case

SE

JCB

VOLUME

NUMBER

and

this

is

the

length

of

each

arm

of

the

SE

bars

shown

The

SE

varies

inversely

with

the

square

root

of

n

so

the

more

often

an

experiment

is

repeated

or

the

more

samples

are

measured

the

smaller

the

SE

becomes

-LRB-

Fig.

-RRB-

This

allows

more

and

more

accurate

estimates

of

the

true

mean

by

the

mean

of

the

experimental

results

M

We

illustrate

and

give

rules

for

n

not

because

we

recommend

using

such

a

small

n

but

because

researchers

currently

often

use

such

small

n

values

and

it

is

necessary

to

be

able

to

interpret

their

papers

It

is

highly

desirable

to

use

larger

n

to

achieve

narrower

inferential

error

bars

and

more

precise

estimates

of

true

population

values

Confidence

interval

-LRB-

CI

-RRB-

Fig.

illustrates

what

happens

if

hypothetically

different

labs

performed

the

same

experiments

with

n

in

each

case

The

CI

error

bars

are

approximately

M

and

they

vary

in

position

because

of

course

M

varies

from

lab

to

lab

and

they

also

vary

in

width

because

SE

varies

Such

error

bars

capture

the

true

mean

on

of

occasions

in

Fig.

the

results

from

out

of

the

labs

happen

to

include

The

trouble

is

in

real

life

we

do

n't

know

and

we

never

know

if

our

error

bar

interval

is

in

the

majority

and

includes

or

by

bad

luck

is

one

of

the

of

cases

that

just

misses

The

error

bars

in

Fig.

are

only

approximately

M

They

are

in

fact

CIs

which

are

designed

by

statisticians

so

in

the

long

run

exactly

will

capture

To

achieve

this

the

interval

needs

to

be

M

t

-LRB-

n

-RRB-

SE

where

t

-LRB-

n

-RRB-

is

a

critical

value

from

tables

of

the

t

statistic

This

critical

value

varies

with

n.

For

n

or

more

it

is

but

for

small

n

it

increases

and

for

n

it

is

Therefore

M

intervals

are

quite

good

approximations

to

CIs

when

n

is

or

more

but

not

for

small

n.

CIs

can

be

thought

of

as

SE

bars

that

have

been

adjusted

by

a

factor

-LRB-

t

-RRB-

so

they

can

be

interpreted

the

same

way

regardless

of

n

This

relation

means

you

can

easily

swap

in

your

mind

's

eye

between

SE

bars

and

CIs

If

a

figure

shows

SE

bars

you

can

mentally

double

them

in

width

to

get

approximate

CIs

as

long

as

n

is

or

more

However

if

n

you

need

to

multiply

the

SE

bars

by

Rule

CIs

capture

on

of

occasions

so

you

can

be

confident

your

interval

includes

SE

bars

can

be

doubled

in

width

to

get

the

approximate

CI

provided

n

is

or

more

If

n

SE

bars

must

be

multiplied

by

to

get

the

approximate

CI

Determining

CIs

requires

slightly

more

calculating

by

the

authors

of

a

paper

but

for

people

reading

it

CIs

make

things

easier

to

understand

as

they

mean

the

same

thing

regardless

of

n.

For

this

reason

in

medicine

CIs

have

been

recommended

for

more

than

years

and

are

required

by

many

journals

-LRB-

-RRB-

Fig.

illustrates

the

relation

between

SD

SE

and

CI

The

data

points

are

shown

as

dots

to

emphasize

the

different

values

of

n

-LRB-

from

to

-RRB-

The

leftmost

error

bars

show

SD

the

same

in

each

case

The

middle

error

bars

show

CIs

and

the

bars

on

the

right

show

SE

bars

both

these

types

of

bars

vary

greatly

with

n

and

are

especially

wide

for

small

n

The

ratio

of

CI/SE

bar

width

is

t

-LRB-

n

-RRB-

the

values

are

shown

at

the

bottom

of

the

figure

Note

also

that

whatever

error

bars

are

shown

it

can

be

helpful

to

the

reader

to

show

the

individual

data

points

especially

for

small

n

as

in

Figs.

and

and

rule

Downloaded

from

on

May

Published

April

Figure

Inferences

between

and

within

groups

Means

and

SE

bars

are

shown

for

an

experiment

where

the

number

of

cells

in

three

independent

clonal

experimental

cell

cultures

-LRB-

E

-RRB-

and

three

independent

clonal

control

cell

cultures

-LRB-

C

-RRB-

was

measured

over

time

Error

bars

can

be

used

to

assess

differences

between

groups

at

the

same

time

point

for

example

by

using

an

overlap

rule

to

estimate

P

for

vs.

or

vs.

but

the

error

bars

shown

here

can

not

be

used

to

assess

within

group

comparisons

for

example

the

change

from

to

more

a

gap

of

SE

indicates

P

and

a

gap

of

SE

indicates

P

-LRB-

Fig.

right

panels

-RRB-

Rule

states

how

SE

bars

relate

to

CIs

Combining

that

relation

with

rule

for

SE

bars

gives

the

rules

for

CIs

which

are

illustrated

in

Fig.

When

n

-LRB-

right

panels

-RRB-

overlap

of

half

of

one

arm

indicates

P

and

just

touching

means

P

To

assess

overlap

use

the

average

of

one

arm

of

the

group

C

interval

and

one

arm

of

the

E

interval

If

n

-LRB-

left

panels

-RRB-

P

when

two

arms

entirely

overlap

so

each

mean

is

about

lined

up

with

the

end

of

the

other

CI

If

the

overlap

is

P

Rule

with

CIs

and

n

overlap

of

one

full

arm

indicates

P

and

overlap

of

half

an

arm

indicates

P

-LRB-

Fig.

left

panels

-RRB-

Repeated

measurements

of

the

same

group

clude

there

is

a

statistically

significant

difference

-LRB-

P

-RRB-

between

and

Rule

in

the

case

of

repeated

measurements

on

the

same

group

-LRB-

e.g.

of

animals

individuals

cultures

or

reactions

-RRB-

CIs

or

SE

bars

are

irrelevant

to

comparisons

within

the

same

group

-LRB-

Fig.

-RRB-

Conclusion

Using

inferential

intervals

to

compare

groups

When

comparing

two

sets

of

results

e.g.

from

n

knock-out

mice

and

n

wild-type

mice

you

can

compare

the

SE

bars

or

the

CIs

on

the

two

means

-LRB-

-RRB-

The

smaller

the

overlap

of

bars

or

the

larger

the

gap

between

bars

the

smaller

the

P

value

and

the

stronger

the

evidence

for

a

true

difference

As

well

as

noting

whether

the

figure

shows

SE

bars

or

CIs

it

is

vital

to

note

n

because

the

rules

giving

approximate

P

are

different

for

n

and

for

n

Fig.

illustrates

the

rules

for

SE

bars

The

panels

on

the

right

show

what

is

needed

when

n

a

gap

equal

to

SE

indicates

P

and

a

gap

of

indicates

P

To

assess

the

gap

use

the

average

SE

for

the

two

groups

meaning

the

average

of

one

arm

of

the

group

C

bars

and

one

arm

of

the

E

bars

However

if

n

-LRB-

the

number

beloved

of

joke

tellers

Snark

hunters

-LRB-

-RRB-

and

experimental

biologists

-RRB-

the

P

value

has

to

be

estimated

differently

In

this

case

P

if

double

the

SE

bars

just

touch

meaning

a

gap

of

SE

Rule

when

n

and

double

the

SE

bars

do

n't

overlap

P

and

if

double

the

SE

bars

just

touch

P

is

close

to

-LRB-

Fig.

leftmost

panel

-RRB-

If

n

is

or

The

rules

illustrated

in

Figs.

and

apply

when

the

means

are

independent

If

two

measurements

are

correlated

as

for

example

with

tests

at

different

times

on

the

same

group

of

animals

or

kinetic

measurements

of

the

same

cultures

or

reactions

the

CIs

-LRB-

or

SEs

-RRB-

do

not

give

the

information

needed

to

assess

the

significance

of

the

differences

between

means

of

the

same

group

at

different

times

because

they

are

not

sensitive

to

correlations

within

the

group

Consider

the

example

in

Fig.

in

which

groups

of

independent

experimental

and

control

cell

cultures

are

each

measured

at

four

times

Error

bars

can

only

be

used

to

compare

the

experimental

to

control

groups

at

any

one

time

point

Whether

the

error

bars

are

CIs

or

SE

bars

they

can

only

be

used

to

assess

between

group

differences

-LRB-

e.g.

vs.

vs.

-RRB-

and

may

not

be

used

to

assess

within

group

differences

such

as

vs.

Assessing

a

within

group

difference

for

example

vs.

requires

an

analysis

that

takes

account

of

the

within

group

correlation

for

example

a

Wilcoxon

or

paired

t

analysis

A

graphical

approach

would

require

finding

the

vs.

difference

for

each

culture

-LRB-

or

animal

-RRB-

in

the

group

then

graphing

the

single

mean

of

those

differences

with

error

bars

that

are

the

SE

or

CI

calculated

from

those

differences

If

that

CI

does

not

in

Error

bars

can

be

valuable

for

understanding

results

in

a

journal

article

and

deciding

whether

the

authors

conclusions

are

justified

by

the

data

However

there

are

pitfalls

When

first

seeing

a

figure

with

error

bars

ask

yourself

What

is

n

Are

they

independent

experiments

or

just

replicates

and

What

kind

of

error

bars

are

they

If

the

figure

legend

gives

you

satisfactory

answers

to

these

questions

you

can

interpret

the

data

but

remember

that

error

bars

and

other

statistics

can

only

be

a

guide

you

also

need

to

use

your

biological

understanding

to

appreciate

the

meaning

of

the

numbers

shown

in

any

figure

This

research

was

supported

by

the

Australian

Research

Council

Correspondence

may

also

be

addressed

to

Geoff

Cumming

-LRB-

g.cumming@latrobe.edu.au

-RRB-

or

Fiona

Fidler

-LRB-

f.fidler@latrobe.edu.au

-RRB-

Downloaded

from

on

May

References

Belia

S.

F.

Fidler

J.

Williams

and

G.

Cumming

Researchers

misunderstand

confidence

intervals

and

standard

error

bars

Psychol

Methods

Cumming

G.

J.

Williams

and

F.

Fidler

Replication

and

researchers

understanding

of

confidence

intervals

and

standard

error

bars

Understanding

Statistics

Vaux

D.L.

Error

message

Nature

Cumming

G.

F.

Fidler

M.

Leonard

P.

Kalinowski

A.

Christiansen

A.

Kleinig

J.

Lo

N.

McMenamin

and

S.

Wilson

Statistical

reform

in

psychology

Is

anything

changing

Psychol

Sci

In

press

Schenker

N.

and

J.F.

Gentleman

On

judging

the

significance

of

differences

by

examining

the

overlap

between

confidence

intervals

Am

Stat

Cumming

G.

and

S.

Finch

Inference

by

eye

Confidence

intervals

and

how

to

read

pictures

of

data

Am

Psychol

International

Committee

of

Medical

Journal

Editors

Uniform

requirements

for

manuscripts

submitted

to

biomedical

journals

Ann

Intern

Med

Carroll

L.

The

hunting

of

the

snark

An

agony

in

fits

Macmillan

London

pp

ERROR

BARS

IN

EXPERIMENTAL

BIOLOGY

CUMMING

ET

AL.

O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O