An

Experimental

Study

of

Search

in

Global

Social

Networks

Peter

Sheridan

Dodds

et

al.

Science

-LRB-

-RRB-

DOI

science

The

following

resources

related

to

this

article

are

available

online

at

-LRB-

this

information

is

current

as

of

May

-RRB-

Updated

information

and

services

including

high-resolution

figures

can

be

found

in

the

online

version

of

this

article

at

Supporting

Online

Material

can

be

found

at

A

list

of

selected

additional

articles

on

the

Science

Web

sites

related

to

this

article

can

be

found

at

This

article

cites

articles

of

which

can

be

accessed

for

free

This

article

has

been

cited

by

article

-LRB-

s

-RRB-

on

the

ISI

Web

of

Science

This

article

has

been

cited

by

articles

hosted

by

HighWire

Press

see

This

article

appears

in

the

following

subject

collections

Sociology

Information

about

obtaining

reprints

of

this

article

or

about

obtaining

permission

to

reproduce

this

article

in

whole

or

in

part

can

be

found

at

Science

-LRB-

print

ISSN

online

ISSN

-RRB-

is

published

weekly

except

the

last

week

in

December

by

the

American

Association

for

the

Advancement

of

Science

New

York

Avenue

NW

Washington

DC

Copyright

by

the

American

Association

for

the

Advancement

of

Science

all

rights

reserved

The

title

Science

is

a

registered

trademark

of

AAAS

Downloaded

from

on

May

REPORTS

An

Experimental

Study

of

Search

in

Global

Social

Networks

Peter

Sheridan

Dodds

Roby

Muhamad

Duncan

J.

We

report

on

a

global

social-search

experiment

in

which

more

than

e-mail

users

attempted

to

reach

one

of

target

persons

in

countries

by

forwarding

messages

to

acquaintances

We

find

that

successful

social

search

is

conducted

primarily

through

intermediate

to

weak

strength

ties

does

not

require

highly

connected

hubs

to

succeed

and

in

contrast

to

unsuccessful

social

search

disproportionately

relies

on

professional

relationships

By

accounting

for

the

attrition

of

message

chains

we

estimate

that

social

searches

can

reach

their

targets

in

a

median

of

five

to

seven

steps

depending

on

the

separation

of

source

and

target

although

small

variations

in

chain

lengths

and

participation

rates

generate

large

differences

in

target

reachability

We

conclude

that

although

global

social

networks

are

in

principle

searchable

actual

success

depends

sensitively

on

individual

incentives

It

has

become

commonplace

to

assert

that

any

individual

in

the

world

can

reach

any

other

individual

through

a

short

chain

of

social

ties

-LRB-

-RRB-

Early

experimental

work

by

Travers

and

Milgram

-LRB-

-RRB-

suggested

that

the

average

length

of

such

chains

is

roughly

six

and

recent

theoretical

-LRB-

-RRB-

and

empirical

-LRB-

-RRB-

work

has

generalized

the

claim

to

a

wide

range

of

nonsocial

networks

However

much

about

this

small

world

hypothesis

is

poorly

understood

and

empirically

unsubstantiated

In

particular

individuals

in

real

social

networks

have

only

limited

local

information

about

the

global

social

network

and

therefore

finding

short

paths

represents

a

nontrivial

search

effort

-LRB-

-RRB-

Moreover

and

contrary

to

accepted

wisdom

experimental

evidence

for

short

global

chain

lengths

is

extremely

limited

-LRB-

-RRB-

For

example

Travers

and

Milgram

report

message

chains

-LRB-

of

which

were

completed

-RRB-

initiated

by

randomly

selected

individuals

from

a

city

other

than

the

target

's

-LRB-

-RRB-

Almost

all

other

empirical

studies

of

large-scale

networks

-LRB-

-RRB-

have

focused

either

on

nonsocial

networks

or

on

crude

proxies

of

social

interaction

such

as

scientific

collaboration

and

studies

specific

to

e-mail

networks

have

so

far

been

limited

to

within

single

institutions

-LRB-

-RRB-

We

have

addressed

these

issues

by

conducting

a

global

Internet-based

social

search

experiment

-LRB-

-RRB-

Participants

registered

online

-LRB-

edu

-RRB-

and

were

randomly

allocated

one

of

target

persons

from

countries

-LRB-

table

-RRB-

Targets

included

a

professor

at

an

Ivy

League

university

an

archival

inspector

in

Estonia

a

technology

consultant

in

India

a

policeman

in

Australia

and

a

veterinarian

in

the

Norwegian

army

Participants

were

informed

that

their

task

was

to

help

relay

a

message

to

their

allocated

target

by

passing

the

message

to

a

social

acquaintance

whom

they

considered

closer

than

themselves

to

the

target

Of

the

individuals

who

registered

about

provided

their

personal

information

and

initiated

message

chains

Because

subsequent

senders

were

effectively

recruited

by

their

own

acquaintances

the

participation

rate

after

the

first

step

increased

to

an

average

of

Including

initial

and

subsequent

senders

data

were

recorded

on

individuals

from

countries

constituting

distinct

message

chains

-LRB-

table

-RRB-

More

than

half

of

all

participants

resided

in

North

America

and

were

middle

class

professional

college

educated

and

Christian

reflecting

commonly

held

notions

of

the

Internet-using

population

-LRB-

-RRB-

In

addition

to

providing

his

or

her

chosen

contact

's

name

and

e-mail

address

each

sender

was

also

required

to

describe

how

he

or

she

had

come

to

know

the

person

along

with

the

type

and

strength

of

the

resulting

relationship

Table

lists

the

frequencies

with

which

different

types

of

relationships

classified

by

type

origin

and

strength

were

invoked

by

our

population

of

active

senders

When

passing

messages

senders

typically

used

friendships

in

preference

to

business

or

family

ties

however

almost

half

of

these

friendships

were

formed

through

either

work

or

school

affiliations

Furthermore

successful

chains

in

comparison

with

incomplete

chains

disproportionately

involved

professional

ties

-LRB-

versus

-RRB-

rather

than

friendship

and

familial

relationships

-LRB-

versus

-RRB-

-LRB-

table

-RRB-

Successful

chains

were

also

more

likely

to

entail

links

that

originated

through

work

or

higher

education

-LRB-

versus

-RRB-

-LRB-

table

-RRB-

Men

passed

messages

more

frequently

to

other

men

-LRB-

-RRB-

and

women

to

other

women

-LRB-

-RRB-

and

this

tendency

to

pass

to

a

same-sex

contact

was

strengthened

by

about

if

the

target

was

the

same

gender

as

the

sender

and

similarly

weakened

in

the

opposite

case

Individuals

in

both

successful

and

unsuccessful

chains

typically

used

ties

to

acquaintances

they

deemed

to

be

fairly

close

However

in

successful

chains

casual

and

not

close

ties

were

chosen

and

more

frequently

than

in

unsuccessful

chains

-LRB-

table

-RRB-

thus

adding

support

and

some

resolution

to

the

longstanding

claim

that

weak

ties

are

disproportionately

responsible

for

social

connectivity

-LRB-

-RRB-

Senders

were

also

asked

why

they

considered

their

nominated

acquaintance

a

suitable

recipient

-LRB-

Table

-RRB-

Two

reasons

geographical

proximity

of

the

acquaintance

to

the

target

and

similarity

of

occupation

accounted

for

at

least

half

of

all

choices

in

general

agreement

with

previous

findings

-LRB-

-RRB-

Geography

clearly

dominated

the

early

stages

of

a

chain

-LRB-

when

senders

were

geographically

distant

-RRB-

but

after

the

third

step

was

cited

less

frequently

than

other

characteristics

of

which

occupation

was

the

most

often

cited

In

contrast

with

previous

claims

-LRB-

-RRB-

the

presence

of

highly

connected

individuals

-LRB-

hubs

-RRB-

appears

to

have

limited

relevance

to

the

kind

of

social

search

embodied

by

our

experiment

-LRB-

social

search

with

large

associated

costs/rewards

or

otherwise

modified

individual

incentives

may

behave

differently

-RRB-

Participants

relatively

rarely

nominated

an

acquaintance

primarily

because

he

or

she

had

many

friends

-LRB-

Table

Friends

-RRB-

and

individuals

in

successful

Table

Type

origin

and

strength

of

social

ties

used

to

direct

messages

Only

the

top

five

categories

in

the

first

two

columns

have

been

listed

The

most

useful

category

of

social

tie

is

medium-strength

friendships

that

originate

in

the

workplace

Type

of

relationship

Friend

Relatives

Co-worker

Sibling

Significant

other

Origin

of

relationship

Work

School/university

Family/relation

Mutual

friend

Internet

Strength

of

relationship

Extremely

close

Very

close

Fairly

close

Casual

Not

close

Institute

for

Social

and

Economic

Research

and

Policy

Columbia

University

West

Street

New

York

NY

USA

of

Sociology

Columbia

University

Amsterdam

Avenue

New

York

NY

USA

To

whom

correspondence

should

be

addressed

Email

SCIENCE

VOL

AUGUST

Downloaded

from

on

May

REPORTS

chains

were

far

less

likely

than

those

in

incomplete

chains

to

send

messages

to

hubs

-LRB-

versus

-RRB-

-LRB-

table

-RRB-

We

also

find

no

evidence

of

message

funneling

-LRB-

-RRB-

through

a

single

acquaintance

of

the

target

At

most

of

messages

passed

through

a

single

acquaintance

of

any

target

and

of

all

chains

were

completed

through

individuals

who

delivered

at

most

three

messages

We

conclude

that

social

search

appears

to

be

largely

an

egalitarian

exercise

not

one

whose

success

depends

on

a

small

minority

of

exceptional

individuals

Although

the

average

participation

rate

-LRB-

about

-RRB-

was

high

relative

to

those

reported

in

most

e-mail

based

surveys

-LRB-

-RRB-

the

compounding

effects

of

attrition

over

multiple

links

resulted

in

exponential

attenuation

of

chains

as

a

function

of

their

length

and

therefore

an

extremely

low

chain

completion

rate

-LRB-

of

chains

reached

their

targets

-RRB-

Chains

may

have

terminated

-LRB-

i

-RRB-

randomly

because

of

individual

apathy

or

disinclination

to

participate

-LRB-

-RRB-

-LRB-

ii

-RRB-

preferentially

at

longer

chain

lengths

corresponding

to

the

claim

that

chains

get

lost

or

are

otherwise

unable

to

reach

their

targets

-LRB-

-RRB-

or

-LRB-

iii

-RRB-

preferentially

at

short

chain

lengths

because

for

example

individuals

nearer

the

target

are

more

likely

to

continue

the

chain

Our

findings

support

the

random-failure

hypothesis

for

two

reasons

First

with

the

exception

of

the

first

step

-LRB-

which

is

special

because

senders

register

rather

than

receive

a

message

from

an

acquaintance

-RRB-

the

attrition

rate

remains

almost

constant

for

all

chain

lengths

at

which

we

have

a

sufficiently

large

N

hence

small

confidence

intervals

-LRB-

Fig.

-RRB-

Second

senders

who

did

not

forward

their

messages

after

one

week

were

asked

why

they

had

not

participated

Less

than

of

those

contacted

claimed

that

they

could

not

think

of

an

appropriate

recipient

suggesting

that

lack

of

interest

or

incentive

not

difficulty

was

the

main

reason

for

chain

termination

To

estimate

the

reachability

of

all

targets

we

first

aggregate

the

completed

chains

across

targets

-LRB-

Fig.

-RRB-

finding

the

average

chain

length

to

be

L

However

this

number

is

misleading

because

it

represents

an

average

only

over

the

completed

chains

and

shorter

chains

are

more

likely

to

be

completed

An

ideal

frequency

distribution

of

chain

lengths

n

-LRB-

L

-RRB-

-LRB-

i.e.

the

chain

lengths

that

would

be

observed

in

the

hypothetical

limit

of

zero

attrition

-RRB-

may

be

estimated

by

accounting

for

observed

attrition

as

follows

n

L

-RRB-

n

-LRB-

L

-RRB-

L

-LRB-

r

i

-RRB-

-LRB-

Fig

i

bars

-RRB-

where

n

-LRB-

L

-RRB-

is

the

observed

number

of

chains

completed

after

L

steps

-LRB-

Fig.

-RRB-

and

rL

is

the

maximum-likelihood

attrition

rate

from

step

L

to

step

L

-LRB-

Fig.

circles

-RRB-

Using

the

observed

values

of

rL

we

have

reconstructed

the

most

likely

ideal

distribution

n

-LRB-

L

-RRB-

-LRB-

Fig.

bars

-RRB-

under

our

assumption

of

random

attrition

Because

the

tail

of

the

distribution

is

poorly

specified

-LRB-

owing

to

the

small

number

of

observed

chains

at

large

L

-RRB-

we

measure

its

median

L

rather

and

this

can

than

its

mean

We

find

L

be

thought

of

as

the

typical

ideal

chain

length

for

a

hypothetical

average

individual

By

repeating

the

above

procedure

for

chains

that

started

and

ended

in

the

same

country

-LRB-

L

-RRB-

or

in

different

countries

-LRB-

L

-RRB-

we

can

disentangle

to

some

extent

the

different

underlying

distributions

of

chains

yielding

an

estimated

range

of

typical

chain

lengths

depending

on

the

geographical

sepL

aration

of

source

and

target

Although

the

range

of

L

and

the

variation

in

attrition

rates

across

targets

do

not

appear

great

the

compounding

effects

of

attrition

over

the

length

of

a

message

chain

can

nevertheless

generate

large

differences

in

message

completion

rates

For

example

a

decrease

of

in

attrition

rates

when

compounded

over

the

same

ideal

distribution

with

L

can

generate

an

increase

in

completion

rate

The

same

attrition

rates

rL

-LRB-

L

-RRB-

-RSB-

when

-LSB-

e.g.

and

applied

over

chains

with

L

respectively

can

lead

to

completion

rates

that

vary

by

up

to

a

factor

of

three

Taken

together

this

evidence

suggests

a

mixed

picture

of

search

in

global

social

networks

On

the

one

hand

all

targets

may

in

fact

be

reachable

from

random

initial

senders

in

only

a

few

steps

with

surprisingly

little

variation

across

targets

in

different

countries

and

professions

On

the

other

hand

small

differences

in

either

participation

rates

or

the

underlying

chain

lengths

can

have

a

dramatic

impact

on

the

apparent

reachability

of

different

targets

Target

-LRB-

a

professor

at

a

prominent

U.S.

university

-RRB-

stands

out

in

this

respect

Because

of

senders

were

college

educated

and

more

than

half

were

American

participants

may

have

anticipated

little

difficulty

in

reaching

him

thus

accounting

for

his

chains

attrition

rate

-LRB-

-RRB-

being

much

lower

than

that

of

any

other

target

-LRB-

to

-RRB-

Target

received

a

notable

of

all

completed

chains

yet

this

result

is

consistent

with

his

true

reachability

being

little

different

from

that

of

other

targets

his

allocated

senders

may

simply

have

been

more

confident

of

success

Our

results

therefore

suggest

that

if

individuals

searching

for

remote

targets

do

not

have

sufficient

incentives

to

proceed

the

small-world

hypothesis

will

not

appear

to

hold

-LRB-

-RRB-

but

that

even

a

slight

increase

in

incentives

can

render

social

searches

success

Table

Reason

for

choosing

next

recipient

All

quantities

are

percentages

Location

recipient

is

geographically

closer

Travel

recipient

has

traveled

to

target

's

region

Family

recipient

's

family

originates

from

target

's

region

Work

recipient

has

occupation

similar

to

target

Education

recipient

has

similar

educational

background

to

target

Friends

recipient

has

many

friends

Cooperative

recipient

is

considered

likely

to

continue

the

chain

Other

includes

recipient

as

the

target

L

N

Location

Travel

Family

Work

Education

Friends

Cooperative

Other

Fig.

Distributions

of

message

chain

lengths

-LRB-

A

-RRB-

Average

per-step

attrition

rates

-LRB-

circles

-RRB-

and

confidence

interval

-LRB-

triangles

-RRB-

-LRB-

B

-RRB-

Histogram

representing

the

number

of

chains

that

are

completed

in

L

steps

-LRB-

L

-RRB-

-LRB-

C

-RRB-

Ideal

histogram

of

chain

lengths

recovered

from

-LRB-

B

-RRB-

by

accounting

for

message

attrition

-LRB-

A

-RRB-

Bars

represent

the

ideal

histogram

recovered

with

average

values

of

r

-LSB-

circles

in

-LRB-

A

-RRB-

-RSB-

for

the

histogram

in

-LRB-

B

-RRB-

lines

represent

a

decomposition

of

the

complete

data

into

chains

that

start

in

the

same

country

as

the

target

-LRB-

circles

-RRB-

and

those

that

start

in

a

different

country

-LRB-

triangles

-RRB-

AUGUST

VOL

SCIENCE

Downloaded

from

on

May

REPORTS

ful

under

broad

conditions

More

generally

the

experimental

approach

adopted

here

suggests

that

empirically

observed

network

structure

can

only

be

meaningfully

interpreted

in

light

of

the

actions

strategies

and

even

perceptions

of

the

individuals

embedded

in

the

network

Network

structure

alone

is

not

everything

References

and

Notes

A.

Wagner

D.

Fell

Proc

R.

Soc

London

B

-LRB-

-RRB-

M.

E.

J.

Newman

Phys

Rev.

E

-LRB-

-RRB-

J.

Kleinberg

Nature

-LRB-

-RRB-

D.

J.

Watts

P.

S.

Dodds

M.

E.

J.

Newman

Science

-LRB-

-RRB-

L.

A.

Adamic

R.

M.

Lukose

A.

R.

Puniyani

B.

A.

Huberman

Phys

Rev.

E

-LRB-

-RRB-

J.

S.

Kleinfeld

Society

-LRB-

-RRB-

C.

Korte

S.

Milgram

J.

Pers

Soc

Psychol

-LRB-

-RRB-

N.

Lin

P.

Dayton

P.

Greenwald

in

Communication

Yearbook

Vol

B.

D.

Ruben

Ed

-LRB-

Transaction

Books

New

Brunswick

NJ

-RRB-

pp.

A.-L

Barabasi

R.

Albert

Science

-LRB-

-RRB-

M.

Faloutsos

P.

Faloutsos

C.

Faloutsos

Comp

Comm

Rev.

-LRB-

-RRB-

L.

A.

Adamic

B.

A.

Huberman

Science

-LRB-

-RRB-

H.

Jeong

B.

Tombor

R.

Albert

Z.

N.

Oltavi

A.-L

Barabasi

Nature

-LRB-

-RRB-

H.

Ebel

L.-I

Mielsch

S.

Bornholdt

Phys

Rev.

E

-LRB-

-RRB-

Materials

and

methods

are

available

as

supporting

material

on

Science

Online

W.

Chen

J.

Boase

B.

Wellman

in

The

Internet

in

Everyday

Life

B.

Wellman

C.

Haythornthwaite

Eds

-LRB-

Blackwell

Oxford

-RRB-

pp.

M.

S.

Granovetter

Am

J.

Sociol

-LRB-

-RRB-

P.

D.

Killworth

H.

R.

Bernard

Soc

Networks

-LRB-

-RRB-

H.

R.

Bernard

P.

D.

Killworth

M.

J.

Evans

C.

McCarty

G.

A.

Shelly

Ethnology

-LRB-

-RRB-

K.

Sheehan

J.

Comput

Mediated

Commun

-LRB-

-RRB-

Available

online

at

-LRB-

-RRB-

H.

C.

White

Soc

Forces

-LRB-

-RRB-

-LRB-

-RRB-

This

research

was

supported

in

part

by

the

National

Science

Foundation

Intel

Corporation

and

Office

of

Naval

Research

Supporting

Online

Material

Methods

Tables

to

December

accepted

May

I.

de

Sola

Pool

M.

Kochen

Soc

Networks

-LRB-

-RRB-

S.

H.

Strogatz

Nature

-LRB-

-RRB-

J.

Travers

S.

Milgram

Sociometry

-LRB-

-RRB-

D.

J.

Watts

S.

H.

Strogatz

Nature

-LRB-

-RRB-

R.

Albert

H.

Jeong

A.-L

Barabasi

Nature

-LRB-

-RRB-

L.

A.

Adamic

in

Lecture

Notes

in

Computer

Science

S.

Abiteboul

A.

Vercoustre

Eds

-LRB-

Springer

Heidelberg

-RRB-

pp.

L.

A.

N.

Amaral

A.

Scala

M.

Barthelemy

H.

E.

Stanley

Proc

Natl.

Acad

Sci

U.S.A.

-LRB-

-RRB-

Phylogenetics

and

the

Cohesion

of

Bacterial

Genomes

Vincent

Daubin

Nancy

A.

Moran

Howard

Gene

acquisition

is

an

ongoing

process

in

many

bacterial

genomes

contributing

to

adaptation

and

ecological

diversification

Lateral

gene

transfer

is

considered

the

primary

explanation

for

discordance

among

gene

phylogenies

and

as

an

obstacle

to

reconstructing

the

tree

of

life

We

measured

the

extent

of

phylogenetic

conflict

and

alien-gene

acquisition

within

quartets

of

sequenced

genomes

Although

comparisons

of

complete

gene

inventories

indicate

appreciable

gain

and

loss

of

genes

orthologs

available

for

phylogenetic

reconstruction

are

consistent

with

a

single

tree

In

all

but

the

most

reduced

bacterial

genomes

there

is

a

substantial

fraction

of

genes

whose

distributions

and

compositional

features

indicate

that

they

originated

by

lateral

gene

transfer

-LRB-

LGT

-RRB-

-LRB-

-RRB-

There

is

also

clear

evidence

of

LGT

between

distantly

related

organisms

based

on

phylogenetic

studies

involving

large

taxonomic

samples

-LRB-

-RRB-

Given

these

findings

incompatibility

of

phylogenies

within

and

among

bacterial

phyla

based

on

different

genes

has

routinely

been

ascribed

to

LGT

-LRB-

-RRB-

However

building

molecular

phylogenies

for

distantly

related

species

is

often

a

difficult

task

and

choice

of

phylogenetic

methods

genes

or

taxa

can

yield

different

results

For

example

there

is

still

no

consensus

on

the

monophyly

of

rodents

-LRB-

-RRB-

or

the

branching

order

of

amniotes

-LRB-

-RRB-

and

these

groups

are

young

compared

to

bacterial

phyla

In

addition

distinguishing

between

orthologous

genes

-LRB-

sequences

that

trace

their

divergence

to

the

splitting

of

organismal

linDepartment

of

Biochemistry

and

Molecular

Biophysics

of

Ecology

and

Evolutionary

Biology

University

of

Arizona

Tucson

AZ

USA

To

whom

correspondence

should

be

addressed

Email

hochman@email.arizona.edu

eages

-RRB-

and

paralogous

-LRB-

duplicated

-RRB-

genes

becomes

increasingly

difficult

when

considering

more

distantly

related

taxa

The

effects

of

LGT

have

been

extended

from

the

deepest

to

the

shallowest

levels

of

bacterial

relationships

Indeed

the

similarities

in

gene

sequence

and

gene

content

that

define

widely

accepted

bacterial

taxa

have

been

proposed

to

reflect

boundaries

to

gene

transfer

rather

than

vertical

transmission

and

common

organismal

ancestry

-LRB-

-RRB-

Thus

LGT

may

overwhelm

attempts

to

reconstruct

the

relationships

among

bacterial

taxa

The

claim

that

the

history

of

bacteria

might

be

more

faithfully

depicted

as

a

net

than

as

a

tree

-LRB-

-RRB-

relies

upon

the

postulate

that

the

substantial

incidence

of

acquired

DNA

within

genomes

is

the

basis

for

findings

of

phylogenetic

incongruence

among

genes

However

the

genes

detected

as

recently

transferred

are

by

and

large

different

from

those

used

to

build

species

phylogenies

The

former

are

disproportionately

A

T-rich

have

restricted

phylogenetic

distributions

and

usually

encode

accessory

functions

In

contrast

species

phylogenies

are

based

on

genes

with

wide

taxonomic

distributions

and

having

key

roles

in

cellular

processes

However

such

differences

are

often

ignored

when

considering

the

impact

of

LGT

on

bacterial

relationships

Although

the

incidence

of

recently

acquired

DNA

in

bacterial

genomes

is

the

most

direct

indication

of

extensive

LGT

among

species

-LRB-

-RRB-

the

question

of

whether

the

incongruence

in

gene

phylogenies

is

linked

to

the

amount

of

new

DNA

in

a

genome

has

not

been

addressed

To

investigate

the

relation

between

DNA

acquisition

and

phylogenetic

incongruence

we

selected

quartets

of

related

sequenced

genomes

whose

phylogenetic

relationships

based

on

small

subunit

ribosomal

RNA

-LRB-

SSU

rRNA

-RRB-

sequences

display

the

branching

topology

shown

in

Fig.

For

each

quartet

we

inferred

both

the

number

of

recently

acquired

and

lost

genes

-LRB-

based

on

their

phylogenetic

distributions

-RRB-

and

the

proportion

of

ortholog

phylogenies

supporting

lateral

transfers

We

applied

a

conservative

method

for

identifying

orthologs

by

including

only

those

genes

having

a

single

significant

match

per

genome

thus

minimizing

the

risks

of

including

hidden

paralogs

descending

from

within-genome

duplication

events

This

contrasts

with

the

commonly

used

reciprocal

besthit

method

-LRB-

-RRB-

to

infer

orthology

which

can

yield

misleading

results

-LRB-

-RRB-

especially

when

paralogs

experience

different

evolutionary

rates

We

retained

all

quartets

of

species

for

which

of

the

genes

from

the

smallest

genome

were

recovered

as

orthologs

We

then

tested

which

of

the

three

possible

trees

was

significantly

supported

for

each

ortholog

family

using

the

Shimodaira-Hasegawa

-LRB-

SH

-RRB-

-LRB-

-RRB-

test

implemented

in

Tree-puzzle

-LRB-

-RRB-

at

the

level

of

significance

-LRB-

-RRB-

This

method

tests

if

an

alignment

significantly

supports

a

tree

by

estimating

the

confidence

limits

of

the

likelihood

estimates

of

the

topologies

SCIENCE

VOL

AUGUST

Downloaded

from

on

May

O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O B I O O O O O O O O O O O O O O O O B O O O O O O O O O B I O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

search
networks
social network
empirical
small world


A data integration methodology for systems biology Daehee Hwang * , Alistair G. Rust * , Stephen Ramsey * , Jennifer J. Smith * , Deena M. Leslie * , Andrea D. Weston * , Pedro de Atauri * , John D. Aitchison * , Leroy Hood * , Andrew F. Siegel , and Hamid Bolouri * * Institute for Systems Biology , 1441 North 34th Street , Seattle , WA 98103 ; and Departments of Management Science , Finance , and Statistics , University of Washington , Seattle WA 98195 Contributed by Leroy Hood , October 5 , 2005 Different experimental technologies measure different aspects of a system and to differing depth and breadth . High-throughput assays have inherently high false-positive and false-negative rates . Moreover , each technology includes systematic biases of a different nature . These differences make network reconstruction from multiple data sets difficult and error-prone . Additionally , because of the rapid rate of progress in biotechnology , there is usually no curated exemplar data set from which one might estimate data integration parameters . To address these concerns , we have developed data integration methods that can handle multiple data sets differing in statistical power , type , size , and network coverage without requiring a curated training data set . Our methodology is general in purpose and may be applied to integrate data from any existing and future technologies . Here we outline our methods and then demonstrate their performance by applying them to simulated data sets . The results show that these methods select truepositive data elements much more accurately than classical approaches . In an accompanying companion paper , we demonstrate the applicability of our approach to biological data . We have integrated our methodology into a free open source software package named POINTILLIST . Fisher 's method mixture distribution models S ystems biology -LRB- 1 , 2 -RRB- aims to understand cellular behavior in terms of the spatiotemporal interactions among cellular components , such as genes , proteins , metabolites , and organelles . In systems biology , one typically perturbs a system and , with highthroughput measurements to identify all pertinent elements and their interactions , integrates them into a biological network to understand the system 's behavior . As such , systems biology is predicated on the integration of experimental data from an ever increasing number of technologies , such as gene expression arrays , proteomics , and chromatin immunoprecipitation on chip assays -LRB- 3 -RRB- . Integration achieves one of the most important imperatives of systems biology , namely it reduces the dimensionality of global data to deliver useful information about the system of interest . A major challenge in systems biology is that technologies that globally interrogate biological systems have inherently high falsepositive and false-negative rates -LRB- 4 -RRB- ; thus , each data type alone has a limited utility . The integration of data from different sources provides an effective means to deal with this issue by reinforcing bona fide observations and reducing false negatives . Moreover , because different experimental technologies provide different insights into a system , the integration of multiple data types offers the greatest information about a particular cellular process . For example , gene perturbation experiments -LRB- e.g. , knockouts or RNA interference -RRB- reveal relationships between genes that may imply direct physical interactions or indirect logical interactions . In contrast , chromatin immunoprecipitation chip data can reveal direct protein DNA interactions or cofactor associations with bound transcription factors -LRB- 3 -RRB- . Combined together , these technologies can provide a much more detailed view of a transcriptional regulatory network than either alone -LRB- 5 -RRB- . There are a number of confounding problems that make data integration nontrivial . First , the types of data to be integrated range from discrete -LRB- e.g. , a protein molecule may be localized to one or PNAS November 29 , 2005 vol . 102 no. 48 more organelles -RRB- to continuous -LRB- e.g. , mRNA expression level -RRB- . Second , each technology used has a different degree of reliability and different amounts of the various types of error . Even when considering multiple data sets generated by a common method , simply taking the intersection of these data sets does not remove random errors completely -LRB- 6 -RRB- . Third , each data set includes its own systematic biases -LRB- 4 , 7 -RRB- . For example , labeling-based mass spectrometry approaches -LRB- e.g. , isotope-coded affinity tag -RRB- tend to favor identification of highly abundant proteins . Small-scale experiments tend to provide strong evidence for a small portion of a network but say little about what may have been missed . Finally , in addition to data generated by high-throughput technologies , there are other attractive sources of data , such as small-scale experiments , curated databases , and computational predictions . To fully realize the potential of systems biology , it is imperative to draw from all of these sources of data . However , curated databases often favor widely studied proteins and genes . Curated databases may also merge data from different strains cell types or from various experimental conditions , and they may contain considerable data on one part of a system while omitting other parts . Computational predictions that extrapolate from earlier experimental data -LSB- e.g. , prediction of protein protein interactions from known interactions of homologous proteins -LRB- 8 -RRB- -RSB- run the risk of perpetuating any systematic bias in the source data -LRB- in addition to false-positive and false-negative errors -RRB- . De novo predictions are even more error prone . There is therefore a pressing need for more effective methods of integrating data . These methods should accommodate various sources of binary , categorical , and continuous valued data acquired from high-throughput experiments , small-scale experiments , databases , and computational predictions ; and should be suitable for dealing with missing data , high error rates , and systematic biases in each data set . In addition , there are few fully verified data sets available for training . Integration methods not requiring a training set have so far been limited to particular classes of data where specific assumptions hold true -LRB- 9 -RRB- . To address these concerns , we have developed a data integration methodology that can handle multiple data sets differing in type , size , and network coverage and does not require a training data set . This methodology uses an optimization algorithm to minimize the numbers of false positives and false negatives , and it makes no assumptions about the number of data sets integrated ; rather , it is for general purposes and may be applied to integrate data from any existing and future technologies . We have integrated our methodology into a freely available software package named POINTILLIST . In this paper , we describe our methodology and its statistical foundations using simple illustrative example data sets . Also in this Conflict of interest statement : No conflicts declared . Abbreviations : LS , Liptak Stouffer 's ; MG , Mudholkar George 's ; NP , nonparametric ; MM , mixture model ; cdf , cumulative density function ; PDF , probability density function ; ESA , enhanced simulated annealing . Present address : Pfizer Global Research and Development , Safety Sciences , Eastern Point Road , Groton , CT 06340 . whom correspondence may be addressed : E-mail : hbolouri@systemsbiology.org or lhood@systemsbiology.org . To 2005 by The National Academy of Sciences of the USA 17296 17301 www.pnas.org cgi doi 10.1073 pnas .0508647102 Fig. 1 . Illustration of classical approaches and unweighted Fisher 's method for integrating two simulated data sets . -LRB- A -RRB- Selection of significant elements using -LRB- i -RRB- 0.05 and 0.025 , -LRB- ii -RRB- union approaches with 0.05 and 0.1 , and -LRB- iii -RRB- unweighted Fisher 's method with 0.05 and 0.1 . intersection approaches with 0.05 and -LRB- B -RRB- Effects of integration statistics on the shape of the decision boundaries . -LRB- C -RRB- Selection of significant elements using weighted Fisher 's method with -LRB- i -RRB- -LRB- ii -RRB- T 0.013 to reduce false-positive errors based on the null hypothesis of the weighted Fisher 's method . For these particular data sets , it can be seen that the selection using T 0.013 maximizes the precision -LRB- Table 1 -RRB- . issue of PNAS -LRB- 10 -RRB- , we demonstrate the utility and efficacy of POINTILLIST by applying it to the integration of 18 data sets to arrive at a network model of the galactose utilization network in yeast . The resulting network recapitulates the known biology of galactose utilization and provides new insights and predictions , some of which we verified experimentally . Methods Simulated Data Sets . Although we developed and tested our methodologies by using many sets of real experimental data , for clarity we base this paper on the simplest set of data that would be sufficient to illustrate the pertinent characteristics of the various methods presented . See the companion paper -LRB- 10 -RRB- for a demonstration of the applicability of our methods to a wide variety of experimental data . We generated simulated data mimicking real high-throughput data as follows . First , true differences -LRB- e.g. , fold changes of gene expression levels -RRB- for the data elements affected by a perturbation -LRB- e.g. , disease -RRB- were drawn with random signs from a noncentral distribution -LRB- gamma distribution with a 1.25 and b 1.25 -RRB- . We used two parameters -LRB- and -RRB- to define the proportion of affected elements -LRB- -RRB- and the fraction of affected elements with negative differences -LRB- -RRB- , respectively . True differences for nonaffected elements -LRB- 1 -RRB- were set to zero -LRB- Figs. 5 11 , which are published as supporting information on the PNAS web site , illustrate characteristic features of the data and our methods -RRB- . Second , normal random noise -LRB- riei -RRB- was added to the true differences -LRB- ti -RRB- of each data set -LRB- Ti -RRB- : Ti ti riei , where ri is a noise level inversely related to the statistical power of the technology producing each data set , i , and ei is standard normal . The affected data elements in a data set with a small noise level produce high absolute Ti values , resulting in low P values in the following test . Finally , a two-tailed test was performed on Ti to produce P values for each data set -LRB- Pi -RRB- : Pi 2Ncdf -LRB- Ti ri -RRB- , where the division using the scaling factor ri is the scale of the noise . The same two data sets were used for Figs. 1 3 -LRB- ri 0.666 for data set 1 and 0.334 for data set 2 , and a b 2 -RRB- . For the more noisy , three-dimensional data used in Fig. 4B , a and b values were randomly selected in the range 1.25 2.0 to generate 3 10 30 data sets -LRB- for the example shown in Fig. 4A , a 1.5 and b 1.5 -RRB- . The noise level -LRB- ri -RRB- was selected from a uniform distribution and multiplied by a scaling factor of three for half the data and four for the rest . Weighted Integration Methods . Several integration methods -LRB- 11 -RRB- , Fig. 2 . Mixture distribution model . -LRB- A -RRB- The results of mixture distribution modeling , which indicates that the estimated H0 and H1 distributions represent the real data distribution -LRB- 2 of the residual was less than the cutoff 0.05 -RRB- . T was chosen as the area under with the H0 distribution from where the H0 and H1 distributions meet to the positive infinite . -LRB- B -RRB- Selection of significant elements using the T and its comparison with the weighted Fisher 's 0.05 . The selection using T method with provides the best accuracy F measure -LRB- Table 1 -RRB- . Hwang et al. . PNAS November 29 , 2005 vol . 102 no. 48 17297 APPLIED BIOLOGICAL SCIENCES such as Fisher 's 2 -LRB- 12 -RRB- and Stouffer 's Z -LRB- 11 -RRB- , have been widely used in statistical metaanalysis to combine P values from k data sets . In this study , we used `` weighted '' versions of the following integration statistics to maximize the overall statistical power of the weighted sum of nonlinearly transformed P values in each method : Fig. 3 . NP weighted Fisher 's method . -LRB- A -RRB- Selected elements after the first iteration . -LRB- B -RRB- Selected elements after the fifth iteration , which shows that the elements selected by P1 -LRB- which has less statistical power -RRB- are being eliminated first . Thus , the iteration procedure correctly captures the relative importance of the data sets . -LRB- C -RRB- The final set of selected elements and its comparison with the selection using the decision boundary from T 0.015 determined by mixture distribution model -LRB- Fig. 3 -RRB- . Selections based on the mixture distribution model and the NP Fisher 's method produce comparable accuracy -LRB- Table 1 -RRB- . k Fisher 's weighted F : F w 2 i 1 w i ln -LRB- P i -RRB- . Mudholkar George 's -LRB- MG -RRB- weighted T : T w 15k 12 5k 2 k k 2 i 1 w i ln Pi 1 Pi . Liptak Stouffer 's -LRB- LS -RRB- weighted Z : Z w 1 w2 i k w iN i 1 1 1 Pi , of the weighted integration statistics for a given significance level 1 -RRB- . This maximization was implemented by using -LRB- 0 enhanced simulated annealing -LRB- ESA -RRB- -LRB- see ref . 13 -RRB- . -LRB- i -RRB- Guess an initial weight vector . -LRB- ii -RRB- For the weight vector , determine by using Monte Carlo simulations a cdf of background data elements -LRB- those satisfying the null hypothesis H0 -RRB- for a given integration statistic . The rejection method -LRB- 14 -RRB- was used to generate random numbers from a central distribution with k 1 -LRB- i.e. , 2 distribution with two degrees of freedom for Fisher 's method , t-distribution with nine degrees of freedom for MG method , or standard normal distribution for LS method -RRB- . -LRB- iii -RRB- Find an overall statistic value -LRB- c -RRB- corre . -LRB- iv -RRB- Define a decision boundary for sponding to cdf -LRB- H0 -RRB- 1 Fw c by generating a grid matrix P j -LCB- P1 , P2 , ... , Pj 1 , Pj 1 , ... , Pk -RCB- using Gauss-Legendre quadrature -LRB- 15 -RRB- and then by computing Pj values for the grid points : pj exp c 2 i j where N 1 -LRB- -RRB- is the inverse of a standard normal cumulative density function -LRB- cdf -RRB- . The weight -LRB- wi -RRB- for the P values of each data set represents a relative measure of statistical power compared with the other data sets . Fig. 6 shows the definitions of false-positive and false-negative error rates and statistical power by using two distributions for nonaffected -LRB- background -RRB- and affected elements , when Fisher 's weighted F was used . Also , Figs. 7 and 8 show the effect of random noise on the distribution of P values of the background and affected elements , respectively . Determination of Weights . We determined the weights by maximizing the overall statistical power -LRB- observed in the data sets ; Fig. 6 -RRB- w i ln pi 2w j , -LRB- v -RRB- Count selected data elements by using this boundary as the objective function to be maximized . -LRB- vi -RRB- Try another weight vector and reject or accept it by using the Metropolis algorithm until the ESA stopping criteria -LRB- 13 -RRB- are met . Finally , the overall P value for each data element was calculated using cdf -LRB- H0 -RRB- . The selected elements should have overall P values less than the value of . Fig. 4 . Comparison of the performance of different integration methods . -LRB- A -RRB- P values for three complex data sets . -LRB- B -RRB- Receiver operating characteristic graph showing the relative performance of different integration methods in terms of falsepositive and true-positive error rates . The MM and NP integration methods outperform the other methods -LRB- see text for details -RRB- . UF , unweighted Fisher 's method ; WF , weighted Fisher 's method ; FT , threshold Fisher 's method . 17298 www.pnas.org cgi doi 10.1073 pnas .0508647102 Hwang et al. . Determination of Significance Threshold -LRB- T -RRB- for Combined P Value . For a given weight vector , we compute the ratios of cumulative observed densities to cumulative expected densities by using the real data distributions in the range of between 0 and a : R a 1 1 Dcdf -LRB- S w H cdf -LRB- S w 0 c a -RRB- c a -RRB- 1 D cdf -LRB- S w a c a -RRB- , where Dcdf -LRB- -RRB- is the cdf of the observed data distribution , and c -LRB- a -RRB- corresponds to the integration statistic Sw -LRB- e.g. , Fw -RRB- value when 0 -RRB- is infinite . As a decreases , the decision boundary a. Note c -LRB- moves toward the origin in P-value space . When there are truepositive elements , this results in a large cumulative density ratio because the expected density is small but the observed density is large . As a increases , however , the ratio decreases because the observed density slowly increases relative to the expected density . The plot of the ratio versus a shows that the curve can be split into two regions , one rich in true positives and the other in background data elements -LRB- Fig. 11 -RRB- . T was determined as the x axis value of the point nearest to the origin . Mixture Distribution Model . For a given weight vector , we define the probability density function -LRB- PDF -RRB- of a mixture distribution model -RRB- PDF -LRB- H0 -RRB- PDF -LRB- H1 -RRB- -LRB- 16 , 17 -RRB- . We directly M as PDF -LRB- M -RRB- -LRB- 1 estimated an H1 distribution by fitting to the distribution of the real 1 -RRB- ; -LRB- ii -RRB- define a data elements as follows : -LRB- i -RRB- guess a -LRB- 0 PDF -LRB- H0 -RRB- by using Monte Carlo simulation ; -LRB- iii -RRB- define a PDF -LRB- H1 -RRB- of integration statistic values -LRB- Sw -RRB- as a distribution of affected data elements by guessing two parameters -LRB- a and b -RRB- for a gamma distribution ; -LRB- iv -RRB- compute the PDF -LRB- M -RRB- by using the equation above ; -LRB- v -RRB- calculate the objective function as the squared sum of the difference between cdf -LRB- M -RRB- and the cdf of the real data Dcdf -LRB- Sw -RRB- Mcdf -LRB- Sw -RRB- 2 ; -LRB- vi -RRB- repeat steps ii v with a different set of gamma 2 distribution parameters -LRB- a and b -RRB- and and then recompute the objective function ; -LRB- vii -RRB- accept or reject these new trials by using the Metropolis algorithm until the ESA stopping criteria are met . T -LRB- see below -RRB- , the initial set of elements is selected with the union method : -LCB- 1,2 , ... , k -RCB- . A weight for each data set was C -LCB- Pi T -RCB- , i then computed as the nonoverlapping area between the P value distributions of C and the set of random background elements -LRB- B -RRB- : demonstrate the application of the methodology to real biological data . Fig. 1 A presents two example data sets generated as described in Methods . Each of our data sets -LRB- assays for genes or proteins or their interactions -RRB- consists of two types of elements : true positives -LRB- the elements affected by an experimental perturbation , which we wish to detect -RRB- and true negatives -LRB- background elements not affected by the perturbation -RRB- . As is common for high-throughput technologies , each data element is presented as a P value -LRB- the probability of observing a value when the corresponding data element is not affected by the experimental perturbation -RRB- . Thus , data elements with lower P values are more likely to be true positives . In Fig. 1 A , the true negatives are distributed uniformly -LRB- shown as gray dots -RRB- . When plotting multiple data sets together , P values for true positives would ideally be expected to be distributed near the origin . In practice , because of the different noise characteristics of each measurement technology , P values for the same data element may be low in one measurement -LRB- along one axis -RRB- and not in another . This results in the distribution of true positives near the axes . Note that the true-positive elements in Fig. 1A are asymmetrically distributed . Data set two -LRB- measured along the y axis -RRB- was generated with a lower level of noise -LRB- thereby containing more statistical power -RRB- and therefore has more true-positive data elements with low P values . Finally , note that , although the true positives are highly correlated -LRB- as expected -RRB- , the true negatives are not correlated . For real high-throughput data , true positives will comprise a small proportion of the total data . Thus data sets from different technologies will be largely uncorrelated -LRB- 19 -RRB- . For the example presented in the companion paper , only 69 genes of 6,000 genes were selected as potential positive elements ; 16,985 protein protein interactions were selected out of 6,0002 ; and 8,555 protein DNA interactions were selected out of 135 6,0002 . The correlation coefficients between the above data sets were all 0.3 . To make it easier to understand and compare the methodologies presented , the same data are used for the analyses presented in Figs. 1 3 . Classical Approaches to Data Integration . Fig. 1 A shows the decision boundaries for three commonly used approaches to data integration : intersection , union -LRB- 20 -RRB- , and unweighted Fisher 's method -LRB- 12 -RRB- . As described above , true-positive data elements tend to be distributed near the axes and especially near the origin . Thus , data integration methods typically divide the total P value space into two regions by determining a decision boundary . Data elements in the region near the axes and or origin , which is rich in true positives , are selected as significant . Data elements in the complementary region -LRB- away from the origin and axes -RRB- are considered true negatives . In the intersection and union methods , significant elements are identified independently in each data set using a cutoff P value -LRB- e.g. , 0.05 -RRB- . For intersection , only the significant elements common to all data sets are selected . This method tends to miss many true-positive data elements near the P1 axis and away from the origin -LRB- see Fig. 1 A -RRB- . The union method improves this situation by selecting elements significant in any one data set but includes many false positives near the P2 axis and may still miss many elements with low geometric-mean P values . Note that the union and intersection methods treat each data set independently . In contrast , Fisher 's method selects significant data elements by using a more sophisticated measure that results in a hyperbolic -LRB- curved -RRB- decision boundary , allowing selection of data elements near both axes and within a larger region near the origin . Two additional integration methods widely used in metaanalysis to combine P values from multiple data sets are MG T and LS Z -LRB- see Methods for formulae -RRB- . Fig. 1B illustrates the differences between these methods and Fisher 's . The three methods involve different transformations of the P values -LRB- see Methods -RRB- , resulting in different shapes of the decision boundary . As can be seen in Fig. 1B , the LS and MG methods favor data elements with low P values in all data sets -LRB- points near the origin -RRB- . In contrast , Fisher 's method PNAS November 29 , 2005 vol . 102 no. 48 17299 Nonparametric -LRB- NP -RRB- Decision Boundary . For a given wi Max -LCB- CDF -LRB- S i , B -RRB- si CDF -LRB- S i , C -RRB- -RCB- , where the instances of Si are the transformed P values for data set 2ln -LRB- Pi -RRB- -RSB- . To quantify the i -LSB- e.g. , when Fisher 's method is used , Si overall statistical power for the two sets C and B , we compute as an objective function , f , the nonoverlapping area between distributions of weighted integration statistic values -LRB- Sw -RRB- of C and B : f Max -LCB- CDF -LRB- S w , B -RRB- si CDF -LRB- S w , C -RRB- -RCB- . Note that f is related to statistical power of C , i.e. , 1 cdf -LRB- Smax , C -RRB- , w where Smax is the Sw value at which the cdf difference is at maximum . w Also , the final set of selected elements depends on T . The initial T value was determined in a manner similar to the mixture distribution method above , except that we approximated the PDF of H1 by using kernel density estimation -LRB- 18 -RRB- . Results Example Data Sets . For the purposes of illustration , here we use two artificially generated example data sets . In this way , we know the data characteristics a priori and can illustrate and evaluate each step in our methodology unambiguously . The simulated data are designed to be as simple as possible while mimicking data from high-throughput technologies . In a companion paper -LRB- 10 -RRB- , we Hwang et al. . APPLIED BIOLOGICAL SCIENCES permits the selection of elements as long as there is at least one very small P value . Thus , Fisher 's method is most effective when P values for true-positive data elements in different data sets tend not to agree -LRB- many data elements scattered near the axes -RRB- , and the MG or LS methods when data sets tend to agree -LRB- many data elements scattered near the origin -RRB- . Importantly , all of these methods treat both data sets equally , producing a symmetric decision boundary , even though our data are highly asymmetric . The methods we present below overcome this limitation by producing asymmetric decision boundaries -LRB- POINTILLIST provides all three methods . The user should select the appropriate method for the given data characteristics -RRB- . For the remainder of this paper , for simplicity , we will use only Fisher 's method without loss of generality . Weighted Integration of P Values . To allow for various levels of statistical power -LRB- reliability -RRB- in different data sets , we use a weighted method for combining P values . In this approach , P values from different high-throughput assays are transformed , scaled -LSB- by a weight representing the reliability -LRB- statistical power -RRB- of the assay -RSB- and summed to generate a combined P value -LRB- see Methods -RRB- . The combined P values are then tested by using Monte Carlo generated H0 distributions as described in Methods . Data elements whose combined P values are below a threshold are accepted as components of the system of interest . All other data elements are rejected . The selection threshold corresponds to a significance level -LRB- -RRB- reflecting the proportion of area below the decision boundary . Infinitely many combinations of weights can satisfy a given -LRB- Fig. 9 -RRB- . For a given , we select weights that maximize the number of elements chosen -LRB- Fig. 10 , elements between the decision boundary and axes -RRB- , which maximizes the number of true positives selected because the majority of data elements near the axes are true positives -LRB- for illustrative examples , see Fig. 1C -RRB- . We used ESA to search for such optimal weights -LRB- see Methods -RRB- . Fig. 1C compares the unweighted -LRB- green line -RRB- and weighted -LRB- blue line -RRB- Fisher 's methods for our example data . In this example , P values from data set 2 are twice as reliable as those in data set 1 , thus skewing the P2 values correspondingly closer to 0 -LRB- see Methods -RRB- . As shown , a weighted integration method produces an asymmetric decision boundary and can therefore capture more of the true positives . Selecting a Joint-Significance Threshold -LRB- -RRB- . As shown in Figs. 7 and explicitly dividing the data elements into two populations -LRB- a putative true-positive set , H1 , and a putative background set , H0 -RRB- , we can estimate false-positive and false-negative rates in each set . This approach allows optimization of the membership of H0 and H1 to minimize the false-positive and false-negative rates , as follows . -LRB- i -RRB- For an initial value of -LRB- e.g. , 0.05 -RRB- , determine the weight vector as described in Weighed Integration of P Values . -LRB- ii -RRB- Assume that the full set of observed data elements -LRB- green histogram in Fig. 2A -RRB- divides into two sets H1 and H0 . -LRB- iii -RRB- The PDF of observed data elements can be approximated by a `` mixture model '' -LRB- MM -RRB- , M : PDF -LRB- M -RRB- -LRB- 1 -RRB- PDF -LRB- H0 -RRB- PDF -LRB- H1 -RRB- , where 0 1 is the proportion of H1 in M -LRB- red curve in Fig. 2 A -RRB- . -LRB- iv -RRB- For a given set of weights , the PDF of H0 -LRB- dashed blue curve in Fig. 2 A -RRB- can be evaluated numerically by Monte Carlo sampling of a uniform distribution -LRB- or the corresponding 2 , T , or Z distributions , see Methods for details -RRB- . -LRB- v -RRB- Guess an initial value for . -LRB- vi -RRB- Estimate the PDF of H1 -LRB- black curve in Fig. 2 A -RRB- as a gamma distribution that best satisfies the relationship in step 2 above using ESA -LRB- see Methods -RRB- . -LRB- vii -RRB- Determine T as the right-hand tail of H0 measured from the point where H1 and H0 PDFs meet -LSB- marked as c -LRB- T -RRB- in Fig. 2 A -RSB- . -LRB- viii -RRB- Repeat steps 1 7 -LRB- repeat step 1 by using T -RRB- until T estimates converge . Finally , the significant elements are selected by using the decision boundary given by T. Fig. 2B shows the decision boundaries arrived at through the 0.015 -RRB- and for 0.05 for above procedure -LRB- black curve , T comparison . Note that MM-based estimation of T reduces the number of false positives considerably while incurring relatively few additional false negatives -LRB- see Table 1 , which is published as supporting information on the PNAS web site -RRB- . The explicit MM also has fewer false negatives than the implicit method presented in the previous section -LRB- see Fig. 4 for a quantitative comparison using more demanding data that confirm that the MM method provides better overall performance -RRB- . Using a NP Decision Boundary . The above methods are all based on 8 , the choice of the - threshold is constrained by the opposing needs to minimize false positives and false negatives . A large value can produce a high false-positive rate , whereas a small value can lead to a high false-negative rate . To estimate a suitable threshold value -LRB- T -RRB- , we note that true-positive data elements cluster near the axes whereas the remaining data elements tend to be uniformly distributed . Therefore , one way to select T would be to find the value of for which the resulting decision boundary best separates uniformly distributed data elements from the nonuniform distribution of data elements near the axes . For a given , the cumulative density of the data elements measures the fraction of data elements selected . The cumulative expected density of the background -LRB- uniformly distributed -RRB- data elements is . Thus to compute T , we calculate the ratios of cumulative observed densities to cumulative expected densities as is increased from zero . T is the x-axis value of the point nearest to the origin -LRB- see Methods and Fig. 11 -RRB- . Because the data integration weights are calculated for a given , it is necessary to recalculate the weights for T. For these recalculated weights , another T is determined . This procedure is repeated until T converges . The black line in Fig. 1C shows the decision boundary arrived at using this method for selecting T. Note how this boundary is much more stringent , resulting in fewer false positives . Improved Threshold Significance Level Selection by Using Mixture Distribution Models . The above approach to estimating T uses only the distribution of the putative background data elements . As such , this approach focuses on minimizing the false positive rate only . By 17300 www.pnas.org cgi doi 10.1073 pnas .0508647102 smooth parametric decision boundaries derived from assigning a reliability parameter -LRB- the weight -RRB- to each type of data . In practice , there are many sources of error for a given measurement error , in turn affecting the resulting P values in an irregular manner . In such cases , smooth parametric decision boundaries may be unable to accommodate these irregularities . To allow estimation of nonsmooth-decision boundaries , we have developed a heuristic NP method , as follows -LRB- see Methods for details -RRB- . First , we construct a first-pass set of candidate true-positive data elements C whose P values are less than a threshold T -LRB- see Methods -RRB- in at least one data set . In Fig. 3A , these are the data elements between the axes and the dashed blue lines . The remaining elements are grouped into the complement data set Cc . We also generate a set of random background elements -LRB- B , see Methods -RRB- . Second , we compute a weight for C and B proportional to the nonoverlapping area between the two PDFs of transformed P values of the elements in C and B -LRB- see Methods -RRB- . Third , we compute the combined P value statistic -LRB- Sw -RRB- . Fourth , we quantify the statistical power of the current memberships of C and B as the nonoverlapping area between the corresponding Sw PDFs of C and B. Fifth , if the nonoverlapping area -LRB- f -RRB- is less than a user-specified threshold -LRB- e.g. , fc 0.99 -RRB- , we remove putative false-positive elements with high Sw values from C and add them to Cc . The green crosses in Fig. 3A indicate data elements retained in C after this step . Sixth , steps 2 through 5 are repeated by using the new C and Cc sets until f reaches fc . Finally , once the iteration process is terminated , potential false negatives are identified as elements in Cc whose Sw values exceed that of the point where the Sw PDFs of C and Cc meet . These elements are then added to the final set of candidate true-positive data elements C. Fig. 3B shows the final set of selected data elements -LRB- green crosses -RRB- . For comparison , the dashed blue curve shows the decision boundary calculated using the MM -LRB- see previous section -RRB- . Hwang et al. . The above procedure is essentially the same as the parametric method described in the preceding section . The main difference is that , rather than moving a decision boundary -LRB- as in previous methods -RRB- , here we move individual data elements between C and Cc . The final decision boundary is irregular because elements in a different part of the P value space are removed as the weights change in each iteration . To speed up the optimization process and avoid potential local minima in the search process , we have limited our optimization process to pruning . To be sure that the final set C includes as many of the true-positive data elements as possible , we initialize C by using a value for T that assures the inclusion of most true data elements at the expense of many false positives . These false positives are then removed iteratively . An adaptive rule is used to determine the fraction of C removed in step 5 : Cf min -LSB- -LRB- fc f -RRB- fc ,0.01 -RSB- . This rule removes 1 % of C in the beginning but decreases the proportion of elements moved as f approaches fc . Comparison of the Performance of Different Integration Methods . amounts of noise , whereas the contour gradient between the weighted Fisher 's method and the MM and NP methods highlights the importance of optimizing T. Overall , MM and NP perform best for minimization of false-positive and false-negative rates , followed by the thresholded Fisher 's method , which mainly reduces the false-positive rate . Handling Missing Values . Missing data points can arise from systematic biases in high-throughput technologies , e.g. , favoring high abundance species . Missing values also occur when we integrate global data sets with curated database information . For methods that combine independent , uniformly distributed P values into a uniformly distributed P value , missing values may be handled by using the available P values to compute each combined P value . The simplest way of handling these missing values would be to assign a fixed P value in place of the missing data -LRB- e.g. , P 0.5 to indicate an equal chance of being a true positive or a true negative , or P 1.0 to discourage elements with missing values from being selected -RRB- . If one or more data sets include many missing values , the above approach can distort the calculated weights . To avoid such a scenario , we exclude data elements with missing values from the calculation of weights , thereby avoiding the above distortion problem . However , we include them in the data selection process so that data elements with missing values can still be selected . The example data used in the previous figures were intentionally simplistic for ease of understanding . To highlight differences in the methods when applied to more complex data sets , here we compare the performance of the methods discussed using 10 examples of integration of three data sets . The individual data sets were generated by using the same scheme as described earlier , but with approximately three to four times more noise and less distinct populations of true positives -LRB- reduced a and b parameters for the gamma distributions -RRB- . See Methods for details . Fig. 4A shows the distribution of the true-positive data elements for an example case . Fig. 4B summarizes the performance of eight different data integration methods on the above 10 examples . For each run of each method , the false-positive rate is plotted along the x axis and the true-positive rate is plotted along the y axis . The 10 examples are identified by numbers 1 10 . Each method is identified by a different marker shape and color -LRB- see legend -RRB- . Instances of the letter M mark the mean performance of the correspondingly colored method . In a manner similar to receiver operating characteristic graphs -LRB- 21 -RRB- , ideal performance is given by X 0 , Y 1 . The Euclidean distance from this point represents the degree of performance degradation from the ideal . The dashed contour lines in Fig. 4B represent loci of equal performance as defined by the above measure . The NP and MM methods outperform all other methods on a case-by-case basis and based on average performance . The performance of the union method depends crucially on the choice of 0.05 -RRB- with U2 -LRB- 0.15 -RRB- , which significance level -LSB- compare U1 -LRB- happens to be close to the of the MM and NP methods -RSB- . The 0.05 -RRB- performs worse than intersection method I1 -LRB- for which 0.053 -RRB- is not indicated all other methods . I2 performance -LRB- because its true-positive rate was significantly 0.5 . As expected , the thresholded Fisher 's method , in which we minimize the number of false positives automatically , produced the smallest false-positive error rate at the expense of lowering the true-positive rate . However , as shown in Table 1 , for less noisy data , the false-positive reduction method can be very effective . Finally , comparison of the weighted and unweighted Fisher 's methods highlights the performance advantage of the weighted method for data sets with unequal 1 . Hood , L. , Heath , J. R. , Phelps , M. E. & Lin , B. -LRB- 2004 -RRB- Science 306 , 640 643 . 2 . Ideker , T. , Galitski , T. & Hood , L. -LRB- 2001 -RRB- Annu . Rev. Genomics Hum . Genet . 2 , 343 372 . 3 . Lee , T. I. , Rinaldi , N. J. , Robert , F. , Odom , D. T. , Bar-Joseph , Z. , Gerber , G. K. , Hannett , N. M. , Harbison , C. T. , Thompson , C. M. , Simon , I. , et al. -LRB- 2002 -RRB- Science 298 , 799 804 . 4 . von Mering , C. & Bork , P. -LRB- 2002 -RRB- Nature 417 , 797 798 . 5 . Bar-Joseph , Z. , Gerber , G. K. , Lee , T. I. , Rinaldi , N. J. , Yoo , J. Y. , Robert , F. , Gordon , D. B. , Fraenkel , E. , Jaakkola , T. S. , Young , R. A. & Gifford , D. K. -LRB- 2003 -RRB- Nat . Biotechnol . 21 , 1337 1342 . 6 . von Mering , C. , Krause , R. , Snel , B. , Cornell , M. , Oliver , S. G. , Fields , S. & Bork , P. -LRB- 2002 -RRB- Nature 417 , 399 403 . 7 . Mrowka , R. , Patzak , A. & Herzel , H. -LRB- 2001 -RRB- Genome Res . 11 , 1971 1973 . 8 . Deane , C. M. , Salwinski , L. , Xenarios , I. & Eisenberg , D. -LRB- 2002 -RRB- Mol . Cell Proteomics 1 , 349 356 . 9 . Gilchrist , M. A. , Salter , L. A. & Wagner , A. -LRB- 2004 -RRB- Bioinformatics 20 , 689 700 . 10 . Hwang , D. , Smith , J. J. , Leslie , D. M. , Weston , A. D. , Rust , A. G. , Ramsey , S. , de Atauri , P. , Siegel , A. F. , Bolouri , H. , Aitchison , J. D. & Hood , L. -LRB- 2005 -RRB- Proc . Natl. Acad . Sci . USA 102 , 17302 17307 . We thank Frederick Roth for insightful and critical analysis of an earlier version of our data integration methodology . A.F.S. holds the Grant I. Butterbaugh Professorship at the University of Washington . 11 . Hedges , L. & Olkin , I. -LRB- 1985 -RRB- Stat . Method Meta-Analysis -LRB- Academic , San Diego -RRB- . 12 . Birnbaum , A. -LRB- 1954 -RRB- J. Am . Stat . Assoc. 49 , 559 574 . 13 . Siarry , P. , Berthiau , G. , Durdin , F. & Haussy , J. -LRB- 1997 -RRB- ACM Trans . Math . Software 23 , 209 228 . 14 . Devroye , L. -LRB- 1997 -RRB- ACM Trans . Model . Comp . Simul . 7 , 447 477 . 15 . Hildebrand , F. -LRB- 1956 -RRB- Introduction to Numerical Analysis -LRB- McGraw Hill , New York -RRB- . 16 . Bailey , T. L. & Elkan , C. -LRB- 1994 -RRB- Proc . Int . Conf . Intell . Syst . Mol . Biol . 2 , 28 36 . 17 . Keller , A. , Nesvizhskii , A. I. , Kolker , E. & Aebersold , R. -LRB- 2002 -RRB- Anal . Chem . 74 , 5383 5392 . 18 . Bowman , A. W. & Azzalini , A. -LRB- 1997 -RRB- Applied Smoothing Techniques for Data Analysis -LRB- Oxford Univ. . Press , Oxford -RRB- . 19 . Jansen , R. , Yu , H. , Greenbaum , D. , Kluger , Y. , Krogan , N. J. , Chung , S. , Emili , A. , Snyder , M. , Greenblatt , J. F. & Gerstein , M. -LRB- 2003 -RRB- Science 302 , 449 453 . 20 . Ideker , T. , Thorsson , V. , Ranish , J. A. , Christmas , R. , Buhler , J. , Eng , J. K. , Bumgarner , R. , Goodlett , D. R. , Aebersold , R. & Hood , L. -LRB- 2001 -RRB- Science 292 , 929 934 . 21 . Swets , J. A. -LRB- 1988 -RRB- Science 240 , 1285 1293 . Hwang et al. . PNAS November 29 , 2005 vol . 102 no. 48 17301 APPLIED BIOLOGICAL SCIENCES Conclusions We have presented a generalized framework for data integration in systems biology . The applicability of our methodology to different types and sizes of data and to different numbers of data sets is demonstrated by application to five different types of data integration in a companion paper -LRB- 10 -RRB- . Although we focused here on presenting our methodology from the perspective of maximizing statistical power , it can also be applied to scenarios for which the different types of data being integrated have systematic differences between them , for example , combining mRNA and protein abundance measurements or in vivo and in vitro measurements . Examples of this type of integration are given in the companion paper -LRB- 10 -RRB- . Data integration can never rule out inclusion of some false positives or loss of some true positives . Our methodology provides a framework for optimizing the tradeoff between these opposing demands . We have implemented all of the methods presented in a free open source software package -LRB- POINTILLIST -RRB- that allows users to select the integration method most appropriate to their needs . Our methodology provides a simple and efficient means for combining multiple sets of noisy data to produce probabilistic models . The final outcome of our data integration procedure is a network model in which nodes represent biomolecular species -LRB- e.g. , genes or proteins -RRB- and edges represent interactions -LRB- e.g. , transcriptional regulation -RRB- . Our methodology associates a P value with each node and edge in the network model . These P values indicate the degree of confidence in a node or edge being a true component of the system of interest -LRB- compared with background control -RRB- .

O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O