Molecular

Systems

Biology

Article

number

doi

Citation

Molecular

Systems

Biology

EMBO

and

Nature

Publishing

Group

All

rights

reserved

REVIEW

How

to

infer

gene

networks

from

expression

profiles

Mukesh

Vincenzo

Alberto

and

Diego

di

Telethon

Institute

of

Genetics

and

Medicine

Via

P

Castellino

Naples

Italy

European

School

of

Molecular

Medicine

Naples

Italy

Department

of

Natural

Sciences

University

of

Naples

Federico

II

Naples

Italy

and

Department

of

Neuroscience

University

of

Naples

Federico

II

Naples

Italy

These

authors

contributed

equally

to

this

work

Corresponding

author

Systems

Biology

Lab

Telethon

Institute

of

Genetics

and

Medicine

Via

P

Castellino

Naples

Italy

Tel.

39 081 6132 319

Fax

39 081 6132 351

E-mail

dibernardo@tigem.it

Received

accepted

Inferring

or

reverse-engineering

gene

networks

can

be

defined

as

the

process

of

identifying

gene

interactions

from

experimental

data

through

computational

analysis

Gene

expression

data

from

microarrays

are

typically

used

for

this

purpose

Here

we

compared

different

reverseengineering

algorithms

for

which

ready-to-use

software

was

available

and

that

had

been

tested

on

experimental

data

sets

We

show

that

reverse-engineering

algorithms

are

indeed

able

to

correctly

infer

regulatory

interactions

among

genes

at

least

when

one

performs

perturbation

experiments

complying

with

the

algorithm

requirements

These

algorithms

are

superior

to

classic

clustering

algorithms

for

the

purpose

of

finding

regulatory

interactions

among

genes

and

although

further

improvements

are

needed

have

reached

a

discreet

performance

for

being

practically

useful

Molecular

Systems

Biology

February

doi

Subject

Categories

metabolic

and

regulatory

networks

simulation

and

data

analysis

Keywords

gene

network

reverse-engineering

gene

expression

transcriptional

regulation

gene

regulation

Introduction

Gene

expression

microarrays

yield

quantitative

and

semiquantitative

data

on

the

cell

status

in

a

specific

condition

and

time

Molecular

biology

is

rapidly

evolving

into

a

quantitative

science

and

as

such

it

is

increasingly

relying

on

engineering

and

physics

to

make

sense

of

high-throughput

data

The

aim

is

to

infer

or

reverse-engineer

from

gene

expression

data

the

regulatory

interactions

among

genes

using

computational

algorithms

There

are

two

broad

classes

of

reverse-engineering

algorithms

-LRB-

Faith

and

Gardner

-RRB-

those

based

on

the

physical

interaction

approach

that

aim

at

identifying

interactions

among

transcription

factors

and

their

target

genes

EMBO

and

Nature

Publishing

Group

-LRB-

gene-to-sequence

interaction

-RRB-

and

those

based

on

the

influence

interaction

approach

that

try

to

relate

the

expression

of

a

gene

to

the

expression

of

the

other

genes

in

the

cell

-LRB-

gene-to-gene

interaction

-RRB-

rather

than

relating

it

to

sequence

motifs

found

in

its

promoter

-LRB-

gene-to-sequence

-RRB-

We

will

refer

to

the

ensemble

of

these

influence

interactions

as

gene

networks

The

interaction

between

two

genes

in

a

gene

network

does

not

necessarily

imply

a

physical

interaction

but

can

also

refer

to

an

indirect

regulation

via

proteins

metabolites

and

ncRNA

that

have

not

been

measured

directly

Influence

interactions

include

physical

interactions

if

the

two

interacting

partners

are

a

transcription

factor

and

its

target

or

two

proteins

in

the

same

complex

Generally

however

the

meaning

of

influence

interactions

is

not

well

defined

and

depends

on

the

mathematical

formalism

used

to

model

the

network

Nonetheless

influence

networks

do

have

practical

utility

for

-LRB-

-RRB-

identifying

functional

modules

that

is

identify

the

subset

of

genes

that

regulate

each

other

with

multiple

-LRB-

indirect

-RRB-

interactions

but

have

few

regulations

to

other

genes

outside

the

subset

-LRB-

-RRB-

predicting

the

behaviour

of

the

system

following

perturbations

that

is

gene

network

models

can

be

used

to

predict

the

response

of

a

network

to

an

external

perturbation

and

to

identify

the

genes

directly

hit

by

the

perturbation

-LRB-

di

Bernardo

et

al

-RRB-

a

situation

often

encountered

in

the

drug

discovery

process

where

one

needs

to

identify

the

genes

that

are

directly

interacting

with

a

compound

of

interest

-LRB-

-RRB-

identifying

real

physical

interactions

by

integrating

the

gene

network

with

additional

information

from

sequence

data

and

other

experimental

data

-LRB-

i.e.

chromatin

immunoprecipitation

yeast

two-hybrid

assay

etc.

-RRB-

In

addition

to

reverse-engineering

algorithms

network

visualisation

tools

are

available

online

to

display

the

network

surrounding

a

gene

of

interest

by

extracting

information

from

the

literature

and

experimental

data

sets

such

as

Cytoscape

-LRB-

Shannon

et

al

-RRB-

-LRB-

-RRB-

and

Osprey

-LRB-

Breitkreutz

et

al

-RRB-

-LRB-

ca/osprey/servlet

Index

-RRB-

Here

we

will

focus

on

gene

network

inference

algorithms

-LRB-

the

influence

approach

-RRB-

A

description

of

other

methods

based

on

the

physical

approach

and

more

details

on

computational

aspects

can

be

found

in

-LRB-

Beer

and

Tavazoie

Tadesse

et

al

Faith

and

Gardner

Prakash

and

Tompa

Ambesi

and

di

Bernardo

Foat

et

al

-RRB-

We

will

also

briefly

describe

two

improper

reverseengineering

tools

-LRB-

MNI

and

TSNI

-RRB-

whose

main

focus

is

not

inferring

interactions

among

genes

from

gene

expression

data

but

rather

identification

of

the

targets

of

the

perturbation

-LRB-

point

-LRB-

-RRB-

above

-RRB-

Among

the

plethora

of

algorithms

proposed

in

the

literature

to

solve

the

network

inference

problem

we

selected

one

algorithm

for

each

class

of

mathematical

formalism

proposed

in

the

literature

for

which

ready-to-use

software

is

available

and

that

had

been

tested

on

experimental

data

sets

Molecular

Systems

Biology

How

to

infer

gene

networks

from

expression

profiles

M

Bansal

et

al

Gene

network

inference

algorithms

We

will

indicate

the

gene

expression

measurement

of

gene

i

with

the

variable

xi

the

set

of

expression

measurements

for

all

the

genes

with

D

and

the

interaction

between

genes

i

and

j

with

aij

D

may

consist

of

time-series

gene

expression

data

of

N

genes

in

M

time

points

-LRB-

i.e.

gene

expression

changing

dynamically

with

time

-RRB-

or

measurements

taken

at

steadystate

in

M

different

conditions

-LRB-

i.e.

gene

expression

levels

in

homeostasis

-RRB-

Some

inference

algorithms

can

work

on

both

kind

of

data

whereas

others

have

been

specifically

designed

to

analyse

one

or

the

other

Depending

on

the

inference

algorithm

used

the

resulting

gene

network

can

be

either

an

undirected

graph

that

is

the

direction

of

the

interaction

is

not

specified

-LRB-

aij

aji

-RRB-

or

a

directed

graph

specifying

the

direction

of

the

interaction

that

is

gene

j

regulates

gene

i

-LRB-

and

not

vice

versa

-RRB-

-LRB-

aijaaji

-RRB-

A

directed

graph

can

also

be

labeled

with

a

sign

and

strength

for

each

interaction

signed

directed

graph

where

aij

has

a

positive

zero

or

negative

value

indicating

activation

no

interaction

and

repression

respectively

An

overview

of

softwares

described

here

is

given

in

Figure

and

Table

I.

understand

what

can

be

gained

by

advanced

gene

network

inference

algorithms

whose

aim

is

to

infer

direct

interactions

among

genes

as

compared

with

simple

clustering

for

the

purpose

of

gene

network

inference

The

most

common

clustering

approach

is

hierarchical

clustering

-LRB-

Eisen

et

al

-RRB-

where

relationships

among

genes

are

represented

by

a

tree

whose

branch

lengths

reflect

the

degree

of

similarity

between

genes

as

assessed

by

a

pairwise

similarity

function

such

as

Pearson

correlation

coefficient

xi

k

xj

k

k

rij

sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

M

M

P

P

k

xj

k

i

k

k

M

P

Coexpression

networks

and

clustering

algorithms

Clustering

although

not

properly

a

network

inference

algorithm

is

the

current

method

of

choice

to

visualise

and

analyse

gene

expression

data

Clustering

is

based

on

the

idea

of

grouping

genes

with

similar

expression

profiles

in

clusters

-LRB-

Eisen

et

al

-RRB-

Similarity

is

measured

by

a

distance

metric

as

for

example

the

correlation

coefficient

among

a

pair

of

genes

The

number

of

clusters

can

be

set

either

automatically

or

by

the

user

depending

on

the

clustering

algorithm

used

-LRB-

Eisen

et

al

Amato

et

al

-RRB-

The

rationale

behind

clustering

is

that

coexpressed

genes

-LRB-

i.e.

genes

in

the

same

cluster

-RRB-

have

a

good

probability

of

being

functionally

related

-LRB-

Eisen

et

al

-RRB-

This

does

not

imply

however

that

there

is

a

direct

interaction

among

the

coexpressed

genes

as

genes

separated

by

one

or

more

intermediaries

-LRB-

indirect

relationships

-RRB-

may

be

highly

coexpressed

It

is

therefore

important

to

For

a

set

of

n

profiles

all

the

pairwise

correlation

coefficients

rij

are

computed

the

highest

value

-LRB-

representing

the

most

similar

pair

of

genes

-RRB-

is

selected

and

a

node

in

the

tree

is

created

for

this

gene

pair

with

a

new

expression

profile

given

by

the

average

of

the

two

profiles

The

process

is

repeated

by

replacing

the

two

genes

with

a

single

node

and

all

pairwise

correlations

among

the

n

profiles

-LRB-

i.e.

n

profiles

from

single

genes

plus

of

the

gene

pair

-RRB-

are

computed

The

process

stops

when

only

one

element

remains

Clusters

are

obtained

by

cutting

the

tree

at

a

specified

branch

level

In

order

to

compare

clustering

to

the

other

network

inference

strategies

we

assumed

that

genes

in

the

same

clusters

regulate

each

other

that

is

each

gene

represents

a

node

in

the

network

and

is

connected

to

all

the

other

genes

in

the

same

cluster

Clustering

will

thus

recover

an

undirected

graph

Bayesian

networks

A

Bayesian

network

is

a

graphical

model

for

probabilistic

relationships

among

a

set

of

random

variables

Xi

where

i

y

n

These

relationships

are

encoded

in

the

structure

of

a

directed

acyclic

graph

G

whose

vertices

-LRB-

or

nodes

-RRB-

are

the

random

variables

Xi

The

relationships

between

the

variables

START

-LRB-

Expression

data

-RRB-

Infer

a

gene

network

No

Predict

targets

of

a

perturbation

Yes

Yes

ARACNE

-LRB-

-RRB-

BANJO

-LRB-

DBN

-RRB-

CLUSTERING

Time

series

What

kind

of

expression

data

Steady

state

What

kind

of

expression

data

Steady

state

Time

series

TSNI

ARACNE

BANJO

-LRB-

BN

-RRB-

NIR

CLUSTERING

MNI

NIR

Figure

Flowchart

to

choose

the

most

suitable

network

inference

algorithms

according

to

the

problem

to

be

addressed

-LRB-

-RRB-

check

for

independence

of

time

points

-LRB-

see

text

for

details

-RRB-

-LRB-

BN

-RRB-

Bayesian

networks

-LRB-

DBN

-RRB-

Dynamic

Bayesian

Networks

Molecular

Systems

Biology

EMBO

and

Nature

Publishing

Group

How

to

infer

gene

networks

from

expression

profiles

M

Bansal

et

al

Table

I

Features

of

the

network

inference

algorithms

reviewed

in

this

tutorial

Software

BANJO

ARACNE

NIR/MNIa

Download

software/banjo

tgardner@bu.edu

Data

type

S/D

S/D

S

Command

line

java-jar

banjo.jar

settingFile

mysettings.txt

arance-i

inputfile-o

outputfile

-LSB-

options

-RSB-

MATLAB

Notes

Good

performance

if

large

datasets

is

available

-LRB-

MbN

-RRB-

Good

performance

even

for

MpN

Not

useful

for

short

time

series

NIR

good

performance

but

requires

knowledge

of

perturbed

genes/MNI

good

performance

for

inferring

targets

of

a

perturbation

Useful

for

finding

coexpressed

genes

but

not

for

network

inference

Hierarchical

clustering

mde-hoon/software/cluster

S/D

GUI

Abbreviations

D

dynamic

time-series

N

number

of

genes

M

number

of

experiments

S

steay-state

a

Predicts

only

targets

of

a

perturbation

-LRB-

see

text

for

details

-RRB-

Bayesian

Networks

D

E

Information-theoretic

D

E

Ordinary

differential

equations

D

E

B

C

B

C

B

C

H

A

H

A

H

A

F

G

F

G

F

G

P

-LRB-

A/B

C

D

E

-RRB-

=P

-LRB-

A/B

C

-RRB-

MI

-LRB-

A

H

-RRB-

MI

-LRB-

A

B

-RRB-

MI

-LRB-

A

D

-RRB-

min

-LCB-

MI

-LRB-

A

B

-RRB-

MI

-LRB-

B

D

-RRB-

-RCB-

dA/dt

B

C

or

more

generally

dA/dt

f

-LRB-

A

B

C

-RRB-

Figure

Bayesian

networks

A

is

conditionally

independent

of

D

and

E

given

B

and

C

information-theoretic

networks

mutual

information

is

for

statistically

independent

variables

and

Data

Processing

Inequality

helps

pruning

the

network

ordinary

differential

equations

deterministic

approach

where

the

rate

of

transcription

of

gene

A

is

a

function

-LRB-

f

-RRB-

of

the

level

of

its

direct

causal

regulators

are

described

by

a

joint

probability

distribution

P

-LRB-

y

Xn

-RRB-

that

is

consistent

with

the

independence

assertions

embedded

in

the

graph

G

and

has

the

form

P

Xn

N

Y

i

P

Xi

xi

jjXj

xj

Xj

p

xj

p

where

the

p

genes

on

which

the

probability

is

conditioned

are

called

the

parents

of

gene

i

and

represent

its

regulators

and

the

joint

probability

density

is

expressed

as

a

product

of

conditional

probabilities

by

applying

the

chain

rule

of

probabilities

and

independence

This

rule

is

based

on

the

Bayes

theorem

P

A

B

P

BjjA

P

A

P

AjjB

P

B

We

observe

that

the

JPD

-LRB-

joint

probability

distribution

-RRB-

can

be

decomposed

as

the

product

of

conditional

probabilities

as

in

Equation

only

if

the

Markov

assumption

holds

that

is

each

variable

Xi

is

independent

of

its

non-descendants

given

its

parent

in

the

directed

acyclic

graph

G

A

schematic

overview

of

the

theory

underlying

Bayesian

networks

is

given

in

Figure

EMBO

and

Nature

Publishing

Group

In

order

to

reverse-engineer

a

Bayesian

network

model

of

a

gene

network

we

must

find

the

directed

acyclic

graph

G

-LRB-

i.e.

the

regulators

of

each

transcript

-RRB-

that

best

describes

the

gene

expression

data

D

where

D

is

assumed

to

be

a

steady-state

data

set

This

is

performed

by

choosing

a

scoring

function

that

evaluates

each

graph

G

-LRB-

i.e.

a

possible

network

topology

-RRB-

with

respect

to

the

gene

expression

data

D

and

then

searching

for

the

graph

G

that

maximises

the

score

The

score

can

be

defined

using

the

Bayes

rule

P

G

=D

P

D

G

P

G

where

P

-LRB-

G

-RRB-

can

either

contain

some

a

P

D

priori

knowledge

on

network

structure

if

available

or

can

be

a

constant

non-informative

prior

and

P

-LRB-

D/G

-RRB-

is

a

function

to

be

chosen

by

the

algorithm

that

evaluates

the

probability

that

the

data

D

has

been

generated

by

the

the

graph

G

The

most

popular

scores

are

the

Bayesian

Information

Criteria

-LRB-

BIC

-RRB-

or

Bayesian

Dirichlet

equivalence

-LRB-

BDe

-RRB-

Both

scores

incorporate

a

penalty

for

complexity

to

guard

against

overfitting

of

data

Trying

out

all

possible

combinations

of

interaction

among

genes

that

is

all

possible

graphs

G

and

choosing

the

G

with

Molecular

Systems

Biology

How

to

infer

gene

networks

from

expression

profiles

M

Bansal

et

al

the

maximum

Bayesian

score

is

an

NP-hard

problem

Therefore

a

heuristic

search

method

is

used

like

the

greedy-hill

climbing

approach

the

Markov

Chain

Monte

Carlo

method

or

simulated

annealing

In

Bayesian

networks

the

learning

problem

is

usually

underdetermined

and

several

high-scoring

networks

are

found

To

address

this

problem

one

can

use

model

averaging

or

bootstrapping

to

select

the

most

probable

regulatory

interactions

and

to

obtain

confidence

estimates

for

the

interactions

For

example

if

a

particular

interaction

between

two

transcripts

repeatedly

occurs

in

high-scoring

models

one

gains

confidence

that

this

edge

is

a

true

dependency

Alternatively

one

can

augment

an

incomplete

data

set

with

prior

information

to

help

select

the

most

likely

model

structure

Bayesian

networks

can

not

contain

cycles

-LRB-

i.e.

no

feedback

loops

-RRB-

This

restriction

is

the

principal

limitation

of

the

Bayesian

network

models

Dynamic

Bayesian

networks

overcome

this

limitation

Dynamic

Bayesian

networks

are

an

extension

of

Bayesian

networks

able

to

infer

interactions

from

a

data

set

D

consisting

of

time-series

rather

than

steady-state

data

We

refer

the

reader

to

-LRB-

Yu

et

al

-RRB-

A

word

of

caution

Bayesian

networks

model

probabilistic

dependencies

among

variables

and

not

causality

that

is

the

parents

of

a

node

are

not

necessarily

also

the

direct

causes

of

its

behaviour

However

we

can

interpret

the

edge

as

a

causal

link

if

we

assume

that

the

Causal

Markov

Condition

holds

This

can

be

stated

simply

as

a

variable

X

is

independent

of

every

other

variable

-LRB-

except

the

targets

of

X

-RRB-

conditional

on

all

its

direct

causes

It

is

not

known

whether

this

assumption

is

a

good

approximation

of

what

happens

in

real

biological

networks

For

more

information

and

a

detailed

study

of

Bayesian

networks

for

gene

network

inference

we

refer

the

reader

to

Pe'er

et

al

-LRB-

-RRB-

Information-theoretic

approaches

Information-theoretic

approaches

use

a

generalisation

of

pairwise

correlation

coefficient

in

equation

-LRB-

-RRB-

called

Mutual

Information

-LRB-

MI

-RRB-

to

compare

expression

profiles

from

a

set

of

microarrays

For

each

pair

of

genes

their

MIij

is

computed

and

the

edge

aij

aji

is

set

to

or

depending

on

a

significance

threshold

to

which

MIij

is

compared

MI

can

be

used

to

measure

the

degree

of

independence

between

two

genes

Mutual

information

MIij

between

gene

i

and

gene

j

is

computed

as

MIi

j

Hi

Hj

Hij

where

H

the

entropy

is

defined

as

n

X

p

xk

log

p

xk

Hi

k

Banjo

Banjo

is

a

gene

network

inference

software

that

has

been

developed

by

the

group

of

Hartemink

-LRB-

Yu

et

al

-RRB-

Banjo

is

based

on

Bayesian

networks

formalism

and

implements

both

Bayesian

and

Dynamic

Bayesian

networks

Therefore

it

can

infer

gene

networks

from

steady-state

gene

expression

data

or

from

time-series

gene

expression

data

In

Banjo

heuristic

approaches

are

used

to

search

the

network

space

to

find

the

network

graph

G

-LRB-

Proposer

Searcher

module

in

Banjo

-RRB-

For

each

network

structure

explored

the

parameters

of

the

conditional

probability

density

distribution

are

inferred

and

an

overall

network

's

score

is

computed

using

the

BDe

metric

in

Banjo

's

Evaluator

module

The

output

network

will

be

the

one

with

the

best

score

-LRB-

Banjo

's

Decider

module

-RRB-

Banjo

outputs

a

signed

directed

graph

indicating

regulation

among

genes

Banjo

can

analyse

both

steady-state

and

timeseries

data

In

the

case

of

steady-state

data

Banjo

as

well

as

the

other

Bayesian

networks

algorithms

is

not

able

to

infer

networks

involving

cycles

-LRB-

e.g.

feedback

or

feed

forward

loops

-RRB-

Other

Bayesian

network

inference

algorithms

for

which

software

is

available

have

been

proposed

-LRB-

Murphy

Friedman

and

Elidan

-RRB-

Molecular

Systems

Biology

The

entropy

Hi

has

many

interesting

properties

specifically

it

reaches

a

maximum

for

uniformly

distributed

variables

that

is

the

higher

the

entropy

the

more

randomly

distributed

are

gene

expression

levels

across

the

experiments

From

the

definition

it

follows

that

MI

becomes

zero

if

the

two

variables

xi

and

xj

are

statistically

independent

-LRB-

P

-LRB-

xixj

-RRB-

P

-LRB-

xi

-RRB-

P

-LRB-

xj

-RRB-

-RRB-

as

their

joint

entropy

Hij

Hi

Hj

A

higher

MI

indicates

that

the

two

genes

are

non-randomly

associated

to

each

other

It

can

be

easily

shown

that

MI

is

symmetric

Mij

Mji

therefore

the

network

is

described

by

an

undirected

graph

G

thus

differing

from

Bayesian

networks

-LRB-

directed

acyclic

graph

-RRB-

MI

is

more

general

than

the

Pearson

correlation

coefficient

This

quantifies

only

linear

dependencies

between

variables

and

a

vanishing

Pearson

correlation

does

not

imply

that

two

variables

are

statistically

independent

In

practical

application

however

MI

and

Pearson

correlation

may

yield

almost

identical

results

-LRB-

Steuer

et

al

-RRB-

The

definition

of

MI

in

equation

-LRB-

-RRB-

requires

each

data

point

that

is

each

experiment

to

be

statistically

independent

from

the

others

Thus

information-theoretic

approaches

as

described

here

can

deal

with

steady-state

gene

expression

data

set

or

with

time-series

data

as

long

as

the

sampling

time

is

long

enough

to

assume

that

each

point

is

independent

of

the

previous

points

Edges

in

networks

derived

by

information-theoretic

approaches

represent

statistical

dependences

among

geneexpression

profiles

As

in

the

case

of

Bayesian

network

the

edge

does

not

represent

a

direct

causal

interaction

between

two

genes

but

only

a

statistical

dependency

A

leap

of

faith

must

be

performed

in

order

to

interpret

the

edge

as

a

direct

causal

interaction

It

is

possible

to

derive

the

information-theoretic

approach

as

a

method

to

approximate

the

JPD

of

gene

expression

profiles

as

it

is

performed

for

Bayesian

networks

We

refer

the

interested

readers

to

Margolin

et

al

-LRB-

-RRB-

ARACNE

ARACNE

-LRB-

Basso

et

al

Margolin

et

al

-RRB-

belongs

to

the

family

of

information-theoretic

approaches

to

gene

network

inference

first

proposed

by

Butte

and

Kohane

-LRB-

-RRB-

with

their

relevance

network

algorithm

ARACNE

computes

Mij

for

all

pairs

of

genes

i

and

j

in

the

data

set

Mij

is

estimated

using

the

method

of

Gaussian

kernel

EMBO

and

Nature

Publishing

Group

How

to

infer

gene

networks

from

expression

profiles

M

Bansal

et

al

density

-LRB-

Steuer

et

al

-RRB-

Once

Mij

for

all

gene

pairs

has

been

computed

ARACNE

excludes

all

the

pairs

for

which

the

hypothesis

of

mutually

independent

genes

can

not

be

ruled

out

-LRB-

Ho

MIij

-RRB-

A

P-value

for

the

hypothesis

computed

using

Montecarlo

simulations

is

associated

to

each

value

of

the

mutual

information

The

final

step

of

this

algorithm

is

a

pruning

step

that

tries

to

reduce

the

number

of

false-positive

-LRB-

i.e.

inferred

interactions

among

two

genes

that

are

not

direct

causal

interaction

in

the

real

biological

pathway

-RRB-

They

use

the

Data

Processing

Inequality

-LRB-

DPI

-RRB-

principle

that

asserts

that

if

both

-LRB-

i

j

-RRB-

and

-LRB-

j

k

-RRB-

are

directly

interacting

and

-LRB-

i

k

-RRB-

is

indirectly

interacting

through

j

then

Mi

kpmin

-LRB-

Mij

Mjk

-RRB-

This

condition

is

necessary

but

not

sufficient

that

is

the

inequality

can

be

satisfied

even

if

-LRB-

i

k

-RRB-

are

directly

interacting

Therefore

the

authors

acknowledge

that

by

applying

this

pruning

step

using

DPI

they

may

be

discarding

some

direct

interactions

as

well

measurements

-LRB-

time-series

network

identification

TSNI

-RRB-

following

transcriptional

perturbations

to

reconstruct

gene

gene

interactions

and

to

identify

the

mediators

of

the

activity

of

a

drug

Other

algorithms

based

on

ODEs

have

been

proposed

in

the

literature

-LRB-

D'haeseleer

et

al

Tegner

et

al

Bonneau

et

al

van

Someren

et

al

-RRB-

The

network

is

described

as

a

system

of

linear

ODEs

-LRB-

de

Jong

-RRB-

representing

the

rate

of

synthesis

of

a

transcript

as

a

function

of

the

concentrations

of

every

other

transcript

in

a

cell

and

the

external

perturbation

X

xi

tk

aij

xj

tk

bi

u

tk

N

j

Ordinary

differential

equations

Reverse-engineering

algorithms

based

on

ordinary

differential

equations

-LRB-

ODEs

-RRB-

relate

changes

in

gene

transcript

concentration

to

each

other

and

to

an

external

perturbation

By

external

perturbation

we

mean

an

experimental

treatment

that

can

alter

the

transcription

rate

of

the

genes

in

the

cell

An

example

of

perturbation

is

the

treatment

with

a

chemical

compound

-LRB-

i.e.

a

drug

-RRB-

or

a

genetic

perturbation

involving

overexpression

or

downregulation

of

particular

genes

This

is

a

deterministic

approach

not

based

on

the

estimation

of

conditional

probabilities

unlike

Bayesian

networks

and

information-theoretic

approaches

A

set

of

ODEs

one

for

each

gene

describes

gene

regulation

as

a

function

of

other

genes

xi

t

fi

xN

u

yi

where

yi

is

a

set

of

parameters

describing

interactions

among

genes

-LRB-

the

edges

of

the

graph

-RRB-

i

y

N

xi

-LRB-

t

-RRB-

is

the

concentration

of

transcript

i

measured

at

time

t

xi

t

dxi

t

dt

is

the

rate

of

transcription

of

transcript

i

N

is

the

number

of

genes

and

u

is

an

external

perturbation

to

the

system

As

ODEs

are

deterministic

the

interactions

among

genes

-LRB-

yi

-RRB-

represent

causal

interactions

and

not

statistical

dependencies

as

the

other

methods

To

reverse-engineer

a

network

using

ODEs

means

to

choose

a

functional

form

for

fi

and

then

to

estimate

the

unknown

parameters

yi

for

each

i

from

the

gene

expression

data

D

using

some

optimisation

technique

The

ODE-based

approaches

yield

signed

directed

graphs

and

can

be

applied

to

both

steady-state

and

time-series

expression

profiles

Another

advantage

of

ODE

approaches

is

that

once

the

parameters

yi

for

all

i

are

known

equation

-LRB-

-RRB-

can

be

used

to

predict

the

behaviour

of

the

network

in

different

conditions

-LRB-

i.e.

gene

knockout

treatment

with

an

external

agent

etc.

-RRB-

where

i

y

N

k

y

M

N

is

the

number

of

genes

M

is

the

number

of

time

points

xi

-LRB-

tk

-RRB-

is

the

concentration

of

transcript

i

measured

at

time

tk

xi

-LRB-

tk

-RRB-

is

the

rate

of

change

of

concentration

of

gene

i

at

time

tk

that

is

the

first

derivative

of

the

mRNA

concentration

of

gene

i

measured

at

time

tk

aij

represents

the

influence

of

gene

j

on

gene

i

bi

represents

the

effect

of

the

external

pertrurbation

on

xi

and

u

-LRB-

tk

-RRB-

represents

the

external

perturbation

at

time

tk

-LRB-

aij

and

bi

are

the

y

in

equation

-LRB-

-RRB-

-RRB-

In

the

case

of

steady-state

data

xi

-LRB-

tk

-RRB-

and

equation

-LRB-

-RRB-

for

gene

i

becomes

independent

of

time

and

can

be

simplified

and

rewritten

in

the

form

of

a

linear

regression

N

X

j

aij

xj

bi

u

NIR

MNI

and

TSNI

In

recent

studies

-LRB-

Gardner

et

al

di

Bernardo

et

al

Bansal

et

al

-RRB-

ODE-based

algorithms

have

been

developed

-LRB-

Network

identification

by

multiple

regression

-LRB-

NIR

-RRB-

and

microarray

network

identifcation

-LRB-

MNI

-RRB-

-RRB-

that

use

a

series

of

steady-state

RNA

expression

measurements

or

time-series

EMBO

and

Nature

Publishing

Group

The

NIR

algorithm

-LRB-

Gardner

et

al

-RRB-

computes

the

edges

aij

from

steady-state

gene

expression

data

using

equation

-LRB-

-RRB-

NIR

needs

as

input

the

gene

expression

profiles

following

each

perturbation

experiment

-LRB-

xj

-RRB-

knowledge

of

which

genes

have

been

directly

perturbed

in

each

perturbation

experiment

-LRB-

biu

-RRB-

and

optionally

the

standard

deviation

of

replicate

measurements

NIR

is

based

on

a

network

sparsity

assumption

that

is

a

maximum

number

of

ingoing

edges

per

gene

-LRB-

i.e.

maximum

number

of

regulators

per

gene

-RRB-

which

can

be

chosen

by

the

user

The

output

is

in

matrix

format

where

each

element

is

the

edge

aij

The

inference

algorithm

reduces

to

solving

equation

-LRB-

-RRB-

for

the

unknown

parameters

aij

that

is

a

classic

linearregression

problem

The

MNI

algorithm

-LRB-

di

Bernardo

et

al

-RRB-

is

based

on

equation

-LRB-

-RRB-

as

well

and

uses

steady-state

data

like

NIR

but

importantly

each

microarray

experiment

can

result

from

any

kind

of

perturbation

that

is

we

do

not

require

knowledge

of

biu

MNI

is

different

from

other

inference

methods

as

the

inferred

network

is

used

not

per

se

but

to

filter

the

gene

expression

profile

following

a

treatment

with

a

compound

to

determine

pathways

and

genes

directly

targeted

by

the

compound

This

is

achieved

in

two

steps

In

the

first

step

the

parameters

aij

are

obtained

from

gene

expression

data

D

in

the

second

step

the

gene

expression

profile

following

compound

treatment

is

measured

-LRB-

xd

with

i

y

N

-RRB-

and

i

equation

-LRB-

-RRB-

is

used

to

compute

the

values

biu

for

each

i

as

aij

is

known

and

u

is

simply

a

constant

representing

the

treatment

bi

different

from

represents

the

genes

that

are

directly

hit

by

the

compound

The

output

is

a

ranked

list

of

genes

genes

at

the

top

of

the

list

are

the

most

likely

targets

of

the

compound

-LRB-

i.e.

the

ones

with

the

highest

value

of

bi

-RRB-

Molecular

Systems

Biology

How

to

infer

gene

networks

from

expression

profiles

M

Bansal

et

al

The

network

inferred

by

MNI

could

be

used

per

se

and

not

only

as

a

filter

however

if

we

do

not

have

any

knowledge

about

which

genes

have

been

perturbed

directly

in

each

perturbation

experiment

in

dataset

D

-LRB-

right-hand

side

in

equation

-LRB-

-RRB-

-RRB-

then

differently

from

NIR

the

solution

to

equation

-LRB-

-RRB-

is

not

unique

and

we

can

only

infer

one

out

of

many

possible

networks

that

can

explain

the

data

What

remains

unique

are

the

predictions

-LRB-

bi

-RRB-

that

is

all

the

possible

networks

predict

the

same

bi

MNI

performance

is

not

tested

here

not

being

a

proper

network

inference

algorithm

but

we

refer

the

interested

readers

to

di

Bernardo

et

al

-LRB-

-RRB-

where

the

performance

is

tested

in

detail

The

TSNI

-LRB-

Time

Series

Network

Identification

-RRB-

algorithm

-LRB-

Bansal

et

al

-RRB-

identifies

the

gene

network

-LRB-

aij

-RRB-

as

well

as

the

direct

targets

of

the

perturbations

-LRB-

bi

-RRB-

TSNI

is

based

on

equation

-LRB-

-RRB-

and

is

applied

when

gene

expression

data

are

dynamic

-LRB-

time-series

-RRB-

To

solve

equation

-LRB-

-RRB-

we

need

the

values

of

xi

-LRB-

tk

-RRB-

for

each

gene

i

and

each

time

point

k

This

can

be

estimated

directly

from

the

time-series

of

gene

expression

profiles

TSNI

assumes

that

a

single

perturbation

experiment

is

performed

-LRB-

e.g.

treatment

with

a

compound

gene

overexpression

etc.

-RRB-

and

M

time

points

following

the

perturbation

are

measured

-LRB-

rather

than

M

different

conditions

at

steady-state

as

for

NIR

and

MNI

-RRB-

For

small

networks

-LRB-

tens

of

genes

-RRB-

it

is

able

to

correctly

infer

the

network

structure

-LRB-

i.e.

aij

-RRB-

For

large

networks

-LRB-

hundreds

of

genes

-RRB-

its

performance

is

best

for

predicting

the

direct

targets

of

a

perturbation

-LRB-

i.e.

bi

-RRB-

-LRB-

for

example

finding

the

direct

targets

of

a

transcription

factor

from

gene

expression

time

series

following

overexpression

of

the

factor

-RRB-

TSNI

is

not

tested

here

but

we

refer

the

reader

to

Bansal

et

al

-LRB-

-RRB-

Reverse-engineering

algorithm

performance

We

performed

a

comparison

using

fake

gene

expression

data

generated

by

a

computer

model

of

gene

regulation

-LRB-

in

silico

data

-RRB-

The

need

of

simulated

data

arises

from

imperfect

knowledge

of

real

networks

in

cells

from

the

lack

of

suitable

gene

expression

data

set

and

of

control

on

the

noise

levels

In

silico

data

enable

one

to

check

the

performance

of

algorithms

against

a

perfectly

known

ground

truth

-LRB-

simulated

networks

in

the

computer

model

-RRB-

To

simulate

gene

expression

data

and

gene

regulation

in

the

form

of

a

network

we

use

linear

ODEs

relating

the

changes

in

gene

transcript

concentration

to

each

other

and

to

the

external

perturbations

Linear

ODEs

can

simulate

gene

networks

as

directed

signed

graphs

with

realistic

dynamics

and

generate

both

steady-state

and

time-series

gene

expression

profiles

Linear

ODEs

are

generic

as

any

non-linear

process

can

be

approximated

to

a

linear

process

as

long

as

the

system

is

not

far

from

equilibrium

whereas

non-linear

processes

are

all

different

from

each

other

There

are

many

other

choices

possible

-LRB-

Brazhnik

et

al

-RRB-

but

we

valued

the

capability

of

linear

ODEs

of

quickly

generating

many

random

networks

with

realistic

behaviour

and

the

availability

of

a

general

mathematical

theory

We

generated

random

networks

with

and

genes

and

with

an

average

in-degree

per

gene

of

and

respectively

For

each

network

we

generated

three

kinds

of

Molecular

Systems

Biology

data

steady-state-simulated

microarray

data

resulting

from

M

global

perturbations

-LRB-

i.e.

all

the

genes

in

the

network

are

perturbed

simultaneously

in

each

perturbation

experiment

-RRB-

steady-state

data

resulting

from

M

local

perturbations

-LRB-

i.e.

a

different

single

gene

in

the

network

is

perturbed

in

each

experiment

-RRB-

and

dynamic

time-series-simulated

microarray

data

resulting

from

perturbing

of

the

genes

simultaneously

and

measuring

M

time

points

following

the

perturbation

experiment

For

all

data

sets

M

was

chosen

equal

to

and

experiments

Noise

was

then

added

to

all

data

sets

by

summing

to

each

simulated

gene

expression

level

in

the

data

set

white

noise

with

zero

mean

and

standard

deviation

equal

to

multiplied

by

the

absolute

value

of

the

simulated

gene

expression

level

-LRB-

Gardner

et

al

-RRB-

All

the

algorithms

were

run

on

all

the

data

sets

using

default

parameters

-LRB-

Supplementary

Table

-RRB-

Banjo

was

not

run

on

the

gene

data

set

as

it

was

crashing

owing

to

memory

limitations

whereas

NIR

needed

an

excessively

long

computation

time

Results

from

the

simulations

are

described

in

Table

II

PPV

stands

for

positive

predictive

value

-LRB-

or

accuracy

-RRB-

defined

as

TP

-LRB-

TP

FP

-RRB-

and

Sensitivity

-LRB-

Se

-RRB-

is

TP

-LRB-

TP

FN

-RRB-

where

TP

true

positive

FP

false

positive

and

FN

false

negative

The

label

Random

refers

to

the

expected

performance

of

an

algorithm

that

selects

a

pair

of

genes

randomly

and

infers

an

edge

between

them

For

example

for

a

fully

connected

network

the

random

algorithm

would

have

a

accuracy

for

all

the

levels

of

sensitivity

-LRB-

as

any

pair

of

genes

is

connected

in

the

real

network

-RRB-

Some

algorithms

infer

the

network

just

as

an

undirected

graph

and

others

as

a

directed

and/or

signed

graph

Thus

in

order

to

facilitate

comparison

among

algorithms

we

computed

PPV

and

Se

by

first

transforming

the

real

-LRB-

signed

directed

graph

-RRB-

and

the

inferred

networks

-LRB-

when

directed

and/or

signed

-RRB-

in

an

undirected

graph

-LRB-

labeled

u

in

the

table

-RRB-

If

the

algorithm

infers

a

directed

graph

and/or

a

signed

directed

graph

we

also

compared

PPV

and

Se

in

this

case

-LRB-

labeled

d

and

s

respectively

in

the

table

-RRB-

When

computing

PPV

and

Se

we

did

not

include

self-feedback

loops

-LRB-

diagonal

elements

of

the

adjacency

matrix

-RRB-

as

all

the

simulated

networks

have

self-feedback

loops

and

this

could

be

an

advantage

for

some

algorithms

as

NIR

that

always

recovers

a

network

with

self-feedbacks

We

observe

that

for

the

global

perturbation

data

set

all

the

algorithms

but

Banjo

-LRB-

Bayesian

networks

-RRB-

fail

as

their

performance

is

comparable

with

the

random

algorithm

-LRB-

hence

the

importance

of

reporting

always

the

random

performance

-RRB-

Banjo

performance

is

poor

when

only

experiments

are

available

and

reaches

a

very

good

accuracy

for

experiments

-LRB-

independently

of

the

number

of

genes

-RRB-

albeit

with

a

very

low

sensitivity

-LRB-

only

few

edges

are

found

-RRB-

The

performance

of

all

the

algorithms

but

Banjo

improves

dramatically

for

the

local

data

set

In

this

case

both

ARACNE

and

NIR

perform

very

well

whereas

Banjo

performance

is

still

random

for

experiments

whereas

it

reaches

a

very

good

accuracy

but

poor

sensitivity

for

the

experiments

set

Clustering

is

better

than

random

but

is

clearly

not

a

good

method

to

infer

gene

networks

Performance

is

again

random

for

the

time-series

dynamic

data

set

In

this

case

we

run

ARACNE

as

well

although

the

time

points

can

not

be

assumed

independent

from

each

other

Banjo

has

been

shown

to

work

EMBO

and

Nature

Publishing

Group

How

to

infer

gene

networks

from

expression

profiles

M

Bansal

et

al

Table

II

Results

of

the

application

of

network

inference

algorithms

on

the

simulated

data

set

ARACNE

PPV

Global

-LRB-

steady-state

-RRB-

u

Se

u

PPV

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

BANJO

Se

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

PPV

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

NIR

Se

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

Clustering

PPV

u

Se

u

Random

PPV

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

u

u

d

s

u

d

s

u

u

d

s

u

d

s

u

d

s

u

d

s

u

d

s

u

u

u

u

u

u

u

u

u

u

u

u

u

u

u

u

u

Local

-LRB-

steady-state

-RRB-

u

u

u

u

u

u

u

u

u

u

u

u

u

u

u

u

u

u

u

Dynamic

-LRB-

time-series

-RRB-

u

u

u

u

u

u

u

u

u

u

u

u

u

u

u

u

u

u

u

u

Abbreviations

PPV

positive

predicted

value

Se

sensitivity

In

bold

are

the

algorithms

that

perform

significantly

better

than

random

using

as

a

random

model

a

Binomial

distribution

on

dynamic

data

but

needs

a

very

high

number

of

experiments

-LRB-

time

points

-RRB-

as

compared

with

the

number

of

genes

-LRB-

Yu

et

al

-RRB-

In

the

local

data

set

most

of

the

algorithms

perform

better

than

random

Banjo

recovers

only

a

few

of

the

hundreds

of

real

interactions

-LRB-

low

sensitivity

and

high

accuracy

-RRB-

ARACNE

recovers

about

half

of

the

real

connections

in

the

network

-LRB-

good

sensitivity

and

good

accuracy

-RRB-

NIR

instead

recovers

EMBO

and

Nature

Publishing

Group

almost

all

of

the

real

interactions

-LRB-

high

sensitivity

and

high

accuracy

-RRB-

clustering

recovers

a

fifth

of

the

real

connections

but

with

low

accuracy

and

most

of

the

connections

recovered

by

clustering

are

found

by

ARACNE

as

well

It

is

interesting

to

ask

what

is

the

average

overlap

between

the

inferred

networks

for

the

different

algorithms

Supplementary

Figure

shows

an

example

of

a

network

recovered

by

each

of

the

four

algorithms

Molecular

Systems

Biology

How

to

infer

gene

networks

from

expression

profiles

M

Bansal

et

al

We

intersected

the

networks

inferred

by

all

the

algorithms

and

found

that

for

the

network

local

dataset

on

average

about

of

the

edges

overlap

across

all

the

four

algorithms

whereas

about

overlap

for

the

network

local

dataset

-LRB-

Supplementary

Table

-RRB-

This

is

due

to

the

fact

that

Banjo

for

the

network

dataset

recovers

very

few

connections

compared

with

the

other

algorithms

thus

the

intersection

has

very

few

edges

in

this

case

excluding

Banjo

when

computing

the

intersection

rescues

the

intersection

overlap

to

about

-LRB-

Supplementary

Table

-RRB-

We

then

checked

the

PPV

and

Se

by

considering

only

the

edges

that

were

found

by

all

the

algorithms

if

any

as

reported

in

Supplementary

Table

As

expected

Se

decreased

but

the

PPV

improved

only

slightly

compared

with

that

of

each

algorithm

considered

separately

In

addition

for

large

networks

-LRB-

genes

-RRB-

the

intersection

among

the

networks

exists

only

of

the

time

whereas

for

small

networks

-LRB-

genes

-RRB-

it

exists

of

the

time

The

performance

of

each

algorithm

can

be

further

improved

by

modifying

their

parameters

-LRB-

refer

to

Supplementary

Table

-RRB-

The

ARACNE

parameter

DPI

-LRB-

threshold

for

data

processing

inequality

-RRB-

varies

from

to

and

controls

the

pruning

of

the

triplets

in

the

network

-LRB-

no

pruning

and

each

triplet

is

broken

at

the

weakest

edge

-RRB-

We

found

a

DPI

to

be

a

conservative

threshold

giving

a

good

compromise

between

Se

and

PPV

-LRB-

Supplementary

Table

-RRB-

Another

parameter

is

the

threshold

on

the

MI

Increasing

these

parameters

allows

one

to

improve

the

PPV

at

the

cost

of

reducing

the

Se

The

MI

level

can

also

be

chosen

automatically

by

ARACNE

which

does

a

fairly

good

job

so

we

suggest

not

to

set

the

MI

threshold

manually

Banjo

gives

the

user

a

variety

of

choices

for

its

parameters

the

running

time

can

be

increased

but

it

does

not

seem

to

affect

the

results

much

-LRB-

Supplementary

Table

-RRB-

so

we

suggest

s

for

networks

and

s

for

networks

the

Proposer

and

Searcher

modules

which

scan

and

score

the

network

topology

to

find

the

best

directed

acyclic

graph

can

be

chosen

from

a

set

of

four

different

algorithms

on

our

data

sets

the

different

choices

did

not

affect

the

results

considerably

The

NIR

algorithm

performance

can

be

affected

by

varying

the

parameter

k

which

defines

the

average

in-degree

per

node

-LRB-

i.e.

each

gene

can

be

regulated

at

most

by

other

k

genes

-RRB-

The

lower

the

k

the

higher

the

PPV

at

the

cost

of

reducing

the

Se

The

performance

of

NIR

on

the

simulated

data

sets

is

biased

as

NIR

is

based

on

linear

ODEs

which

are

also

used

to

generate

the

fake

simulated

gene

expression

data

however

as

noise

is

added

to

the

simulated

data

the

reported

performance

should

not

be

too

far

from

the

true

one

NIR

seems

to

perform

better

than

the

other

algorithms

but

it

also

requires

more

information

that

is

the

genes

that

have

been

directly

perturbed

in

each

microarray

experiment

-LRB-

for

example

which

gene

has

been

knocked-out

etc.

-RRB-

Application

to

experimental

data

In

order

to

test

different

softwares

we

also

collected

the

experimental

data

sets

described

in

Table

III

and

included

in

the

Supplementary

material

The

microarray

data

to

be

given

as

input

to

the

algorithms

need

no

specific

pre-processing

just

normalisation

and

selection

of

the

genes

that

have

responded

significantly

to

the

perturbation

experiments

using

standard

techniques

We

chose

three

different

organisms

and

data

sets

of

different

sizes

two

large

data

sets

-LRB-

A

and

B

-RRB-

two

medium

data

sets

-LRB-

C

and

D

-RRB-

and

two

small

data

sets

-LRB-

E

and

F

-RRB-

We

tested

each

algorithm

on

the

largest

number

of

data

sets

possible

In

each

case

we

used

default

parameters

Banjo

could

not

run

on

data

set

A

and

B

owing

to

the

large

size

of

the

dataset

NIR

can

be

applied

only

to

dataset

E

as

it

requires

steady-state

experiments

and

knowledge

of

the

perturbed

gene

in

each

experiment

Hierarchical

clustering

was

applied

to

all

data

sets

Table

IV

summarises

the

results

but

it

should

not

be

used

for

comparative

purposes

between

the

different

algorithms

owing

to

the

limited

number

of

data

and

to

the

imperfect

knowledge

of

the

real

network

In

silico

analysis

performed

in

the

previous

section

is

better

suited

for

this

task

ARACNE

performs

well

on

datasets

A

and

C

whereas

the

other

algorithms

are

not

significantly

better

than

random

ARACNE

is

not

better

than

random

for

data

set

B

and

better

than

random

for

dataset

D

whereas

Banjo

is

considerably

better

than

random

for

data

set

D

albeit

with

very

low

sensitivity

in

line

with

the

in

silico

results

For

the

same

dataset

D

clustering

performs

better

than

random

with

a

lower

accuracy

than

Banjo

but

a

better

sensitivity

The

overall

low

performance

on

dataset

B

as

compared

with

the

other

data

sets

is

probably

due

either

to

higher

noise

levels

in

this

dataset

or

to

imperfect

knowledge

of

the

real

network

-LRB-

transcription

network

in

yeast

-RRB-

Data

sets

E

and

F

are

not

very

informative

since

the

real

networks

are

small

and

densely

connected

and

therefore

the

random

algorithm

performs

very

well

In

any

case

only

NIR

performs

significantly

better

than

random

for

dataset

E

and

only

clustering

does

significantly

better

than

random

for

dataset

F.

Table

III

Experimental

data

sets

used

as

examples

ID

A

B

C

D

E

F

Cell/organism

HumanBcells

S.

cerevisiae

HumanBcells

S.

cerevisiae

E.

coli

E.

coli

Type

S

S

S

S

S

T

Samples

254 300 254 300

Genes

Reference

-LRB-

Basso

et

al

-RRB-

-LRB-

Hughes

et

al

-RRB-

-LRB-

Basso

et

al

-RRB-

-LRB-

Hughes

et

al

-RRB-

-LRB-

Gardner

et

al

-RRB-

gardnerlab.bu.edu

True

network

Twenty-six

Myc

targets

-LRB-

Basso

et

al

-RRB-

Eight

hundred

and

forty-four

TF

gene

interactions

-LRB-

Lee

et

al

-RRB-

Myc

targets

non-targets

-LRB-

Basso

et

al

-RRB-

Subset

of

TF

gene

interactions

-LRB-

Lee

et

al

-RRB-

Nine-gene

network

-LRB-

Gardner

et

al

-RRB-

Nine-gene

network

-LRB-

Gardner

et

al

-RRB-

Abbreviations

S

steady-state

T

time-series

Molecular

Systems

Biology

EMBO

and

Nature

Publishing

Group

How

to

infer

gene

networks

from

expression

profiles

M

Bansal

et

al

Table

IV

Results

of

the

application

of

network

inference

algorithms

on

the

experiment

data

sets

Data

sets

PPV

A

B

C

D

E

F

u

u

u

u

u

u

ARACNE

Se

u

u

u

u

u

u

BANJO

PPV

u

u

u

d

s

u

d

s

Se

u

u

u

d

s

u

d

s

PPV

u

d

s

NIR

Se

u

d

s

PPV

Clustering

se

u

u

u

u

u

u

Random

PPV

u

u

u

u

u

d

s

u

d

s

u

u

u

u

u

u

Abbreviations

PPV

positive

predicted

value

Se

sensitivity

In

bold

are

the

algorithms

that

perform

significantly

better

than

random

-LRB-

-RRB-

using

as

a

random

model

a

Binomial

distribution

Discussion

and

conclusions

In

silico

analysis

gives

reliable

guidelines

on

algorithms

performance

in

line

with

the

results

obtained

on

real

data

sets

ARACNE

performs

well

for

steady-state

data

and

can

be

applied

also

when

few

experiments

are

available

as

compared

with

the

number

of

genes

but

it

is

not

suited

for

the

analysis

of

short

time-series

data

This

is

to

be

expected

owing

to

the

requirement

of

statistically

independent

experiments

Banjo

is

very

accurate

but

with

a

very

low

sensitivity

on

steady-state

data

when

more

than

different

perturbation

experiments

are

available

independently

of

the

number

of

genes

whereas

it

fails

for

time-series

data

Banjo

-LRB-

and

Bayesian

networks

in

general

-RRB-

is

a

probabilistic

algorithm

requiring

the

estimation

of

probability

density

distributions

a

task

that

requires

large

number

of

data

points

NIR

works

very

well

for

steady-state

data

also

when

few

experiments

are

available

but

requires

knowledge

on

the

genes

that

have

been

perturbed

directly

in

each

perturbation

experiment

NIR

is

a

deterministic

algorithm

and

if

the

noise

on

the

data

is

small

it

does

not

require

large

data

sets

as

it

is

based

on

linear

regression

Clustering

although

not

a

reverse-engineering

algorithm

can

give

some

information

on

the

network

structure

when

a

large

number

of

experiments

is

available

as

confirmed

by

both

in

silico

and

experimental

analysis

albeit

with

a

much

lower

accuracy

than

the

other

reverse-engineering

algorithms

The

different

reverse-engineering

methods

considered

here

infer

networks

that

overlap

for

about

of

the

edges

for

small

networks

and

even

less

for

larger

networks

Interestingly

if

all

algorithms

agree

on

an

interaction

between

two

genes

-LRB-

an

edge

in

the

network

-RRB-

this

interaction

is

not

more

likely

to

be

true

than

the

ones

inferred

by

a

single

algorithm

Therefore

it

is

not

a

good

idea

to

trust

an

interaction

more

just

because

more

than

one

reverse-engineering

algorithm

finds

it

Indeed

the

different

mathematical

models

used

by

the

reverseengineering

algorithms

have

complementary

abilities

for

example

ARACNE

may

correctly

infer

an

interaction

that

NIR

does

not

find

and

vice

versa

hence

in

the

intersection

of

the

two

algorithms

both

edges

will

disappear

causing

a

drop

in

sensitivity

without

any

gain

in

accuracy

-LRB-

PPV

-RRB-

Taking

the

union

of

the

interactions

found

by

all

the

algorithms

is

not

a

good

option

as

this

will

cause

a

large

drop

in

accuracy

This

observation

leads

us

to

conclude

that

it

should

be

possible

to

EMBO

and

Nature

Publishing

Group

develop

better

approaches

by

subdividing

the

microarray

dataset

in

smaller

subsets

and

then

by

applying

the

most

appropriate

algorithm

to

each

microarray

subset

How

to

choose

the

subsets

and

how

to

decide

which

is

the

best

algorithm

to

use

are

still

open

questions

A

general

consideration

is

that

the

nature

of

experiments

performed

in

order

to

perturb

the

cells

and

measure

gene

expression

profiles

can

make

the

task

of

inference

easier

-LRB-

or

harder

-RRB-

From

our

results

local

perturbation

experiments

that

is

single

gene

overexpression

or

knockdown

seem

to

be

much

more

informative

than

global

perturbation

experiments

that

is

overexpressing

tens

of

genes

simultaneously

or

submitting

the

cells

to

a

strong

shock

Time-series

data

allow

one

to

investigate

the

dynamics

of

activation

-LRB-

inhibition

-RRB-

of

genes

in

response

to

a

specific

perturbation

These

data

can

be

useful

to

infer

the

direct

molecular

mediators

-LRB-

targets

-RRB-

of

the

perturbation

in

the

cell

-LRB-

Bansal

et

al

-RRB-

but

trying

to

infer

the

network

among

all

the

genes

responding

to

the

perturbation

from

time-series

data

does

not

yield

acceptable

results

Reverse-engineering

algorithms

using

time-series

data

need

to

be

improved

One

of

the

reasons

for

the

poor

performance

of

time-series

reverseengineering

algorithms

is

the

smaller

amount

of

information

contained

in

time-series

data

when

compared

with

steadystate

data

Time-series

are

usually

measured

following

the

perturbation

of

one

or

few

genes

in

the

cell

whereas

steadystate

data

are

obtained

by

performing

multiple

perturbations

to

the

cell

thus

eliciting

a

richer

response

One

way

to

improve

performance

in

the

time-series

case

is

to

perform

more

than

one

time-series

experiment

by

perturbing

different

genes

each

time

but

this

may

be

expensive

another

solution

could

be

to

perform

only

one

perturbation

experiment

but

with

a

richer

dynamics

for

example

the

perturbed

gene

should

be

overexpressed

and

then

allowed

to

return

to

its

endogenous

level

while

measuring

gene

expression

changes

of

the

other

genes

Richer

dynamics

in

the

perturbation

will

yield

richer

dynamics

in

the

response

and

thus

more

informative

data

Gene

network

inference

algorithms

are

becoming

accurate

enough

to

be

practically

useful

at

least

when

steady-state

gene

expression

data

are

available

but

efforts

must

be

directed

in

assessing

algorithm

performances

In

a

few

years

gene

network

inference

will

become

as

common

as

clustering

for

microarray

data

analysis

These

algorithms

will

become

more

Molecular

Systems

Biology

How

to

infer

gene

networks

from

expression

profiles

M

Bansal

et

al

integrative

by

exploiting

in

addition

to

expression

profiles

protein

protein

interaction

data

sequence

data

protein

modification

data

metabolic

data

and

more

in

the

inference

process

-LRB-

Workman

et

al

-RRB-

Supplementary

information

Supplementary

information

is

available

at

the

Molecular

Systems

Biology

website

-LRB-

-RRB-

References

Amato

R

Ciaramella

A

Deniskina

N

Del

Mondo

C

di

Bernardo

D

Donalek

C

Longo

G

Mangano

G

Miele

G

Raiconi

G

Staiano

A

Tagliaferri

R

-LRB-

-RRB-

A

multi-step

approach

to

time

series

analysis

and

gene

expression

clustering

Bioinformatics

Ambesi

A

di

Bernardo

D

-LRB-

-RRB-

Computational

biology

and

drug

discovery

From

single-target

to

network

drugs

Curr

Bioinform

Bansal

M

Della

Gatta

G

di

Bernardo

D

-LRB-

-RRB-

Inference

of

gene

regulatory

networks

and

compound

mode

of

action

from

time

course

gene

expression

profiles

Bioinformatics

Basso

K

Margolin

AA

Stolovitzky

G

Klein

U

Dalla-Favera

R

Califano

A

-LRB-

-RRB-

Reverse

engineering

of

regulatory

networks

in

human

B

cells

Nat

Genet

Beer

MA

Tavazoie

S

-LRB-

-RRB-

Predicting

gene

expression

from

sequence

Cell

Bonneau

R

Reiss

D

Shannon

P

Facciotti

M

Hood

L

Baliga

N

Thorsson

V

-LRB-

-RRB-

The

inferelator

an

algorithm

for

learning

parsimonious

regulatory

networks

from

systems-biology

data

sets

de

novo

Genome

Biol

Brazhnik

P

de

la

Fuente

A

Mendes

P

-LRB-

-RRB-

Artificial

gene

networks

for

objective

comparison

of

analysis

algorithms

Bioinformatics

-LRB-

Suppl

-RRB-

Breitkreutz

B

Stark

C

Tyers

M

-LRB-

-RRB-

Osprey

a

network

visualization

system

Genome

Biol

Butte

A

Kohane

I

-LRB-

-RRB-

Mutual

information

relevance

networks

functional

genomic

clustering

using

pairwise

entropy

measurements

Pac

Symp

Biocomput

de

Jong

H

-LRB-

-RRB-

Modeling

and

simulation

of

genetic

regulatory

systems

a

literature

review

J

Comp

Biol

D'haeseleer

P

Wen

X

Fuhrman

S

Somogyi

R

-LRB-

-RRB-

Linear

modeling

of

mrna

expression

levels

during

cns

development

and

injury

Pac

Symp

Biocomput

di

Bernardo

D

Thompson

M

Gardner

T

Chobot

S

Eastwood

E

Wojtovich

A

Elliott

S

Schaus

S

Collins

J

-LRB-

-RRB-

Chemogenomic

profiling

on

a

genome-wide

scale

using

reverse-engineered

gene

networks

Nat

Biotechnol

Eisen

M

Spellman

P

Brown

P

Botstein

D

-LRB-

-RRB-

Cluster

analysis

and

display

of

genome-wide

expression

patterns

Proc

Natl

Acad

Sci

USA

Faith

J

Gardner

T

-LRB-

-RRB-

Reverse-engineering

transcription

control

networks

Phys

Life

Rev

Foat

B

Morozov

A

Bussemaker

HJ

-LRB-

-RRB-

Statistical

mechanical

modeling

of

genome-wide

transcription

factor

occupancy

data

by

matrixreduce

Bioinformatics

Friedman

N

Elidan

G

-LRB-

-RRB-

Bayesian

network

software

libB

Available

from

Gardner

T

di

Bernardo

D

Lorenz

D

Collins

J

-LRB-

-RRB-

Inferring

genetic

networks

and

identifying

compound

mode

of

action

via

expression

profiling

Science

Hughes

TR

Marton

MJ

Jones

AR

Roberts

CJ

Stoughton

R

Armour

CD

Bennett

HA

Coffey

E

Dai

H

He

YD

Kidd

MJ

King

AM

Meyer

MR

Slade

D

Lum

PY

Stepaniants

SB

Shoemaker

DD

Gachotte

D

Chakraburtty

K

Simon

J

Bard

M

Friend

SH

-LRB-

-RRB-

Functional

discovery

via

a

compendium

of

expression

profiles

Cell

Lee

TI

Rinaldi

NJ

Robert

F

Odom

DT

Bar-Joseph

Z

Gerber

GK

Hannett

NM

Harbison

CT

Thompson

CM

Simon

I

Zeitlinger

J

Jennings

EG

Murray

HL

Gordon

DB

Ren

B

Wyrick

JJ

Tagne

J-B

Volkert

TL

Fraenkel

E

Gifford

DK

Young

RA

-LRB-

-RRB-

Transcriptional

regulatory

networks

in

Saccharomyces

cerevisae

Science

Margolin

A

Nemenman

I

Basso

K

Wiggins

C

Stolovitzky

G

Della

Favera

R

Califano

A

-LRB-

-RRB-

Aracne

an

algorithm

for

the

reconstruction

of

gene

regulatory

networks

in

a

mammalian

cellular

context

BMC

Bioinformatics

-LRB-

arXiv

q

bio.MN

-RRB-

Murphy

K

-LRB-

-RRB-

The

bayes

net

toolbox

for

matlab

Comput

Sci

Stat

Pe'er

D

Nachman

I

Linial

M

Friedman

N

-LRB-

-RRB-

Using

bayesian

networks

to

analyze

expression

data

J

Comput

Biol

Prakash

A

Tompa

M

-LRB-

-RRB-

Discovery

of

regulatory

elements

in

vertebrates

through

comparative

genomics

Nat

Biotechnol

Shannon

P

Markiel

A

Ozier

O

Baliga

N

Wang

J

Ramage

D

Amin

D

Schwikowski

B

Ideker

T

-LRB-

-RRB-

Cytoscape

a

software

environment

for

integrated

models

of

biomolecular

interaction

networks

Genome

Res

Steuer

R

Kurths

J

Daub

CO

Weise

J

Selbig

J

-LRB-

-RRB-

The

mutual

information

detecting

and

evaluating

dependencies

between

variables

Bioinformatics

-LRB-

Suppl

-RRB-

Evaluation

Stud

Tadesse

M

Vannucci

M

Lio

P

-LRB-

-RRB-

Identification

of

DNA

regulatory

motifs

using

bayesian

variable

selection

Bioinformatics

Tegner

J

Yeung

MK

Hasty

J

Collins

JJ

-LRB-

-RRB-

Reverse

engineering

gene

networks

integrating

genetic

perturbations

with

dynamical

modeling

Proc

Natl

Acad

Sci

USA

van

Someren

E

Vaes

B

Steegenga

W

Sijbers

A

Dechering

K

Reinders

M

-LRB-

-RRB-

Least

absolute

regression

network

analysis

of

the

murine

osteoblast

differentiation

network

Bioinformatics

Workman

C

Mak

H

McCuine

S

Tagne

J

Agarwal

M

Ozier

O

Begley

T

Samson

L

T

I

-LRB-

-RRB-

A

systems

approach

to

mapping

DNA

damage

response

pathways

Science

Yu

J

Smith

VA

Wang

PP

Hartemink

AJ

Jarvis

ED

-LRB-

-RRB-

Advances

to

bayesian

network

inference

for

generating

causal

networks

from

observational

biological

data

Bioinformatics

Molecular

Systems

Biology

EMBO

and

Nature

Publishing

Group