Reverse

Engineering

of

Biological

Complexity

Marie

E.

Csete

et

al.

Science

-LRB-

-RRB-

DOI

science

The

following

resources

related

to

this

article

are

available

online

at

-LRB-

this

information

is

current

as

of

May

-RRB-

Updated

information

and

services

including

high-resolution

figures

can

be

found

in

the

online

version

of

this

article

at

Supporting

Online

Material

can

be

found

at

This

article

cites

articles

of

which

can

be

accessed

for

free

This

article

has

been

cited

by

article

-LRB-

s

-RRB-

on

the

ISI

Web

of

Science

This

article

has

been

cited

by

articles

hosted

by

HighWire

Press

see

This

article

appears

in

the

following

subject

collections

Cell

Biology

Information

about

obtaining

reprints

of

this

article

or

about

obtaining

permission

to

reproduce

this

article

in

whole

or

in

part

can

be

found

at

Science

-LRB-

print

ISSN

online

ISSN

-RRB-

is

published

weekly

except

the

last

week

in

December

by

the

American

Association

for

the

Advancement

of

Science

New

York

Avenue

NW

Washington

DC

Copyright

by

the

American

Association

for

the

Advancement

of

Science

all

rights

reserved

The

title

Science

is

a

registered

trademark

of

AAAS

Downloaded

from

on

May

SYSTEMS

BIOLOGY

THE

GENOME

LEGOME

AND

BEYOND

readable

model

definition

that

enables

models

to

be

exchanged

between

software

tools

Systems

Biology

Workbench

-LRB-

SBW

-RRB-

is

built

on

SBML

and

provides

a

framework

of

modular

open-source

software

for

systems

biology

research

Both

SBML

and

SBW

are

collective

efforts

of

a

number

of

research

institutions

sharing

the

same

vision

-LRB-

-RRB-

How

does

the

idea

of

systems

biology

impact

pharmaceutical

industries

and

medical

practice

The

most

feasible

application

of

systems

biology

research

is

to

create

a

detailed

model

of

cell

regulation

focused

on

particular

signal-transduction

cascades

and

molecules

to

provide

system-level

insights

into

mechanismbased

drug

discovery

-LRB-

-RRB-

Such

models

may

help

to

identify

feedback

mechanisms

that

offset

the

effects

of

drugs

and

predict

systemic

side

effects

It

may

even

be

possible

to

use

a

multiple

drug

system

to

guide

the

state

of

malfunctioning

cells

to

the

desired

state

with

minimal

side

effects

Such

a

systemic

response

can

not

be

rationally

predicted

without

a

model

of

intracellular

biochemical

and

genetic

interactions

It

is

not

inconceivable

that

the

U.S.

Food

and

Drug

Administration

may

one

day

mandate

simulation-based

screening

of

therapeutic

agents

just

as

plans

for

all

highrise

building

are

required

to

undergo

structural

dynamics

analysis

to

confirm

earthquake

resistance

Although

systems

biology

is

in

its

infancy

its

potential

benefits

are

enormous

in

both

scientific

and

practical

terms

A

transition

is

occurring

in

biology

from

the

molecular

level

to

the

system

level

that

promises

to

revolutionize

our

understanding

of

complex

biological

regulatory

systems

and

to

provide

major

new

opportunities

for

practical

application

of

such

knowledge

N.

Weiner

Cybernetics

or

Control

and

Communication

in

the

Animal

and

the

Machine

-LRB-

MIT

Press

Cambridge

MA

-RRB-

H.

Kitano

Foundations

of

Systems

Biology

-LRB-

MIT

Press

Cambridge

MA

-RRB-

E.

H.

Davidson

et

al.

Science

-LRB-

-RRB-

Examples

of

such

databases

are

Signal

Transduction

Knowledge

Environment

-LRB-

STKE

-RRB-

KEGG

-LRB-

-RRB-

EcoCyc

-LRB-

http

ecocyc.org

-RRB-

M.

Eisen

et

al.

Proc

Natl.

Acad

Sci

U.S.A.

-LRB-

-RRB-

S.

Chu

et

al.

Science

-LRB-

-RRB-

S.

Onami

et

al.

in

Foundations

of

Systems

Biology

H.

Kitano

Ed

-LRB-

MIT

Press

Cambridge

MA

-RRB-

pp.

S.

Imoto

et

al.

Pacific

Symposium

on

Biocomputing

-LRB-

World

Scientific

Singapore

-RRB-

pp.

C.

Yoo

et

al.

Pacific

Symposium

on

Biocomputing

-LRB-

World

Scientific

Singapore

-RRB-

pp.

T.

Ideker

et

al.

Pacific

Symposium

on

Biocomputing

-LRB-

World

Scientific

Singapore

-RRB-

pp.

J.

Edwards

et

al.

Nature

Biotechnol

-LRB-

-RRB-

M.

Borisuk

et

al.

J.

Theor

Biol

-LRB-

-RRB-

K.

Chen

et

al.

Mol

Biol

Cell

-LRB-

-RRB-

M.

E.

Csete

J.

C.

Doyle

Science

-LRB-

-RRB-

N.

Barkai

et

al.

Nature

-LRB-

-RRB-

U.

Alon

et

al.

Nature

-LRB-

-RRB-

T.-M

Yi

et

al.

Proc

Natl.

Acad

Sci

U.S.A.

-LRB-

-RRB-

V.

Dassaw

et

al.

Nature

-LRB-

-RRB-

G.

Weng

et

al.

Science

-LRB-

-RRB-

Alliance

for

Cellular

Signaling

-LRB-

-RRB-

M.

Burns

et

al.

Science

-LRB-

-RRB-

R.

Anderson

et

al.

Nucleic

Acids

Res

-LRB-

-RRB-

P.

Simpson

et

al.

Proc

Natl.

Acad

Sci

U.S.A.

-LRB-

-RRB-

P.

Gilles

et

al.

Nature

Biotechnol

-LRB-

-RRB-

Additional

information

can

be

obtained

at

http

or

J.

Gibbs

Science

-LRB-

-RRB-

C.

Sander

Science

-LRB-

-RRB-

D.

Noble

Science

-LRB-

-RRB-

I

thank

J.

Doyle

M.

Simon

and

members

of

ERATO

Kitano

project

for

fruitful

discussions

Supported

by

the

ERATO

and

BIRD

program

of

the

Japan

Science

and

Technology

Corporation

and

the

Rice

Genome

and

Simulation

Project

of

the

Ministry

of

Agriculture

Japan

References

and

Notes

REVIEW

Reverse

Engineering

of

Biological

Complexity

Marie

E.

and

John

C.

Advanced

technologies

and

biology

have

extremely

different

physical

implementations

but

they

are

far

more

alike

in

systems-level

organization

than

is

widely

appreciated

Convergent

evolution

in

both

domains

produces

modular

architectures

that

are

composed

of

elaborate

hierarchies

of

protocols

and

layers

of

feedback

regulation

are

driven

by

demand

for

robustness

to

uncertain

environments

and

use

often

imprecise

components

This

complexity

may

be

largely

hidden

in

idealized

laboratory

settings

and

in

normal

operation

becoming

conspicuous

only

when

contributing

to

rare

cascading

failures

These

puzzling

and

paradoxical

features

are

neither

accidental

nor

artificial

but

derive

from

a

deep

and

necessary

interplay

between

complexity

and

robustness

modularity

feedback

and

fragility

This

review

describes

insights

from

engineering

theory

and

practice

that

can

shed

some

light

on

biological

complexity

The

theory

and

practice

of

complex

engineering

systems

have

progressed

so

radically

that

they

often

embody

Arthur

C.

Clarke

's

dictum

Any

sufficiently

advanced

technology

is

indistinguishable

from

magic

Systems-level

Departments

of

Anesthesiology

and

Cell

and

Developmental

Biology

University

of

Michigan

Medical

School

Ann

Arbor

MI

USA

and

Dynamical

Systems

Electrical

Engineering

and

Bioengineering

California

Institute

of

Technology

Pasadena

CA

USA

To

whom

correspondence

should

be

addressed

Email

doyle@cds.caltech.edu

approaches

in

biology

have

a

long

history

-LRB-

-RRB-

but

are

just

now

receiving

renewed

mainstream

attention

-LRB-

-RRB-

whereas

systems-level

design

has

consistently

been

at

the

core

of

modern

engineering

motivating

its

most

sophisticated

theories

in

controls

information

and

computation

The

hidden

nature

of

complexity

-LRB-

magic

-RRB-

and

discipline

fragmentation

within

engineering

have

been

barriers

to

a

dialog

with

biology

A

key

starting

point

in

developing

a

conceptual

and

theoretical

bridge

to

biology

is

robustness

the

preservation

of

particular

characteristics

despite

uncertain

ty

in

components

or

the

environment

-LRB-

-RRB-

Biologists

and

biophysicists

new

to

studying

complex

networks

often

express

surprise

at

a

biological

network

's

apparent

robustness

-LRB-

-RRB-

They

find

that

perfect

adaptation

and

homeostatic

regulation

are

robust

properties

of

networks

-LRB-

-RRB-

despite

exploratory

mechanisms

that

can

seem

gratuitously

uncertain

-LRB-

-RRB-

Some

even

conclude

that

these

mechanisms

and

their

resulting

features

seem

absent

in

engineering

-LRB-

-RRB-

However

ironically

it

is

in

the

nature

of

their

robustness

and

complexity

that

biology

and

advanced

engineering

are

most

alike

-LRB-

-RRB-

Good

design

in

both

cases

-LRB-

e.g.

cells

and

bodies

cars

and

airplanes

-RRB-

means

that

users

are

largely

unaware

of

hidden

complexities

except

through

system

failures

Furthermore

the

robustness

and

fragility

features

of

complex

systems

are

both

shared

and

necessary

Although

the

need

for

universal

principles

of

complexity

and

corresponding

mathematical

tools

is

widely

recognized

-LRB-

-RRB-

sharp

differences

arise

as

to

what

is

fundamental

about

complexity

and

what

mathematics

is

needed

-LRB-

-RRB-

This

article

sketches

one

possible

view

using

experience

and

theoretical

insights

from

engineering

complexity

that

are

relevant

to

biology

MARCH

VOL

SCIENCE

Downloaded

from

on

May

SYSTEMS

BIOLOGY

THE

GENOME

LEGOME

AND

BEYOND

We

hope

to

dispel

some

common

misconceptions

and

to

renew

a

dialog

between

engineering

theorists

and

their

biologist

and

clinician

colleagues

speed

and

thus

-LRB-

from

an

allometric

scaling

viewpoint

-RRB-

would

exhibit

no

deleterious

laboratory

phenotype

Redundancy

does

not

explain

this

finding

-LRB-

-RRB-

Rather

the

mutant

has

lost

control

systems

and

robustness

required

for

real

flight

outside

the

lab

Allometric

scaling

emphasizes

the

essential

similarities

between

these

variants

and

a

toy

scale

model

-LRB-

and

a

fruit

fly

-RRB-

whereas

our

interest

is

their

huge

differences

in

complexity

Similarly

minimal

cellular

life

requires

a

few

hundred

genes

-LRB-

-RRB-

yet

even

Escherichia

coli

have

genes

less

than

of

which

have

been

classified

as

essential

-LRB-

-RRB-

The

likely

reason

for

this

excess

complexity

is

also

the

presence

of

complex

regulatory

networks

for

robustness

In

technology

as

well

as

in

organisms

such

robustness

tradeoffs

drive

the

evolution

of

spiraling

complexity

As

an

example

of

spiraling

complexity

to

battle

fragility

consider

our

use

of

as

a

nutrient

-LRB-

electron

acceptor

-RRB-

which

obligates

us

to

use

complex

feedback

control

mechanisms

to

ensure

both

sufficient

and

protection

from

toxicity

Distributed

multiscaled

networks

maintain

precise

internal

local

concentrations

throughout

the

body

both

acutely

and

chronically

Dependency

on

such

regulation

makes

its

failure

lethal

of

course

but

an

additional

fragility

created

by

this

exquisitely

controlled

environment

is

that

it

creates

an

attractive

ecosystem

for

parasites

whose

systems

can

thus

be

more

streamlined

Host

robustness

to

parasites

then

requires

a

separate

complex

immune

control

system

In

the

developing

immune

system

T

cells

are

educated

to

recognize

self

from

nonself

in

the

thymus

They

are

then

selected

to

proceed

on

to

the

periphery

-LRB-

positive

selection

-RRB-

or

if

they

are

inaccurate

sensors

they

self-destruct

A

fragility

of

this

exceedingly

complex

immune

system

is

autoimmune

disease

an

example

being

primary

biliary

cirrhosis

-LRB-

PBC

-RRB-

in

which

self-reactive

lymphocytes

slip

through

the

immunity

education

program

Autoimmune

injury

to

bile

ducts

causes

toxic

bile

acids

to

accumulate

Injured

hepatocytes

fail

to

clear

hormones

contributing

to

increased

pressures

in

the

liver

circulation

and

even

to

rupture

of

connected

venous

systems

Pressure-induced

distortion

of

blood

vessels

in

the

spleen

traps

platelets

and

damaged

hepatocytes

undersynthesize

bloodclotting

proteins

both

exaggerating

blood

loss

after

trauma

Unable

to

capitalize

on

the

usual

homeostatic

feedback

interactions

with

the

liver

virtually

every

organ

including

brain

and

kidney

can

fail

all

initiated

by

superficially

minuscule

autoimmune

damage

Medical

interventions

for

PBC

-LRB-

i.e.

drugs

and

transplantation

-RRB-

are

further

control

systems

adding

to

spiraling

complexity

robustness

and

fragility

Genetic

variation

in

the

enzyme

family

leads

to

considerable

interindividual

variation

in

drug

handling

and

side

effects

Imbalance

of

the

network

can

lead

to

accumulation

of

toxins

including

carcinogens

Polypharmacy

a

common

necessity

results

in

even

more

unpredictable

interactions

because

drugs

modulate

activity

Liver

transplantation

is

now

standard

therapy

for

PBC

Immunosuppression

must

be

sufficient

to

quash

the

immune

mechanism

that

recognizes

a

foreign

invader

but

too

much

immunosuppression

allows

infection

and

tumors

to

go

unchecked

Hence

the

fragilities

of

transplantation

are

infection

and

tumors

Complexity

Optimality

and

Convergent

Evolution

The

differences

between

biology

and

technology

-LRB-

and

between

organisms

-RRB-

are

obvious

particularly

at

the

molecular

and

device

level

Nevertheless

convergent

evolution

a

wellestablished

concept

in

both

engineering

and

evolutionary

biology

yields

remarkable

similarities

at

higher

levels

of

organization

Recently

engineering

systems

have

begun

to

have

almost

biological

levels

of

complexity

For

example

a

Boeing

is

fully

fly-bywire

with

different

subsystem

modules

organized

via

elaborate

protocols

into

complex

control

systems

and

networks

including

roughly

computers

that

can

automate

all

vehicle

functions

In

terms

of

cost

and

complexity

the

is

essentially

a

vast

control

system

and

computer

network

that

just

happens

to

fly

The

consequence

of

good

design

is

that

its

regulatory

complexity

is

hidden

from

passengers

-LRB-

except

when

they

use

entertainment

systems

-RRB-

The

internal

activity

level

is

staggering

however

-LRB-

e.g.

the

data

rate

recorded

on

the

internal

state

during

final

production

testing

is

nearly

equivalent

to

one

human

genome

every

minute

-RRB-

Commercial

aircraft

are

not

the

only

systems

undergoing

such

explosions

in

complexity

as

a

result

of

advanced

controls

and

embedded

networking

virtually

all

technologies

are

evolving

similarly

-LRB-

-RRB-

We

claim

that

this

technological

evolution

of

complexity

is

convergent

with

that

of

biology

A

striking

example

of

convergent

evolution

is

Fig.

comparing

cruise

speed

to

mass

M

over

orders

of

magnitude

from

the

and

to

fruit

flies

-LRB-

-RRB-

The

essential

assumption

in

allometric

scaling

theory

is

that

convergent

evolution

leads

to

nearly

optimal

systems

with

similar

gross

characteristics

It

follows

that

simple

arguments

based

on

optimal

design

can

explain

functional

relations

between

variables

across

many

scales

-LRB-

-RRB-

Here

a

wellknown

elementary

argument

-LRB-

-RRB-

shows

good

correspondence

with

the

data

and

yields

explanations

for

deviations

The

popular

allometric

scaling

theories

-LRB-

connecting

say

efficiency

and

geometry

-RRB-

are

appealing

They

are

simple

accessible

suggestive

evidence

confirming

convergent

evolution

and

engineering

optimality

Such

theories

are

largely

irrelevant

to

complexity

directly

but

an

understanding

of

them

leads

to

what

is

relevant

The

scaling

theory

described

by

Fig.

does

not

distinguish

between

flight

in

the

atmosphere

and

in

a

laboratory

wind

tunnel

In

the

latter

context

a

much

simpler

mutant

with

nearly

all

of

its

aeronome

knocked

out

would

have

roughly

the

same

lift

mass

and

cruise

Modularity

and

Protocols

What

emerges

from

these

examples

is

that

spiraling

complexity

feedback

regulation

robustness

fragility

and

cascading

failures

are

heavily

intertwined

as

is

well

known

to

biologists

and

engineers

alike

Equally

important

and

well

known

is

the

obvious

role

that

modularity

plays

at

every

level

from

base

pairs

and

amino

acids

to

genes

and

proteins

from

organelles

and

membranes

to

pathways

and

networks

and

finally

to

organs

and

organ

axes

-LRB-

-RRB-

and

in

every

complex

process

from

development

-LRB-

-RRB-

to

evolution

-LRB-

-RRB-

Although

their

meaning

varies

modules

generally

are

components

parts

or

subsystems

of

a

larger

system

that

contain

some

or

all

of

the

following

features

-LRB-

i

-RRB-

identifiable

interfaces

-LRB-

usually

involving

protocols

-RRB-

to

other

modules

-LRB-

ii

-RRB-

can

be

modified

and

evolved

somewhat

independently

-LRB-

iii

-RRB-

facilitate

simplified

or

abstract

modeling

-LRB-

iv

-RRB-

maintain

some

identity

when

isolated

or

rearranged

yet

-LRB-

v

-RRB-

derive

additional

identity

from

the

rest

of

the

system

The

organization

and

design

of

advanced

technologies

suggest

universal

principles

relevant

to

biology

linking

modularity

with

the

robust

yet

fragile

nature

of

complex

systems

Truly

universal

principles

should

manifest

Fig.

Optimal

cruise

speed

at

sea

level

versus

mass

-LRB-

log-log

-RRB-

for

organisms

and

airplanes

Line

is

theoretical

prediction

-LRB-

-RRB-

with

V

cM

and

-LRB-

-RRB-

Shorter

wings

for

speed

and

maneuverability

-LRB-

triangles

-RRB-

yield

higher

cruise

speeds

than

those

optimized

for

soaring

-LRB-

diamonds

-RRB-

Most

systems

-LRB-

circles

-RRB-

are

compromises

Humans

are

not

selected

for

powering

flight

and

are

far

from

optimal

-LRB-

square

-RRB-

Data

and

theory

are

from

-LRB-

-RRB-

SCIENCE

VOL

MARCH

Downloaded

from

on

May

SYSTEMS

BIOLOGY

THE

GENOME

LEGOME

AND

BEYOND

themselves

in

at

least

limited

ways

in

scalemodel

-LRB-

toy

-RRB-

systems

just

as

allometric

scaling

does

Consider

the

ubiquitous

Lego

toy

system

-LRB-

-RRB-

The

signature

feature

of

Lego

is

the

patented

snap

connection

for

easy

but

stable

assembly

of

components

The

snap

is

the

basic

Lego

protocol

and

Lego

bricks

are

its

basic

modules

We

claim

that

protocols

are

far

more

important

to

biologic

complexity

than

are

modules

They

are

complementary

and

intertwined

but

are

important

to

distinguish

In

everyday

usage

protocols

are

rules

designed

to

manage

relationships

and

processes

smoothly

and

effectively

If

modules

are

ingredients

parts

components

subsystems

and

players

then

protocols

describe

the

corresponding

recipes

architectures

rules

interfaces

etiquettes

and

codes

of

conduct

-LRB-

-RRB-

Protocols

here

are

rules

that

prescribe

allowed

interfaces

between

modules

permitting

system

functions

that

could

not

be

achieved

by

isolated

modules

Protocols

also

facilitate

the

addition

of

new

protocols

and

organization

into

collections

of

mutually

supportive

protocol

suites

Like

modules

they

simplify

modeling

and

abstraction

and

as

such

may

often

be

largely

in

the

eye

of

the

beholder

A

good

protocol

is

one

that

supplies

both

robustness

and

evolvability

Lego

exhibits

multilayer

robustness

from

components

and

toys

to

the

product

line

Lego

bricks

and

toys

are

reusable

and

robust

to

trauma

and

the

snap

is

versatile

permitting

endless

varieties

of

toys

from

an

array

of

components

This

makes

both

a

given

Lego

collection

and

the

entire

toy

system

evolvable

to

changes

in

what

one

chooses

to

build

to

the

addition

of

new

Lego-compatible

parts

and

to

novel

toy

designs

Evolution

here

is

simply

robustness

to

-LRB-

possibly

large

-RRB-

changes

on

long

time

scales

The

low

cost

of

modules

and

the

popularity

of

the

system

confer

other

forms

of

robustness

and

evolvability

lost

parts

are

easily

replaced

and

enthusiasts

constantly

design

new

modules

and

toys

The

Lego

protocol

also

creates

fragilities

at

every

level

Superficially

minuscule

damage

to

the

snap

at

a

key

interface

may

cause

an

entire

toy

to

fail

yet

noninterfacing

parts

of

bricks

may

be

heavily

damaged

with

minimal

impact

The

success

of

Lego

means

that

any

new

snap

even

a

superior

one

would

not

be

easily

adopted

Selection

pressures

thus

preserve

a

protocol

in

two

ways

Protocols

facilitate

evolution

and

are

difficult

to

change

It

is

instructive

to

compare

the

robustness

properties

-LRB-

basic

performance

ability

to

withstand

trauma

versatility

of

allowed

interconnections

reusability

of

modules

cost

of

parts

and

labor

and

evolvability

-RRB-

of

the

standard

Lego

snap

protocol

-LRB-

called

the

wild

type

WT

-RRB-

with

those

of

other

hypothetical

protocols

-LRB-

denoted

Smooth

Glue

and

Mold

-RRB-

Smooth

bricks

without

snaps

have

unconstrained

interconnections

but

the

results

are

much

less

robust

to

trauma

severely

limiting

the

range

of

toys

Glue

in

addition

to

the

WT

snap

increases

ability

to

withstand

trauma

but

sharply

decreases

component

reusability

Injection

Molding

entire

toys

goes

even

further

Thus

each

mutation

offers

advantages

with

both

different

robustness

and

fragility

but

uniformly

improves

on

WT

's

overall

robustness

WT

is

fine-tuned

for

robustness

We

claim

that

this

kind

of

optimality

and

robustness

is

most

important

to

biological

complexity

As

systems

become

more

complex

protocols

facilitate

the

layering

of

additional

protocols

particularly

involving

feedback

and

signaling

Suppose

we

want

to

make

a

Lego

structure

incrementally

more

useful

and

versatile

by

evolving

it

to

be

-LRB-

i

-RRB-

mobile

then

-LRB-

ii

-RRB-

motorized

then

-LRB-

iii

-RRB-

able

to

avoid

collisions

in

a

maze

of

obstacles

The

first

increment

is

easy

to

achieve

with

Lego

protocol

compatible

axles

and

wheels

Motorizing

toys

involves

a

second

increment

in

complexity

requiring

protocols

for

motor

and

battery

interconnection

as

well

as

a

separate

protocol

for

gears

All

can

be

integrated

into

a

motorized

protocol

suite

to

make

modular

subassemblies

of

batteries

motors

gears

axles

and

wheels

These

are

available

inexpensive

additions

The

third

increment

increases

cost

and

complexity

by

orders

of

magnitude

requiring

layers

of

protocols

and

modules

for

sensing

actuation

and

feedback

controls

plus

subsidiary

but

essential

ones

for

communications

and

computing

-LRB-

-RRB-

All

are

available

but

it

is

here

that

we

begin

to

see

the

true

complexity

of

advanced

technologies

Unfortunately

we

also

start

to

lose

the

easily

described

intuitive

story

of

the

basic

protocols

Minimal

descriptions

of

advanced

Lego

features

enabling

sensing

and

feedback

control

literally

fill

books

but

the

protocols

also

facilitate

the

building

of

elaborate

robust

toys

precisely

because

this

complexity

is

largely

hidden

from

users

This

is

consistent

with

the

claim

that

biological

complexity

too

is

dominated

not

by

minimal

function

but

by

the

protocols

and

regulatory

feedback

loops

that

provide

robustness

and

evolvability

This

added

complexity

also

creates

new

and

often

extreme

fragilities

Removing

a

toy

's

control

system

might

cause

reversion

to

mere

mobility

but

a

small

change

in

an

otherwise

intact

control

system

could

cause

wild

catastrophic

behavior

For

example

a

small

software

bug

might

easily

lead

to

collision

seeking

a

fragility

absent

in

simpler

toys

Similarly

large

multicellular

organisms

are

unaffected

by

the

death

of

a

single

cell

but

failure

of

one

cell

's

control

system

can

lead

to

fatal

autoimmune

diseases

or

cancer

The

snap

protocol

is

concretely

instantiated

only

in

Lego

modules

but

it

is

also

easy

to

identify

the

protocol

itself

as

a

useful

and

informative

abstraction

The

snap

protocol

is

more

fundamental

to

Lego

than

are

any

individual

modules

Similarly

we

have

no

trouble

distinguishing

the

many

higher

level

protocols

that

organize

sensing

and

feedback

from

the

hardware

modules

themselves

In

biology

the

identification

of

protocols

is

easiest

when

shared

by

many

different

modules

as

in

Lego

Thus

abstractions

such

as

gene

regulation

-LRB-

-RRB-

covalent

modification

membrane

potentials

metabolic

and

signal

transduction

pathways

action

potentials

and

even

transcription-translation

the

cell

cycle

and

DNA

replication

could

all

be

reasonably

described

as

protocols

-LRB-

-RRB-

with

their

attendant

modular

implementations

in

various

activators

and

repressors

kinases

and

phosphatases

ion

channels

receptors

heterotrimeric

guanine

nucleotide

binding

proteins

-LRB-

G

proteins

-RRB-

and

so

on

The

cardiovascular

system

has

protocols

for

gas

and

nutrient

exchange

and

transport

implemented

in

heart

lung

vascular

networks

and

blood

modules

The

immune

system

involves

elaborate

protocols

for

complement

and

cell-mediated

activation

implemented

in

modules

such

as

T

cells

natural

killer

cells

major

histocompatibility

complex

molecules

and

antibodies

Metazoan

development

has

highly

conserved

protocols

-LRB-

-RRB-

Appropriate

temporal

and

spatial

expression

during

development

-LRB-

-RRB-

is

regulated

by

enormous

numbers

of

feedback

strategies

-LRB-

-RRB-

These

and

many

other

protocols

facilitate

robust

development

and

function

in

ways

similar

to

Lego

protocols

and

they

produce

similar

fragilities

-LRB-

-RRB-

Thinking

in

terms

of

protocols

in

addition

to

genes

organisms

and

populations

as

foci

of

natural

selection

may

be

a

useful

abstraction

for

understanding

the

evolution

of

complexity

-LRB-

-RRB-

Good

protocols

allow

new

functions

to

be

built

from

existing

components

and

allow

new

components

to

be

added

or

to

evolve

from

existing

ones

powerfully

enhancing

both

engineering

and

evolutionary

tinkering

Protocols

enable

modularity

and

robustness

but

are

in

turn

sources

of

fragility

Successful

protocols

become

highly

conserved

because

they

both

facilitate

evolution

and

are

difficult

to

change

Lego

has

a

perfectly

complete

legome

of

all

parts

including

full

structure

and

function

A

similar

compendium

is

far

from

available

for

even

simple

organisms

Yet

understanding

a

collision-avoiding

software-intensive

feedback-regulated

Lego

robot

would

require

extensive

reverse

engineering

of

additional

layers

of

protocols

and

modules

beyond

the

legome

That

the

legome

would

not

be

sufficient

is

no

surprise

but

for

reverse

engineering

such

details

may

not

be

entirely

necessary

-LRB-

see

below

-RRB-

Imagine

that

such

a

Lego

robot

was

a

prototype

for

a

single

toy

that

dispensed

entirely

with

the

Lego

modules

in

favor

of

custom

implementation

Similar

to

Mold

this

toy

could

easily

have

much

more

robustness

to

trauma

be

faster

and

navigate

more

complex

obstacles

but

at

the

MARCH

VOL

SCIENCE

Downloaded

from

on

May

SYSTEMS

BIOLOGY

THE

GENOME

LEGOME

AND

BEYOND

expense

of

limited

part

reuse

The

modules

and

lower

level

protocols

most

of

the

legome

would

be

completely

different

yet

we

might

claim

that

the

essence

of

the

toy

and

what

the

prototype

aimed

to

capture

remained

That

essence

involves

the

protocols

that

organized

the

sensors

actuators

and

feedback

control

system

that

enables

the

obstacle

avoidance

and

contributes

almost

the

entire

cost

and

complexity

These

too

are

governed

by

protocols

but

also

by

entirely

new

laws

certainties

Defining

fragility

as

logS

note

that

F

iff

S

iff

logS

-LRB-

-RRB-

F

makes

logS

amplifying

d

and

uncertainty

in

A

and

F

makes

logS

-LRB-

-RRB-

Unfortunately

this

story

is

incomplete

and

even

misleading

without

dynamics

The

simplest

possibility

is

for

A

and

C

to

be

first-order

differential

equations

C

A

x

a

k

gu

k

y

u

d

r

a

x

-LRB-

-RRB-

vary

widely

with

minimal

effect

-LRB-

-RRB-

Integral

feedback

is

used

ubiquitously

in

engineering

-LRB-

-RRB-

and

is

likely

to

be

ubiquitous

in

biology

as

well

to

achieve

everything

from

homeostatic

regulation

to

perfect

adaptation

and

preliminary

investigations

confirm

this

impression

-LRB-

-RRB-

One

reason

is

that

integral

feedback

is

both

sufficient

and

necessary

for

perfect

and

robust

steady-state

tracking

Intuitively

necessity

follows

from

the

fact

that

in

steady

state

a

y

d

must

perfectly

cancel

any

constant

-LRB-

step

in

-RRB-

d

whereas

the

input

u

to

A

can

not

depend

on

this

d

because

y

does

not

Thus

A

-LRB-

or

C

-RRB-

must

contain

an

internal

model

of

the

dynamics

of

d

which

for

step

changes

is

a

pure

integrator

-LRB-

-RRB-

which

produces

unbounded

outputs

to

constant

inputs

Thus

open-loop

hypersensitivity

is

necessary

for

closed-loop

robustness

and

Fig.

is

not

an

accident

Fragility

enters

in

the

transient

response

When

g

is

increased

the

response

is

faster

but

oscillatory

-LRB-

Fig.

A

and

B

-RRB-

Figure

plots

fragility

logS

-LRB-

-RRB-

versus

where

Y

-LRB-

-RRB-

and

D

-LRB-

-RRB-

are

Fourier

transforms

of

y

and

d

and

logS

log

Y

D

-LRB-

-RRB-

Elementary

Feedback

Concepts

Protocols

are

the

most

important

aspect

of

modularity

and

the

most

complex

and

critical

protocols

are

for

feedback

control

and

the

sensing

computing

communication

and

actuation

that

implement

it

Feedback

control

is

both

a

powerful

and

dangerous

strategy

for

creating

robustness

to

external

disturbances

and

internal

component

variations

Properly

balanced

it

delivers

such

a

huge

benefit

that

both

engineers

and

evolution

capitalize

extensively

on

feedback

to

build

and

support

complex

systems

-LRB-

-RRB-

Detailed

elaboration

of

the

nature

of

regulatory

feedback

underlying

complexity

is

beyond

the

scope

of

this

article

but

an

elementary

toy

model

illustrates

the

necessity

of

feedback

to

the

function

of

complex

systems

as

well

as

feedback

's

conservation

of

fragility

law

This

is

arguably

the

most

critical

and

rigorously

established

robustness

tradeoff

in

complex

systems

In

most

technologies

as

well

as

in

biochemistry

it

is

relatively

easy

to

build

either

uncertain

high-gain

components

or

precise

low-gain

ones

but

the

precise

high-gain

systems

essential

to

both

biology

and

technology

are

impossible

or

prohibitively

expensive

to

make

unless

a

feedback

strategy

like

that

in

Fig.

is

used

The

simplest

case

is

steadystate

gain

where

after

some

transient

r

and

d

are

held

constant

and

y

too

approaches

a

constant

y

Rr

Sd

-LRB-

-RRB-

where

R

and

S

are

responses

of

y

to

s

and

d

respectively

Solving

y

d

ACy

Ar

gives

y

ASr

Sd

S

C

r

Sd

C

is

a

low-pass

filter

with

internal

state

x

and

and

A

is

a

pure

parameters

integrator

with

state

a

and

gain

g

-LRB-

-RRB-

This

type

of

control

is

called

integral

feedback

The

parameters

g

and

might

typically

be

functions

of

underlying

physical

quantities

such

as

temperature

binding

affinities

concentrations

etc.

and

thus

might

vary

widely

The

response

y

-LRB-

t

-RRB-

to

steps

in

r

and

d

are

shown

in

Fig.

over

two

orders

of

magnitude

in

g

and

This

simple

protocol

of

integral

feedback

produces

extremely

robust

external

behavior

even

from

wildly

varying

components

-LRB-

the

blue

solid

versus

red

dashed

lines

in

Fig.

-RRB-

It

is

easily

shown

that

this

system

is

stable

iff

and

and

converges

to

the

steady

state

y

-LRB-

-RRB-

r

independently

of

arbitrarily

large

variations

in

gain

g

and

disturbance

d

-LRB-

-RRB-

If

y

-LRB-

-RRB-

r

is

a

high-gain

amplifier

as

well

-LRB-

-RRB-

The

individual

values

of

g

and

influence

the

rate

of

convergence

to

steady

state

but

only

the

ratio

determines

its

value

Thus

robust

high

steady-state

gain

can

be

achieved

with

uncertain

and

small

parameters

with

the

right

feedback

protocol

Figure

shows

that

variations

in

both

g

and

of

orders

of

magnitude

have

modest

impact

and

only

on

early

transient

behavior

The

protocol

here

is

the

structure

of

the

equations

including

the

integral

feedback

and

the

signs

of

the

parameters

Modules

are

the

implementations

of

the

actuator

and

controller

As

with

Lego

the

protocol

must

be

finetuned

-LRB-

because

rewiring

components

or

flipping

signs

typically

creates

exponentially

growing

instabilities

-RRB-

but

this

allows

the

modules

to

F

AC

-LRB-

-RRB-

F

where

F

is

the

feedback

gain

Ideally

perfect

control

would

have

S

because

that

gives

y

r/C

-LRB-

R

-RRB-

completely

independent

of

arbitrary

variations

in

A

and

d

If

A

and

then

F

S

and

y

r/C

Then

R

amplifies

r

and

is

perfectly

robust

to

external

disturbance

d

and

to

variations

in

A.

Choosing

C

small

and

precise

with

A

sufficiently

large

and

even

sloppy

is

one

effective

efficient

and

robust

way

to

make

y

a

high-gain

function

of

r.

S

measures

the

deviation

from

perfect

control

and

feedback

can

attenuate

or

greatly

amplify

the

effects

of

unS

Fig.

Minimal

feedback

system

with

actuator

A

and

controller/sensor

C

The

goal

is

for

response

y

to

amplify

reference

r

independent

of

external

disturbance

d

and

variations

in

A

The

signals

u

and

a

are

the

input

and

output

of

the

actuator

A

and

x

is

the

output

of

C.

Fig.

Closed

-LRB-

blue

-RRB-

versus

open

red

-RRB-

loop

response

y

-LRB-

t

-RRB-

to

step

-LRB-

changes

at

t

in

-LRB-

A

-RRB-

d

-LRB-

t

-RRB-

-LRB-

r

-RRB-

and

-LRB-

B

-RRB-

r

-LRB-

t

-RRB-

-LRB-

d

-RRB-

for

g

and

and

Note

the

extreme

divergence

-LRB-

-RRB-

versus

convergence

-LRB-

-RRB-

as

t

-LRB-

C

-RRB-

is

a

zoom

of

-LRB-

A

-RRB-

with

added

for

each

value

of

g.

and

-LRB-

D

-RRB-

logS

-LRB-

-RRB-

versus

for

responses

in

-LRB-

A

-RRB-

The

peaks

in

logS

-LRB-

-RRB-

correspond

to

the

oscillations

in

-LRB-

A

-RRB-

and

-LRB-

B

-RRB-

Note

the

equal

areas

under

the

curves

for

logS

-LRB-

-RRB-

SCIENCE

VOL

MARCH

Downloaded

from

on

May

SYSTEMS

BIOLOGY

THE

GENOME

LEGOME

AND

BEYOND

For

increasing

g

low-frequency

robustness

-LRB-

logS

-LRB-

-RRB-

-RRB-

is

improved

but

at

the

expense

of

increased

fragility

-LRB-

logS

-LRB-

-RRB-

-RRB-

at

higher

frequencies

-LRB-

-RRB-

Indeed

it

can

be

proven

that

for

all

g

d

plex

structures

dominate

control

system

design

Presumably

such

tradeoffs

dominate

and

constrain

evolution

and

biology

as

well

L.

von

Bertalanffy

Modern

Theories

of

Development

An

Introduction

to

Theoretical

Biology

-LRB-

Oxford

Univ.

Press

New

York

-RRB-

M.

A.

Savageau

Biochemical

Systems

Theory

-LRB-

Addison-Wesley

Reading

MA

-RRB-

Genetics

-LRB-

-RRB-

C.

V.

Rao

A.

P.

Arkin

Annu

Rev.

Biomed

Eng

-LRB-

-RRB-

M.

Dickinson

et

al.

Science

-LRB-

-RRB-

D.

Endy

R.

Brent

Nature

-LRB-

-RRB-

L.

H.

Hartwell

J.

J.

Hopfield

S.

Leibler

A.

W.

Murray

Nature

-LRB-

-RRB-

D.

A.

Lauffenburger

Proc

Natl.

Acad

Sci

U.S.A.

-LRB-

-RRB-

M.

Freeman

Nature

-LRB-

-RRB-

D.

Noble

Science

-LRB-

-RRB-

E.

H.

Davidson

et

al.

Science

-LRB-

-RRB-

The

supplements

at

Science

Online

-LRB-

org/cgi/content

-RRB-

and

at

doyle

contain

additional

details

biology

and

engineering

examples

proofs

of

math

results

and

references

Every

topic

considered

here

has

a

long

history

in

biology

-LRB-

and

engineering

-RRB-

renewed

mainstream

interest

in

the

postgenomics

era

and

intense

current

research

activity

Space

limits

force

a

focus

on

recent

reviews

that

have

better

historical

coverage

than

is

possible

here

Alliance

for

Cellular

Signaling

-LRB-

org

-RRB-

Complex

systems

may

be

nonlinear

heterogeneous

large-scale

hierarchical

adaptive

etc.

but

these

are

subsidiary

issues

here

Fragility

here

has

the

specific

meaning

of

large

and

deleterious

changes

in

particular

system

properties

attributable

to

possibly

small

but

specific

variations

in

the

environment

or

in

components

Robustness

is

often

used

more

broadly

but

here

it

means

roughly

the

inverse

of

fragility

Their

subtle

interplay

is

the

point

of

this

article

U.

Alon

M.

G.

Surette

N.

Barkai

S.

Leibler

Nature

-LRB-

-RRB-

G.

von

Dassow

et

al.

Nature

-LRB-

-RRB-

J.

Gerhart

M.

Kirschner

Proc

Natl.

Acad

Sci

U.S.A.

-LRB-

-RRB-

Such

mechanisms

include

chemotaxis

immune

response

microtubule

formation

angiogenesis

neurogenesis

and

evolution

itself

See

-LRB-

-RRB-

M.

Kirschner

J.

Gerhart

T.

Mitchison

Cell

-LRB-

-RRB-

M.

Chicurel

Nature

-LRB-

-RRB-

This

too

has

a

long

history

see

-LRB-

-RRB-

Special

issue

on

Complex

Systems

Science

-LRB-

April

-RRB-

J.

M.

Carlson

J.

Doyle

Proc

Natl.

Acad

Sci

U.S.A.

-LRB-

-RRB-

HowStuffWorks

Inc.

-LRB-

-RRB-

H.

Tennekes

The

Simple

Science

of

Flight

From

Insects

to

Jumbo

Jets

-LRB-

MIT

Press

Cambridge

MA

-RRB-

J.

H.

Brown

G.

B.

West

Eds.

Scaling

in

Biology

-LRB-

Oxford

Univ.

Press

New

York

-RRB-

J.

Whitfield

Nature

-LRB-

-RRB-

Lift

L

must

offset

total

weight

W

A

is

wing

area

is

air

density

and

CL

depends

on

geometry

-RRB-

Assuming

fixed

shapes

and

materials

-LRB-

and

eliminating

A

yields

V

CW

or

V

cM

where

and

c

are

constants

For

more

discussion

of

this

greatly

oversimplified

treatment

see

-LRB-

-RRB-

Redundancy

is

a

critical

strategy

for

robustness

to

component

failure

but

it

is

already

widely

appreciated

and

adds

modest

complexity

so

it

is

not

discussed

here

Mycoplasma

genitalium

has

genes

and

minimal

sets

for

a

free-living

cell

are

estimated

to

be

less

than

genes

-LRB-

-RRB-

Essential

here

refers

to

laboratory

viability

from

single

gene

knockouts

See

Profiling

of

E.

coli

Chromosome

-LRB-

PEC

-RRB-

-LRB-

-RRB-

The

LEGO

Group

-LRB-

-RRB-

D.

Baum

M.

Gasperi

R.

Hempel

L.

Villa

Extreme

Mindstorms

An

Advanced

Guide

to

Lego

Mindstorms

-LRB-

Apress

Berkeley

CA

-RRB-

Laws

in

society

are

protocols

unlike

laws

in

science

which

are

necessary

conditions

that

constrain

-LRB-

but

are

not

subject

to

-RRB-

design

or

selection

-LRB-

e.g.

energy

mass

References

and

Notes

Implications

for

Biology

and

Engineering

The

success

of

systems

biology

will

certainly

require

modeling

and

simulation

tools

from

engineering

-LRB-

-RRB-

where

experience

shows

that

brute-force

computational

approaches

are

hopeless

for

complex

systems

involving

protocols

and

feedback

Highly

fragile

features

require

highly

sophisticated

modeling

whereas

robust

features

often

have

adequate

models

that

are

greatly

simplified

requiring

a

middle-out

approach

-LRB-

-RRB-

For

example

if

Fig.

is

for

a

module

in

a

larger

system

the

steady-state

gain

y

-LRB-

-RRB-

r

depends

only

on

and

no

other

parameters

potentially

simplifying

experiments

and

modeling

If

transient

dynamics

or

component

failure

were

of

interest

more

details

would

be

needed

determined

more

by

the

rest

of

the

system

than

by

the

internal

components

Many

challenges

of

postgenomic

biology

are

converging

to

the

challenges

facing

engineers

building

complex

networks

of

networks

and

engineering

theory

and

practice

are

undergoing

a

revolution

as

radical

as

biology

's

The

simple

ideas

here

only

hint

at

the

possibilities

For

example

more

complex

control

protocols

than

Fig.

used

in

both

engineering

and

biology

can

ameliorate

-LRB-

though

not

eliminate

-RRB-

the

constraint

in

Eq

but

sophisticated

theory

is

needed

to

elucidate

the

issues

Realistic

models

of

biological

networks

will

not

be

simple

and

will

require

multiple

feedback

signals

nonlinear

component

dynamics

numerous

uncertain

parameters

stochastic

noise

models

-LRB-

-RRB-

parasitic

dynamics

and

other

uncertainty

models

Scaling

to

deal

with

large

networks

will

be

a

major

challenge

Fortunately

researchers

in

robust

control

theory

dynamical

systems

and

related

areas

have

been

vigorously

pursuing

mathematics

and

software

tools

to

address

exactly

these

issues

and

apply

them

to

complex

engineering

systems

-LRB-

-RRB-

Biological

applications

are

new

but

progress

so

far

is

encouraging

Experiments

modeling

and

simulation

and

theory

all

have

fragilities

but

they

are

complementary

and

through

the

right

protocols

they

have

the

potential

to

create

a

robust

closedloop

systems

biology

-LRB-

-RRB-

Biologists

frustrating

experience

with

theory

has

been

primarily

in

an

open-loop

mode

where

simple

and

attractive

ideas

can

be

wrong

but

receive

enormous

attention

Biology

is

the

only

science

where

feedback

control

and

protocols

play

a

dominant

role

so

it

should

not

be

surprising

that

there

would

be

popular

theories

coming

from

within

science

that

did

not

emphasize

these

issues

Biologists

and

engineers

now

have

enough

examples

of

complex

systems

that

they

can

close

the

loop

and

eliminate

specious

theories

-LRB-

-RRB-

We

should

compare

notes

logS

-LRB-

-RRB-

so

net

fragility

is

in

this

sense

a

conserved

quantity

Robustness

-LRB-

logS

-LRB-

-RRB-

-RRB-

is

paid

for

by

an

equal

fragility

-LRB-

logS

-LRB-

-RRB-

-RRB-

which

amplifies

d

and

uncertainty

in

A

-LRB-

-RRB-

This

quite

general

result

also

holds

for

arbitrary

parameters

control

systems

and

disturbances

-LRB-

-RRB-

Thus

there

are

always

nonconstant

-LRB-

e.g.

sinusoidal

-RRB-

d

-LRB-

t

-RRB-

that

would

be

amplified

in

y

-LRB-

t

-RRB-

Such

d

could

be

perfectly

rejected

too

but

only

by

adding

internal

models

as

complex

as

the

external

environment

that

generates

d.

Although

such

modeling

is

possible

only

for

simple

idealized

laboratory

environments

even

approximate

attempts

can

drive

an

extreme

complexity

spiral

in

real

systems

and

any

controller

is

still

subject

to

the

constraint

in

Eq

The

key

to

good

control

design

then

is

to

ensure

that

this

fragility

is

tolerable

and

occurs

where

uncertainties

are

relatively

small

Even

these

simple

toy

examples

show

the

robust

yet

fragile

features

of

complex

regulatory

networks

Their

outward

signatures

are

extremely

constant

regulated

variables

-LRB-

yet

occasional

cryptic

fluctuations

-RRB-

as

well

as

extraordinary

robustness

to

component

variations

-LRB-

yet

rare

but

catastrophic

cascading

failures

-RRB-

These

apparently

paradoxical

combinations

can

easily

be

a

source

of

confusion

to

experimentalists

clinicians

and

theoreticians

alike

-LRB-

-RRB-

but

are

intrinsic

features

of

highly

optimized

feedback

regulation

Because

net

robustness

and

fragility

are

constrained

quantities

they

must

be

manipulated

and

controlled

with

and

within

complex

networks

even

more

so

than

energy

and

materials

Figure

shows

how

extreme

open-loop

versus

closed-loop

behavior

can

be

and

thus

how

dangerous

loss

of

control

is

to

a

system

relying

on

it

The

tradeoff

in

Eq

shows

that

even

when

working

perfectly

net

fragility

is

constrained

and

thus

some

transient

amplification

is

unavoidable

The

necessity

of

integral

feedback

and

the

fragility

constraint

in

Eq

thus

describe

laws

not

protocols

perhaps

the

two

simplest

such

laws

from

control

theory

Controllers

that

are

more

complex

with

additional

dynamics

and

multiple

sensors

and

actuators

offer

more

refinement

in

performing

robustness-fragility

tradeoffs

Adding

to

regulatory

complexity

is

also

relatively

easy

in

an

evolutionary

sense

Faster

components

allow

for

faster

closed-loop

responses

All

are

used

in

both

biology

and

engineering

but

all

are

still

ultimately

subject

to

Eq

Control

engineers

must

contend

with

this

tradeoff

and

its

generalizations

to

more

com

MARCH

VOL

SCIENCE

Downloaded

from

on

May

SYSTEMS

BIOLOGY

THE

GENOME

LEGOME

AND

BEYOND

fragility

-RRB-

A

key

research

problem

is

distinguishing

among

laws

protocols

and

historical

accidents

The

central

dogma

can

be

thought

of

as

a

protocol

with

DNA

RNA

protein

RNA

polymerase

ribosome

etc.

as

modules

A

compelling

case

but

using

very

different

terminology

is

made

in

-LRB-

-RRB-

Steady

state

here

means

simply

that

all

variables

in

Fig.

-LRB-

r

d

y

A

C

etc.

-RRB-

approach

constants

which

can

be

solved

for

algebraically

iff

means

if

and

only

if

An

important

use

of

positive

feedback

is

to

deliberately

destabilize

equilibria

and

amplify

small

differences

to

create

switches

and

to

break

symmetries

and

homogeneities

This

can

create

patterns

that

are

then

maintained

using

negative

feedback

Positive

feedback

is

also

critical

to

autocatalysis

in

growth

and

metabolism

da/dt

a

gu

means

that

a

-LRB-

the

output

of

A

-RRB-

is

a

time

integral

of

gu

where

u

is

the

input

to

A.

Stability

is

easily

shown

using

standard

methods

of

linear

systems

Steady-state

values

can

be

found

-LRB-

in

a

stable

system

-RRB-

by

setting

all

time

derivatives

to

yielding

or

y

-LRB-

-RRB-

r.

Mechanisms

often

exist

that

allow

controller

parameters

-LRB-

e.g.

and

-RRB-

to

be

much

less

uncertain

than

g

and

d

It

is

often

even

easier

to

make

ratios

such

as

largely

invariant

to

variations

in

underlying

physical

quantities

affecting

the

individual

and

If

precise

gain

is

required

then

the

ratio

must

also

be

precise

The

national

power

grid

has

integral

control

at

the

power

plants

to

regulate

frequency

and

voltages

of

delivered

power

oil

refineries

have

such

control

loops

and

Internet

congestion

control

involves

a

form

implemented

as

part

of

TCP

-LRB-

transport

control

protocol

-RRB-

See

-LRB-

-RRB-

for

more

details

proofs

and

examples

T.

M.

Yi

Y.

Huang

M.

I.

Simon

J.

Doyle

Proc

Natl.

Acad

Sci

U.S.A.

-LRB-

-RRB-

A.

Levchenko

P.

A.

Iglesias

Biophys

J.

-LRB-

-RRB-

H.

El-Samad

J.

P.

Goff

M.

Khammash

J.

Theor

Biol

-LRB-

-RRB-

This

argument

can

be

made

rigorous

and

is

a

standard

elementary

result

in

control

theory

It

is

a

special

case

of

the

internal

model

principle

For

sufficiently

large

g

the

frequency

domain

peak

and

time

domain

transients

become

unacceptably

large

although

still

stable

One

interpretation

is

that

negative

feedback

is

always

balanced

by

an

equal

and

opposite

positive

feedback

Strictly

speaking

with

dynamics

this

is

not

well

defined

and

logS

-LRB-

-RRB-

gives

the

correct

generalization

Relatively

rare

circumstances

can

involve

an

inequality

-LRB-

-RRB-

This

is

worse

but

it

means

that

Eq

is

an

inequality

constraint

rather

than

a

pure

conservation

law

See

-LRB-

-RRB-

The

robust

yet

fragile

nature

of

highly

optimized

complex

regulatory

networks

can

be

mistakenly

attributed

to

various

kinds

of

bifurcations

and

orderdisorder

transitions

-LRB-

e.g.

phase

transitions

critical

phenomena

edge-of-chaos

pattern

formation

etc.

-RRB-

See

-LRB-

-RRB-

The

development

of

the

Boeing

alone

required

a

global

software

and

computing

infrastructure

with

roughly

workstations

terabytes

of

data

and

a

billion-dollar

price

tag

Systems

Biology

Workbench

-LRB-

sbw

-RRB-

D.

T.

Gillespie

J.

Chem

Phys

-LRB-

-RRB-

K.

Zhou

J.

Doyle

K.

Glover

Robust

and

Optimal

Control

-LRB-

Prentice-Hall

Upper

Saddle

River

NJ

-RRB-

E.

Sontag

Mathematical

Control

Theory

Deterministic

Finite

Dimensional

Systems

-LRB-

Springer

New

York

-RRB-

The

idea

of

closed-loop

systems

biology

is

a

purely

metaphorical

application

of

the

message

of

this

article

to

biology

research

itself

Biologists

already

iterate

-LRB-

loop

-RRB-

between

experiment

and

at

least

informal

modeling

and

simulation

This

process

might

be

enhanced

by

closing

the

loop

with

theory

and

infrastructure

modules

which

would

also

benefit

One

such

example

is

Internet

technology

which

is

rich

in

protocols

and

feedback

and

is

beginning

to

have

a

rich

theory

Even

though

it

is

poorly

understood

by

nonexperts

and

has

become

a

focus

of

many

specious

theories

details

and

enormous

data

sets

are

available

and

it

makes

an

attractive

example

to

compare

with

biological

networks

See

-LRB-

-RRB-

N.

Wiener

Cybernetics

-LRB-

Wiley

New

York

-RRB-

A.

R.

Mushegian

E.

V.

Koonin

Proc

Natl.

Acad

Sci

U.S.A.

-LRB-

-RRB-

H.

W.

Bode

Network

Analysis

and

Feedback

Amplifier

Design

-LRB-

Krieger

Melbourne

FL

-RRB-

REVIEW

A

Genomic

Regulatory

Network

for

Development

Eric

H.

Davidson

Jonathan

P.

Rast

Paola

Oliveri

Andrew

Ransick

Cristina

Calestani

Chiou-Hwa

Yuh

Takuya

Minokawa

Gabriele

Amore

Veronica

Hinman

Cesar

Arenas-Mena

Ochan

Otim

C.

Titus

Brown

Carolina

B.

Livi

Pei

Yun

Lee

Roger

Revilla

Alistair

G.

Rust

Zheng

jun

Pan

Maria

J.

Schilstra

Peter

J.

C.

Clarke

Maria

I.

Arnone

Lee

Rowen

R.

Andrew

Cameron

David

R.

McClay

Leroy

Hood

Hamid

Development

of

the

body

plan

is

controlled

by

large

networks

of

regulatory

genes

A

gene

regulatory

network

that

controls

the

specification

of

endoderm

and

mesoderm

in

the

sea

urchin

embryo

is

summarized

here

The

network

was

derived

from

large-scale

perturbation

analyses

in

combination

with

computational

methodologies

genomic

data

cis-regulatory

analysis

and

molecular

embryology

The

network

contains

over

genes

at

present

and

each

node

can

be

directly

verified

at

the

DNA

sequence

level

by

cis-regulatory

analysis

Its

architecture

reveals

specific

and

general

aspects

of

development

such

as

how

given

cells

generate

their

ordained

fates

in

the

embryo

and

why

the

process

moves

inexorably

forward

in

developmental

time

The

mechanism

causing

cats

to

beget

cats

and

fish

to

beget

fish

is

hardwired

in

the

genomic

DNA

because

the

species

specificity

of

the

body

plan

is

the

cardinal

heritable

property

But

despite

all

the

examples

of

how

individual

genes

affect

the

developmental

process

there

is

yet

no

case

where

the

lines

of

causality

can

be

mapped

from

the

genomic

sequence

to

a

major

process

of

bilaterian

development

One

reason

for

this

is

that

most

of

the

developmental

systems

that

have

been

intensively

studied

produce

adult

body

parts

such

as

the

third

instar

Drosophila

wing

disc

or

the

vertebrate

hindbrain

during

rhombomere

specification

or

the

heart

anlagen

of

flies

and

mice

-LRB-

-RRB-

These

systems

present

tough

challenges

because

they

go

through

successive

stages

of

pattern

formation

in

order

to

generate

complex

morphologies

and

their

development

is

initiated

from

states

that

are

already

complex

Furthermore

traditional

molecular

genetic

and

developmental

biological

approaches

have

focused

on

determining

the

functions

of

one

or

a

few

genes

at

a

time

an

approach

that

is

not

adequate

for

analysis

of

large

regulatory

control

systems

organized

as

networks

The

heart

of

such

networks

consists

of

genes

encoding

transcription

factors

and

the

cis-regulatory

elements

that

control

the

expression

of

those

genes

Each

of

these

cis-regulatory

elements

receives

multiple

inputs

from

other

genes

in

the

network

these

inputs

are

the

transcription

factors

for

which

the

element

contains

the

specific

target

site

sequences

The

functional

linkages

of

which

the

network

is

composed

are

those

between

the

outputs

of

regulatory

genes

and

the

sets

of

genomic

target

sites

to

which

their

products

bind

Therefore

these

linkages

can

be

tested

and

verified

by

cis-regulatory

analysis

This

means

identifying

the

control

elements

and

their

key

target

sites

and

experimentally

determining

their

functional

significance

The

view

taken

here

is

that

understanding

why

a

given

developmental

process

Division

of

Biology

California

Institute

of

Technology

Pasadena

CA

USA

and

Technology

Research

Centre

University

of

Hertfordshire

UK

Zoologica

Anton

Dohrn

Naples

Italy

for

Systems

Biology

Seattle

WA

USA

of

Zoology

Duke

University

Durham

NC

USA

To

whom

correspondence

should

be

addressed

Email

davidson@caltech.edu

Present

address

European

Bioinformatics

Institute

Wellcome

Trust

Genome

Campus

Hinxton

Cambridgeshire

UK

Present

address

Altera

European

Technology

Centre

Holmers

Farm

Way

High

Wycombe

Buckinghamshire

UK

SCIENCE

VOL

MARCH

Downloaded

from

on

May

O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

robustness
engineering
networks
systems biology