BIOINFORMATICS

Sequence

analysis

ORIGINAL

PAPER

Vol

no.

pages

doi

Multiple

structural

alignment

and

clustering

of

RNA

sequences

Elfar

Jakob

H.

and

Jan

Division

of

Genetics

and

Bioinformatics

IBHV

and

Center

for

Bioinformatics

and

of

Natural

Sciences

Faculty

of

Life

Sciences

University

of

Copenhagen

Frederiksberg

C

Denmark

Received

on

November

revised

on

January

accepted

on

February

Advance

Access

publication

February

Associate

Editor

Dmitrij

Frishman

ABSTRACT

Motivation

An

apparent

paradox

in

computational

RNA

structure

prediction

is

that

many

methods

in

advance

require

a

multiple

alignment

of

a

set

of

related

sequences

when

searching

for

a

common

structure

between

them

However

such

a

multiple

alignment

is

hard

to

obtain

even

for

few

sequences

with

low

sequence

similarity

without

simultaneously

folding

and

aligning

them

Furthermore

it

is

of

interest

to

conduct

a

multiple

alignment

of

RNA

sequence

candidates

found

from

searching

as

few

as

two

genomic

sequences

Results

Here

based

on

the

PMcomp

program

we

present

a

global

multiple

alignment

program

FOLDALIGNM

which

performs

especially

well

on

few

sequences

with

low

sequence

similarity

and

is

comparable

in

performance

with

state

of

the

art

programs

in

general

In

addition

it

can

cluster

sequences

based

on

sequence

and

structure

similarity

and

output

a

multiple

alignment

for

each

cluster

Furthermore

preliminary

results

with

local

datasets

indicate

that

the

program

is

useful

for

post

processing

FOLDALIGN

pairwise

scans

Availability

The

program

FOLDALIGNM

is

implemented

in

JAVA

and

is

along

with

some

accompanying

PERL

scripts

available

at

http

foldalign.ku.dk

Contact

gorodkin@genome.ku.dk

INTRODUCTION

Traditionally

most

RNA

molecules

have

been

regarded

as

merely

an

intermediate

on

the

DNA

mRNA

protein

pathway

The

main

exceptions

were

transfer

and

ribosomal

RNAs

both

involved

in

the

process

of

translation

However

since

the

late

the

view

on

RNA

has

changed

dramatically

with

the

discovery

of

a

steadily

increasing

number

of

untranslated

RNAs

These

untranslated

RNAs

are

of

great

importance

in

all

three

domains

of

life

-LRB-

Eddy

Huttenhoffer

et

al.

Mattick

-RRB-

Obviously

the

structure

is

of

importance

to

understand

the

function

of

the

RNA

and

whereas

single

sequence

folding

methods

-LRB-

Ding

Hofacker

Zucker

-RRB-

provide

some

information

about

the

structure

it

can

most

reliably

be

detected

from

a

set

of

related

sequences

-LRB-

if

available

-RRB-

-LRB-

Westhof

and

Michel

Westhof

et

al.

-RRB-

For

example

of

all

base

pairs

Gutell

and

co-workers

had

found

by

hand

To

whom

correspondence

should

be

addressed

-LRB-

Gutell

et

al.

-RRB-

are

indeed

present

in

the

structure

of

the

ribosome

-LRB-

Ban

et

al.

-RRB-

Whereas

an

expert

in

a

specific

RNA

by

hand

can

determine

the

base

pairs

through

tedious

study

of

compensating

changes

in

a

large

set

of

sequences

the

number

of

sequences

might

in

general

be

few

and

they

might

also

have

low

sequence

similarity

Hence

with

the

increasing

number

of

novel

RNAs

the

need

for

automated

structure

prediction

became

clear

and

a

number

of

methods

were

introduced

These

include

the

first

FOLDALIGN

version

-LRB-

Gorodkin

et

al.

-RRB-

as

well

as

methods

such

as

Dynalign

-LRB-

Uzilov

et

al.

-RRB-

Pfold

-LRB-

Knudsen

and

Hein

-RRB-

RNAalifold

-LRB-

Hofacker

et

al.

-RRB-

COVE

-LRB-

Eddy

and

Durbin

-RRB-

Stemloc

-LRB-

Holmes

-RRB-

and

RNAcast

-LRB-

Reeder

and

Giegerich

-RRB-

to

mention

a

few

examples

Until

recently

some

of

the

most

reliable

methods

to

computationally

find

non-coding

RNAs

-LRB-

ncRNAs

-RRB-

were

still

limited

to

simple

sequence

similarity

searches

with

known

ncRNAs

or

scanning

with

stochastic

context-free

grammars

-LRB-

SCFG

-RRB-

for

the

already

known

ncRNAs

This

is

implemented

in

programs

such

as

COVE

its

successor

INFERNAL

-LRB-

Eddy

-RRB-

and

RSEARCH

which

allows

for

searching

a

sequence

database

for

homologs

of

a

given

structured

RNA

molecule

-LRB-

Klein

and

Eddy

-RRB-

Speed

improvements

on

INFERNAL

have

been

implemented

in

RAVENNA

which

utilizes

hidden

Markov

models

-LRB-

HMMs

-RRB-

for

filtering

speeding

the

scanning

considerably

-LRB-

Weinberg

and

Ruzzo

b

-RRB-

However

these

methods

can

not

in

general

identify

new

families

of

ncRNAs

Two

recent

efficient

approaches

include

EvoFold

-LRB-

Pedersen

et

al.

-RRB-

and

RNAz

-LRB-

Washietl

et

al.

-RRB-

which

are

efficient

enough

to

be

applied

to

genome-wide

scans

and

can

detect

new

conserved

structural

RNAs

but

only

in

pre-aligned

sequences

Furthermore

EvoFold

and

RNAz

are

sensitive

to

alignment

errors

and

can

only

perform

a

scan

using

predefined

fixed

window

sizes

A

more

recent

very

promising

program

is

CMfinder

-LRB-

Yao

et

al.

-RRB-

which

unlike

EvoFold

and

RNAz

is

independent

of

alignments

and

is

local

Recent

implementation

of

the

Sankoff

algorithm

-LRB-

Sankoff

-RRB-

FOLDALIGN

provides

an

effective

approach

for

searching

two

sequences

for

common

local

structure

motifs

-LRB-

Havgaard

et

al.

-RRB-

FOLDALIGN

uses

two

types

of

parameters

to

calculate

scores

energy

and

sequence

similarity

FOLDALIGN

is

particularly

effective

in

low

sequence

similarity

ranges

and

was

recently

used

to

scan

human-mouse

pairs

unalignable

in

primary

sequence

for

conserved

structures

resulting

in

thousands

of

significant

candidates

-LRB-

Torarinsson

et

al.

-RRB-

The

Author

Published

by

Oxford

University

Press

All

rights

reserved

For

Permissions

please

email

journals.permissions@oxfordjournals.org

Multiple

structural

alignment

and

clustering

of

RNA

sequences

To

further

make

it

possible

to

collect

data

from

pairwise

scans

of

multiple

organisms

we

here

present

an

approach

which

can

conduct

clustering

followed

by

global

multiple

structural

alignments

of

the

candidates

found

from

the

pairwise

scans

The

approach

is

based

on

the

principles

of

PMcomp

and

PMmulti

-LRB-

Hofacker

et

al.

-RRB-

but

it

deviates

in

a

number

of

ways

described

below

In

particular

it

can

use

pair

probability

score

matrices

based

on

FOLDALIGN

pairwise

scans

METHODS

Multiple

alignments

were

in

the

early

version

of

FOLDALIGN

found

through

a

greedy

approach

-LRB-

Gorodkin

et

al.

-RRB-

However

a

greedy

approach

with

the

current

more

extensive

FOLDALIGN

version

is

not

suitable

The

current

pairwise

version

of

FOLDALIGN

employs

a

scoring

scheme

in

which

a

joint

score

composed

of

folding

energy

substitution

of

entire

base

pairs

and

single

nucleotides

is

maximized

-LRB-

Havgaard

et

al.

-RRB-

This

complicates

implementation

of

a

greedy

approach

in

the

same

manner

and

is

also

expected

to

be

time-consuming

In

contrast

the

PMcomp

and

PMmulti

algorithms

promise

good

speed

assuming

an

efficient

implementation

Whereas

PMmulti

lacks

the

ability

to

conduct

local

multiple

structural

RNA

alignments

it

is

here

implemented

as

a

possible

layer

on

top

of

a

pairwise

FOLDALIGN

scan

We

do

this

by

calculating

the

base-pair

probability

matrices

using

FOLDALIGN

-LRB-

see

end

of

Section

-RRB-

and

not

McCaskill

's

algorithm

-LRB-

McCaskill

-RRB-

as

implemented

in

PMcomp

-LRB-

Hofacker

et

al.

-RRB-

PMcomp

is

currently

available

as

a

Perl

implementation

and

hence

is

relatively

slow

Therefore

we

re-implemented

PMcomp

in

JAVA

with

several

improvements

This

re-implementation

which

we

named

FOLDALIGNM

can

align

both

McCaskill

's

and

FOLDALIGN

generated

probability

matrices

In

addition

it

can

cluster

similar

candidates

and

perform

alignments

to

a

given

structure

and

sequence

points

are

used

to

divide

the

structure

into

unbranched

segments

This

is

done

by

using

a

short-term

matrix

-LRB-

S

in

Equation

-LRB-

-RRB-

-RRB-

and

a

long-term

matrix

-LRB-

in

Equation

-LRB-

-RRB-

-RRB-

where

the

short-term

matrix

only

has

to

remember

the

last

position

we

came

from

so

we

can

do

this

using

O

-LRB-

-RRB-

memory

The

long-term

matrix

only

contains

the

significant

base

pairs

and

the

branch

points

Because

each

base

can

take

part

in

at

most

many

base

pairs

there

are

only

O

-LRB-

L

-RRB-

significant

base

pairs

in

each

sequence

thus

the

long-term

matrix

requires

only

O

-LRB-

L

-RRB-

memory

-LRB-

and

not

O

-LRB-

-RRB-

because

we

only

allow

length

difference

between

any

subsequences

-RRB-

In

the

second

step

we

treat

all

segments

as

given

by

our

multibranch

point

list

independently

using

only

O

-LRB-

-RRB-

memory

and

CPU

time

for

each

segment

where

M

is

the

length

of

the

segment

In

addition

also

following

Havgaard

et

al.

-LRB-

submitted

for

publication

-RRB-

we

prune

away

low

scoring

cells

saving

CPU

time

-LRB-

depending

on

the

sequences

-RRB-

Pruning

is

simply

done

by

excluding

sub-alignments

from

the

matrix

when

the

score

is

smaller

than

a

given

threshold

Let

Si

j

k

l

be

the

score

of

the

best

match

of

the

subsequences

A

-LSB-

i

j

-RSB-

and

B

-LSB-

k

l

-RSB-

With

this

definition

we

can

obtain

the

following

dynamic

programming

recursions

Si

j

k

l

S

i

j

k

l

Si

j

k

l

Si

j

k

l

Si

j

k

l

max

Si

j

k

l

Ai

Bk

Si

j

k

l

Aj

Bl

Si

j

k

l

A

B

Ai

Aj

Bk

Bl

ij

kl

o

n

max

Si

m

k

n

m

j

n

l

PMcomp

improvements

The

PMcomp

algorithm

is

in

general

implemented

as

described

by

Hofacker

et

al.

-LRB-

-RRB-

but

with

several

improvements

Briefly

PMcomp

aligns

two

sequences

A

and

B

by

aligning

their

base-pair

probability

matrices

PA

and

PB

which

are

computed

by

the

McCaskill

's

algorithm

-LRB-

McCaskill

-RRB-

Only

base-pair

probabilities

above

a

given

cutoff

which

we

note

p

are

considered

In

general

PMcomp

looks

for

an

alignment

of

the

sequences

A

and

B

with

Ngap

insertions

or

deletions

together

with

a

consensus

secondary

structure

Z

such

that

i

X

h

A

B

Ai

Aj

Bk

Bl

Ngap

ij

kl

ij

kl

X

Ai

Bk

max

with

the

initialization

SLA

LA

LB

LB

where

LA

and

LB

are

the

lengths

of

sequence

A

and

B

respectively

The

first

four

terms

account

for

gaps

in

one

of

the

two

sequences

The

next

two

terms

describe

the

extension

of

both

subsequences

with

an

unpaired

position

The

next

term

describes

a

base-pair

match

in

both

subsequences

and

finally

the

last

term

describes

the

joining

of

two

alignments

to

get

a

multibranched

structure

This

is

only

performed

when

the

right

part

has

base

pairs

between

-LRB-

m

and

j

-RRB-

and

-LRB-

n

and

l

-RRB-

kept

in

and

the

left

part

of

the

structure

is

limited

to

the

cases

where

i

is

base

paired

and

k

is

base

paired

-LRB-

Havgaard

et

al.

-RRB-

The

last

term

is

not

needed

in

step

two

where

we

only

align

unbranched

segments

because

we

know

that

they

should

be

unbranched

from

step

one

where

we

locate

all

the

multibranch

points

This

recursion

differs

from

that

of

PMcomp

in

two

ways

first

of

all

we

can

move

in

both

directions

on

both

sequences

and

secondly

PMcomp

joins

together

the

last

two

terms

because

they

calculate

everything

in

one

step

and

not

in

two

as

we

do

The

maximum

allowed

length

difference

gives

the

following

restrictions

i

k

j

i

l

k

PMmulti

constructs

a

progressive

multiple

alignment

using

PMcomp

-LRB-

Hofacker

et

al.

-RRB-

PMmulti

use

the

weighted

pair-group

clustering

method

to

produce

a

guide

tree

with

similarity

scores

For

a

multiple

alignment

of

N

RNAs

the

similarity

scores

are

computed

using

N

-LRB-

N

-RRB-

cheap

string-like

alignments

which

only

cost

-LRB-

O

-LRB-

-RRB-

-RRB-

in

time

-LRB-

Bonhoeffer

et

al.

-RRB-

The

more

expensive

PMcomp

algorithm

is

restricted

to

the

N

progressive

alignments

along

the

guide

tree

i.e.

first

it

aligns

the

highest

scoring

pair

calculates

a

consensus

base-pair

probability

matrix

for

these

and

aligns

the

next

sequence

to

this

consensus

matrix

then

makes

a

new

consensus

matrix

for

these

three

sequences

and

so

on

Hence

the

program

always

compares

two

probability

matrices

FOLDALIGNM

also

contains

the

same

progressive

multiple-alignment

approach

except

when

using

FOLDALIGN

base

probability

matrices

where

we

simply

use

the

pairwise

FOLDALIGN

scores

which

are

already

available

to

produce

the

guide

tree

Z

where

A

is

the

weight

of

pair

-LRB-

i

j

-RRB-

from

sequence

A

is

the

gap

ij

penalty

and

the

scores

represented

by

and

describe

the

substitution

of

unpaired

bases

and

base

pairs

respectively

-LRB-

Hofacker

et

al.

-RRB-

Here

we

have

added

the

FOLDALIGN

parameter

which

is

the

maximum

allowed

length

difference

in

an

alignment

between

two

subsequences

In

general

this

gives

complexities

of

O

-LRB-

-RRB-

in

time

and

memory

where

L

is

the

length

of

the

sequence

reducing

the

PMcomp

complexities

of

O

-LRB-

-RRB-

Furthermore

we

also

include

the

following

memory

and

time

improvements

presented

in

the

updated

version

of

FOLDALIGN

for

pairwise

-LRB-

local

-RRB-

alignments

-LRB-

Havgaard

et

al.

submitted

for

publication

-RRB-

The

dynamic

programming

matrix

is

filled

in

two

steps

as

described

in

details

by

Havgaard

et

al.

-LRB-

submitted

for

publication

-RRB-

Briefly

in

the

first

step

only

the

multibranch

points

are

located

which

are

the

positions

where

we

join

two

substructures

The

multibranch

E.Torarinsson

et

al.

In

PMcomp

the

consensus

base-pair

probability

matrix

is

defined

for

a

pair

of

columns

pq

in

the

alignment

of

A

and

B

as

-LRB-

qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

PAjq

PAp

lq

for

matches

ip

k

PAB

p

q

for

gaps

where

ip

and

jq

are

the

positions

corresponding

to

p

and

q

in

sub-alignment

A

respectively

kp

and

lq

are

defined

analogously

for

sequence

B

This

means

that

whenever

one

sub-alignment

contains

a

gap

or

has

a

very

low

base-pair

probability

then

the

structural

information

from

the

other

sub-alignment

is

lost

This

tends

to

remove

most

base

pairs

when

aligning

many

sequences

In

order

to

avoid

this

problem

in

PMcomp

we

add

a

step

to

redefine

the

final

alignment

In

this

step

we

compute

the

most

frequently

occurring

base

pairs

in

all

the

progressive

alignments

make

a

new

base

pair

probability

matrix

with

these

base

pairs

and

assign

the

probability

of

to

each

base

pair

and

align

each

sequence

once

again

to

this

consensus

matrix

More

specifically

we

count

for

every

column

in

the

alignment

the

number

of

sequences

in

the

alignment

that

base

pair

at

this

column

If

more

than

Forty

percent

of

the

sequences

base

pair

at

these

columns

they

are

added

to

a

new

consensus

probability

matrix

with

probability

Here

we

are

not

defining

a

strict

consensus

structure

but

a

looser

list

of

frequently

occurring

base

pairs

that

we

align

again

to

each

sequence

Forty

percent

was

chosen

after

testing

several

different

parameters

but

it

can

also

be

given

as

an

argument

to

the

program

This

tends

to

improve

on

the

alignment

especially

when

dealing

with

many

sequences

To

generate

FOLDALIGN

base-pair

probability

matrices

we

compute

all

N

-LRB-

N

-RRB-

pairwise

alignments

For

each

sequence

the

frequency

of

each

base

pair

is

simply

the

observed

frequency

i.e.

when

aligning

sequences

and

a

given

base

pair

in

sequence

A

was

predicted

in

five

of

all

possible

pairwise

alignments

the

probability

of

this

base

pair

in

sequence

A

would

then

be

Performance

measures

The

accuracy

of

all

predictions

is

computed

at

the

base-pair

level

relative

to

the

Rfam

base-pair

assignments

ignoring

non-canonical

base

pairs

Let

Pt

be

the

correctly

predicted

base

pairs

-LRB-

assuming

that

the

Rfam

assignment

is

correct

-RRB-

Pf

the

falsely

predicted

base

pairs

and

Nf

the

true

base

pairs

that

are

not

predicted

The

sensitivity

is

defined

as

Pt

-LRB-

Pt

Nf

-RRB-

positive

predictive

value

-LRB-

PPV

-RRB-

as

Pt

-LRB-

Pt

Pf

-RRB-

-LRB-

also

referred

to

as

specificity

in

some

articles

-RRB-

and

overall

prediction

accuracy

as

their

geometric

mean

For

good

sensitivity

and

PPV

the

latter

metric

approximates

Matthews

correlation

coefficient

-LRB-

MCC

-RRB-

-LRB-

Gorodkin

et

al.

-RRB-

We

calculate

the

prediction

accuracy

for

each

sequence

and

report

the

average

for

the

whole

family

as

the

average

of

every

sequence

belonging

to

that

family

As

we

want

to

compare

our

results

to

that

of

CMfinder

-LRB-

Yao

et

al.

-RRB-

which

uses

this

performance

measure

we

also

use

this

measure

For

evaluating

the

cluster

prediction

we

calculate

MCC

-LRB-

Matthews

-RRB-

Pt

Nt

Pf

Nf

MCC

Pt

Pf

Pt

Nf

Nt

Pf

Nt

Nf

Data

Clustering

can

also

cluster

the

candidates

that

are

similar

in

structure

and

sequence

based

on

all-against-all

pairwise

FOLDALIGN

scores

As

an

initial

step

we

used

a

simple

clustering

procedure

described

in

the

following

somewhat

similar

to

simple

single-linkage

clustering

In

the

clustering

we

start

by

sorting

the

pairwise

FOLDALIGN

alignments

based

on

their

scores

Then

we

add

the

sequences

in

the

highest

scoring

alignment

to

the

first

cluster

and

go

through

the

list

of

descending

scores

and

if

the

score

is

above

a

given

cutoff

-LRB-

an

option

to

the

program

-RRB-

do

the

following

-LRB-

-RRB-

If

neither

sequence

exists

in

a

cluster

make

a

new

cluster

and

add

them

to

it

-LRB-

-RRB-

If

one

of

the

sequences

exists

in

a

cluster

add

the

other

one

to

that

clustering

-LRB-

-RRB-

If

both

sequences

are

already

in

the

same

cluster

do

nothing

-LRB-

-RRB-

If

the

sequences

are

in

different

clusters

note

the

score

of

the

alignment

and

add

this

score

to

the

score

between

these

two

clusters

Go

through

the

clusters

and

check

if

there

exists

a

score

between

clusters

-LRB-

step

-RRB-

i.e.

check

if

there

are

any

high

scoring

pairs

whose

sequences

belong

to

different

clusters

If

so

check

if

the

average

score

between

the

sequences

in

these

clusters

is

higher

than

a

given

cutoff

which

depends

directly

on

the

first

cutoff

mentioned

above

If

so

merge

the

clusters

otherwise

do

not

merge

the

clusters

The

choice

of

cutoff

is

crucial

for

the

clustering

procedure

By

choosing

a

high

cutoff

one

gets

small

well-defined

clusters

with

higher

specificity

whereas

a

low

cutoff

would

result

in

large

clusters

with

higher

sensitivity

This

simple

clustering

approach

worked

very

well

-LRB-

see

Results

section

-RRB-

so

we

did

not

implement

a

more

advanced

clustering

scheme

FOLDALIGNM

Five

different

datasets

were

used

to

test

FOLDALIGNM

For

a

general

evaluation

of

performance

on

global

alignments

we

used

the

CMfinder

datasets

from

Yao

et

al.

-LRB-

-RRB-

These

consist

of

different

families

from

the

Rfam

seed

alignments

-LRB-

Griffiths-Jones

et

al.

-RRB-

and

contain

sequences

per

family

with

average

sequence

similarities

ranging

from

to

and

average

lengths

from

to

We

excluded

the

family

Entero

OriR

because

all

the

sequences

miss

half

of

the

motif

in

the

evaluation

made

by

Yao

et

al.

-LRB-

-RRB-

Cobalamin

was

also

excluded

because

it

has

sequences

with

extreme

differences

in

lengths

which

due

to

memory

limitations

in

our

method

is

not

feasible

We

refer

to

this

dataset

as

the

CMfinder

dataset

We

also

made

a

more

comprehensive

dataset

for

three

different

families

from

the

Rfam

seed

alignments

-LRB-

Griffiths-Jones

et

al.

-RRB-

This

dataset

includes

three

families

where

each

family

includes

subsets

with

sequences

and

and

sequence

similarity

i.e.

there

are

subsets

with

two

sequences

that

each

has

sequence

similarity

to

each

other

etc.

This

gives

subsets

for

each

family

We

used

the

families

tRNA

bacterial

SRP

and

eukaryotic

and

archaeal

SRP

because

they

have

well-defined

structures

-LRB-

Griffiths-Jones

et

al.

Rosenblad

et

al.

-RRB-

and

are

the

only

families

where

it

is

possible

to

make

such

a

comprehensive

dataset

from

the

Rfam

seed

sequences

In

fact

we

could

only

make

a

complete

dataset

for

tRNA

The

other

families

were

complete

only

for

two

to

five

sequences

and

it

was

in

some

cases

not

possible

to

generate

different

sets

with

six

to

ten

sequences

for

some

specific

sequence

similarities

Still

the

dataset

was

adequate

in

providing

good

understanding

about

the

different

strengths

of

the

algorithms

We

refer

to

this

set

as

the

global

comprehensive

dataset

To

test

the

clustering

function

the

same

families

as

Yao

et

al.

used

were

reused

We

constructed

different

datasets

where

we

randomly

chose

between

five

and

nine

sequences

from

each

of

the

families

put

them

all

together

and

ran

our

clustering

procedure

on

this

dataset

This

is

our

global

clustering

set

For

the

local

clustering

we

prepared

two

datasets

a

local

pairwise

set

to

test

clustering

on

top

of

FOLDALIGN

local

pairwise

scans

and

a

local

multiple

set

to

test

multiple

local

clustering

using

all-against-all

FOLDALIGN

The

local

pairwise

dataset

contained

tRNA

THI

two

and

two

Purine

sequence

pairs

from

the

FOLDALIGN

dataset

-LRB-

Havgaard

et

al.

-RRB-

Each

motif

lies

within

its

long

genomic

context

and

each

pair

has

less

than

sequence

similarity

and

is

energetically

indistinguishable

from

its

genomic

context

For

the

local

mutiple

dataset

we

used

from

the

long

sequences

eight

tRNAs

Multiple

structural

alignment

and

clustering

of

RNA

sequences

seven

and

five

Purines

in

addition

we

shuffled

the

nucleotides

for

two

randomly

chosen

sequences

from

each

of

these

families

and

added

those

to

our

set

The

shuffling

preserves

the

dinucleotide

frequencies

-LRB-

Altschul

and

Erickson

Workman

and

Krogh

-RRB-

RESULTS

The

performance

of

FOLDALIGNM

was

evaluated

on

the

five

different

datasets

described

in

Materials

and

methods

section

These

are

-LRB-

i

-RRB-

the

CMfinder

dataset

-LRB-

ii

-RRB-

our

global

comprehensive

dataset

-LRB-

iii

-RRB-

global

clustering

dataset

-LRB-

iv

-RRB-

a

local

pairwise

set

and

-LRB-

v

-RRB-

a

local

multiple

set

Global

alignment

For

our

general

test

we

used

a

global

dataset

generated

and

tested

by

Yao

et

al.

-LRB-

-RRB-

on

their

own

algorithm

CMfinder

and

several

others

Using

the

fast

version

of

our

program

which

uses

McCaskill

's

base-pairing

matrices

like

PMcomp

does

we

compared

our

performance

to

that

of

CMfinder

and

their

runs

with

Pfold

-LRB-

Knudsen

and

Hein

-RRB-

and

RNAalifold

-LRB-

Hofacker

et

al.

-RRB-

on

these

families

In

addition

we

also

ran

STRAL

-LRB-

Dalli

et

al.

-RRB-

with

RNAalifold

Stemloc

-LRB-

Holmes

-RRB-

and

RNAcast

-LRB-

Reeder

and

Giegerich

-RRB-

using

the

default

parameters

STRAL

and

RNAalifold

performed

as

expected

slightly

better

than

using

Clustalw

and

RNAalifold

with

average

MCC

of

FOLDALIGNM

and

CMfinder

showed

comparable

performances

which

again

perform

significantly

better

than

the

other

approaches

-LRB-

see

Table

-RRB-

FOLDALIGNM

outputs

two

different

alignments

one

similar

to

that

of

PMmulti

and

one

where

we

used

our

redefining

method

to

redefine

the

alignments

-LRB-

see

Material

and

methods

sections

-RRB-

We

calculated

the

performance

of

FOLDALIGNM

using

McCaskill

's

base-pair

probability

matrices

for

both

alignments

The

average

for

Table

Performance

on

Rfam

families

from

the

CMfinder

dataset

Family

Entero

CRE

Histone

IRE

Intron

gpII

Lysine

Purine

RFN

Rhino

CRE

SECIS

S

box

Tymo

tRNA-like

ctRNA

glmS

AVG

Number

of

sequences

FOLD

ALIGNM

our

redefined

alignments

was

whereas

the

average

was

for

the

original

alignments

so

we

always

used

the

redefined

one

For

this

dataset

FOLDALIGNM

uses

min

to

output

the

results

for

most

of

these

families

although

the

longer

families

and

those

with

large

differences

in

sequence

length

can

take

a

few

hours

to

run

Programs

like

RNACast

and

Stemloc

are

seemingly

not

performing

too

well

but

if

one

only

considers

the

families

where

they

actually

produce

a

prediction

they

perform

quite

well

with

an

average

MCC

of

and

respectively

RNAcast

is

very

fast

so

it

can

be

nice

as

a

starting

point

since

it

will

usually

perform

well

if

it

finds

a

consensus

shape

for

the

different

input

sequences

Playing

around

with

different

parameters

would

probably

result

in

better

performances

especially

for

those

two

algorithms

In

order

to

gain

better

insight

into

the

different

properties

of

FOLDALIGNM

and

CMfinder

we

constructed

and

ran

tests

on

a

global

comprehensive

dataset

This

dataset

contains

subsets

with

different

number

of

sequences

and

different

sequence

similarity

ranges

FOLDALIGN

is

particularly

effective

on

pairs

with

low

sequence

similarity

This

is

also

the

case

for

FOLDALIGNM

which

outperforms

CMfinder

in

the

sequence

identity

range

-LRB-

Table

and

Fig.

-RRB-

The

performance

is

similar

in

the

sequence

identity

range

and

CMfinder

performs

better

in

the

similarity

range

FOLDALIGNM

has

a

tendency

to

perform

better

on

few

sequences

which

is

not

surprising

in

that

CMfinder

is

designed

with

more

sequences

in

mind

where

it

is

very

effective

Global

clustering

When

using

all-against-all

FOLDALIGN

pairwise

alignments

to

make

the

base-pair

probability

matrix

one

also

gets

pairwise

CM

finder

Clustal/Pfold

FOLDALIGNM

Clustal/Alifold

Stem

loc

RNA

cast

The

numbers

are

approximated

Matthew

's

correlation

coefficients

as

described

in

Methods

section

matrices

is

run

with

McCaskill-generated

base-pair

probability

E.Torarinsson

et

al.

Table

Comprehensive

four-family

dataset

for

different

pairwise

sequence

similarity

ranges

tRNA

CMfinder

FOLDALIGNM

McCaskill

FOLDALIGNM

FOLDALIGN

SRP-euk-arch

CMfinder

FOLDALIGNM

McCaskill

FOLDALIGNM

FOLDALIGN

SRP-bacterial

CMfinder

FOLDALIGNM

McCaskill

FOLDALIGNM

FOLDALIGN

The

numbers

are

approximated

Matthew

's

correlation

coefficients

as

described

in

Methods

section

FOLDALIGNM

McCaskill

version

uses

McCaskill

's

base-pair

probability

matrix

whereas

FOLDALIGNM

FOLDALIGN

uses

a

FOLDALIGN

generated

matrix

These

numbers

represent

the

averages

for

the

datasets

containing

sequences

where

each

sequence

in

every

set

have

the

noted

sequence

similarity

to

every

other

sequence

in

its

set

tRNA

sequence

similarity

Average

MCC

Number

of

sequences

per

alignment

tRNA

sequence

similarity

Average

MCC

Number

of

sequences

per

alignment

CMfinder

FoldalignM

McCaskill

Average

MCC

Number

of

sequences

per

alignment

FoldalignM

Foldalign

Average

MCC

Number

of

sequences

per

alignment

tRNA

sequence

similarity

tRNA

sequence

similarity

Fig.

The

average

MCC

performance

of

CMfinder

sets

of

tRNA

FOLDALIGNM

McCaskill

and

FOLDALIGNM

FOLDALIGN

on

different

sequence

similarity

scores

between

all

the

sequences

These

scores

can

be

used

to

cluster

similar

sequences

together

and

then

we

generate

a

multiple

alignment

for

each

cluster

To

test

this

we

again

used

the

family

dataset

from

Yao

et

al.

-LRB-

-RRB-

We

constructed

different

datasets

where

we

randomly

chose

between

and

sequences

from

each

of

the

families

and

ran

our

clustering

procedure

on

this

dataset

We

calculated

Matthew

's

correlation

coefficient

by

considering

every

possible

pair

Multiple

structural

alignment

and

clustering

of

RNA

sequences

of

sequences

counting

true

and

false

positives

and

negatives

as

follows

a

true

positive

is

a

pair

of

sequences

in

the

same

family

and

same

cluster

a

false

positive

is

a

pair

of

sequences

not

in

the

same

family

but

in

the

same

cluster

a

true

negative

is

a

pair

of

sequences

not

in

the

same

family

and

not

in

the

same

cluster

and

finally

a

false

negative

is

a

pair

of

sequences

in

the

same

family

but

not

in

the

same

cluster

The

average

MCC

for

these

datasets

is

The

following

results

were

averaged

for

the

datasets

From

every

dataset

containing

sequences

different

clusters

were

predicted

Nineteen

sequences

do

not

belong

to

any

cluster

Seven

clusters

were

perfectly

predicted

containing

all

the

sequences

in

the

respective

family

Five

clusters

were

correct

but

were

missing

one

or

more

sequences

from

the

family

Six

families

were

split

into

two

-LRB-

generally

-RRB-

or

more

clusters

and

one

cluster

contained

sequences

from

two

or

more

families

Usually

a

single

sequence

from

one

miRNA

family

joined

another

miRNA

family

cluster

To

test

the

robustness

of

the

clustering

we

added

the

shuffled

sequences

maintaining

dinucleotide

frequencies

-LRB-

Altschul

and

Erickson

Workman

and

Krogh

-RRB-

of

two

families

to

one

of

our

datasets

and

ran

it

again

The

results

for

this

dataset

did

not

change

and

all

the

shuffled

sequences

were

predicted

to

be

outside

the

clusters

CONCLUSION

Semi-local

clustering

Since

FOLDALIGN

performs

local

pairwise

alignments

we

can

further

conduct

semi-local

clustering

Even

though

it

is

beyond

the

scope

of

this

article

to

do

local

clustering

for

multiple

sequences

we

could

still

perform

some

putative

studies

on

the

possibilities

for

doing

local

clustering

and

alignments

using

FOLDALIGN

input

For

this

we

used

tRNA

THI

two

and

two

Purine

sequence

pairs

We

extracted

the

best

hit

from

every

pairwise

scan

added

the

flanking

genomic

nucleotides

to

both

ends

and

ran

local

all-against-all

pairwise

FOLDALIGN

alignments

Clustering

and

aligning

the

results

from

this

resulted

in

three

good

clusters

one

with

all

four

one

with

all

four

Purines

and

finally

one

with

of

the

tRNAs

Both

the

localization

and

the

structure

prediction

were

very

good

with

an

average

MCC

of

for

the

three

clusters

FOLDALIGN

does

not

perform

very

well

on

THI

and

therefore

these

did

not

cluster

well

together

These

results

indicate

that

we

can

cluster

sequences

with

well-defined

structures

from

a

pairwise

scan

We

also

made

a

dataset

to

test

basic

local

multiple

alignment

and

clustering

using

a

dataset

with

eight

tRNAs

seven

five

Purines

and

six

shuffled

sequences

Running

all-against-all

local

pairwise

FOLDALIGN

followed

by

clustering

and

multiple

alignment

on

this

dataset

resulted

in

three

perfect

clusters

with

good

localization

and

good

structure

prediction

averaging

a

MCC

of

for

the

three

clusters

Despite

significant

improvements

to

the

latest

version

of

FOLDALIGN

-LRB-

Havgaard

et

al.

submitted

for

publication

-RRB-

it

is

still

time-consuming

to

run

larger

datasets

pairwise

all-against-all

with

FOLDALIGN

This

makes

the

use

of

FOLDALIGN

generated

base-pair

probability

matrices

more

attractive

when

you

already

have

the

FOLDALIGN

output

available

We

have

based

on

the

PMcomp

program

presented

a

global

multiple

alignment

algorithm

that

considers

both

sequence

and

structure

It

has

comparable

performance

to

other

recent

programs

and

is

especially

effective

on

sequences

with

low

sequence

similarities

In

addition

to

this

our

program

FOLDALIGNM

is

also

capable

of

clustering

similar

sequences

together

and

providing

a

multiple

alignment

for

each

cluster

Further

the

current

version

should

already

be

useful

in

clustering

results

from

a

scan

such

as

that

performed

by

Torarinsson

et

al.

-LRB-

-RRB-

Perspectives

for

FOLDALIGNM

include

implementing

a

more

advanced

clustering

approach

like

for

example

as

described

by

Ding

et

al.

-LRB-

-RRB-

using

hierarchical

clustering

-LRB-

Johnson

-RRB-

and

then

estimating

the

number

of

clusters

with

the

CH

index

-LRB-

Calinski

and

Harabasz

-RRB-

which

was

assessed

to

be

the

best

in

a

comprehensive

study

-LRB-

Milligan

and

Cooper

-RRB-

Although

our

simple

clustering

works

well

it

could

be

interesting

to

compare

to

state-of-the-art

clustering

algorithms

Furthermore

we

would

like

to

implement

a

local

version

Our

preliminary

results

where

we

localize

the

hits

using

all-against-all

FOLDALIGN

are

very

promising

and

the

performance

is

likely

to

increase

with

a

local

version

of

FOLDALIGNM

Also

when

clustering

large

datasets

it

is

not

necessary

to

run

all-against-all

FOLDALIGN

as

we

do

now

since

many

sequences

could

be

pre-filtered

based

on

features

such

as

length

and

sequence

and

structure

similarity

before

running

the

more

expensive

all-against-all

FOLDALIGN

Our

algorithm

is

sensitive

to

very

large

differences

between

the

lengths

of

the

sequences

and

is

quite

memory

intensive

on

long

sequences

When

this

work

was

being

finished

we

realized

that

another

re-implementation

of

PMComp

LocaRNA

-LRB-

Will

et

al.

submitted

for

publication

-RRB-

deals

very

elegantly

with

this

problem

and

uses

only

O

-LRB-

-RRB-

memory

ACKNOWLEDGEMENTS

This

work

was

supported

by

Danish

Research

Council

for

production

and

technology

and

the

Danish

Center

for

Scientific

Computation

We

thank

Ivo

L.

Hofacker

Peter

F.

Stadler

Paul

P.

Gardner

and

Stinus

Lindgreen

for

comments

on

this

manuscript

Conflict

of

Interest

declared

REFERENCES

Altschul

S.F.

and

Erikson

B.W.

-LRB-

-RRB-

Significance

of

nucleotide

sequence

alignments

a

method

for

random

sequence

permutation

that

preserves

dinucleotide

and

codon

usage

Mol

Biol

Evol.

Ban

N.

et

al.

-LRB-

-RRB-

The

complete

atomic

structure

of

the

large

ribosomal

subunit

at

A

resolution

Science

Bonhoeffer

S.

et

al.

-LRB-

-RRB-

RNA

multistructure

landscapes

Eur

Biophys

J.

Calinski

R.B.

and

Harabasz

J.

-LRB-

-RRB-

A

dendrite

method

for

cluster

analysis

Comm

Stat.

Dalli

D.

et

al.

-LRB-

-RRB-

StrAl

progressive

alignment

of

non-coding

RNA

using

base

pairing

probability

vectors

in

quadratic

time

Bioinformatics

Ding

Y.

et

al.

-LRB-

-RRB-

Sfold

web

server

for

statistical

folding

and

rational

design

of

nucleic

acids

Nucleic

Acids

Res.

E.Torarinsson

et

al.

Ding

Y.

et

al.

-LRB-

-RRB-

RNA

secondary

structure

prediction

by

centroids

in

a

Boltzmann

weighted

ensemble

RNA

Eddy

S.R.

and

Durbin

R.

-LRB-

-RRB-

RNA

sequence

analysis

using

covariance

models

Nucleic

Acids

Res.

Eddy

S.R.

-LRB-

-RRB-

Non-coding

RNA

genes

and

the

modern

RNA

world

Nat

Rev.

Genet.

Eddy

S.R.

-LRB-

-RRB-

A

memory

efficient

dynamic

programming

algorithm

for

optimal

structural

alignment

of

a

sequence

to

an

RNA

secondary

structure

BMC

Bioinformatics

Gorodkin

J.

et

al.

-LRB-

-RRB-

Finding

the

most

significant

common

sequence

and

structure

motifs

in

a

set

of

RNA

sequences

Nucleic

Acids

Res.

Gorodkin

J.

et

al.

-LRB-

-RRB-

Discovering

common

stem-loop

motifs

in

unaligned

RNA

sequences

Nucleic

Acids

Res.

Griffiths-Jones

S.

et

al.

-LRB-

-RRB-

Rfam

an

RNA

family

database

Nuclelic

Acids

Res.

Gutell

R.

et

al.

-LRB-

-RRB-

The

accuracy

of

ribosomal

RNA

comparative

structure

models

Curr

Opin

Struct

Biol.

Havgaard

J.H.

et

al.

-LRB-

-RRB-

Pairwise

local

structural

alignment

of

RNA

sequences

with

sequence

similarity

less

than

Bioinformatics

Hofacker

I.L.

-LRB-

-RRB-

Fast

folding

and

comparison

of

RNA

secondary

structures

Monatsh

Chem.

Hofacker

I.L.

et

al.

-LRB-

-RRB-

Secondary

structure

prediction

for

aligned

RNA

sequences

J.

Mol

Biol.

Hofacker

I.L.

et

al.

-LRB-

-RRB-

Alignment

of

RNA

base

pairing

probability

matrices

Bioinfomatics

Holmes

I.

-LRB-

-RRB-

Accelerated

probabilistic

inference

of

RNA

structure

evolution

BMC

Bioinformatics

Huttenhoffer

A.

et

al.

-LRB-

-RRB-

RNomics

identification

and

function

of

small

non-messenger

RNAs

Curr

Opin

Chem

Biol.

Johnson

S.C.

-LRB-

-RRB-

Hierarchical

clustering

schemes

Psychometrika

Klein

R.J.

and

Eddy

S.R.

-LRB-

-RRB-

RSEARCH

finding

homologs

of

single

structured

RNA

sequences

BMC

Bioinformatics

Knudsen

B.

and

Hein

J.

-LRB-

-RRB-

RNA

secondary

structure

prediction

using

stochastic

context-free

grammars

and

evolutionary

history

Bioinformatics

Matthews

B.W.

-LRB-

-RRB-

Comparison

of

the

predicted

and

observed

secondary

structure

of

phage

lysozyme

Biochem

Biophys

Acta

Mattick

J.S.

-LRB-

-RRB-

Non-coding

RNAs

the

architects

of

eukaryotic

complexity

EMBO

Rep

McCaskill

J.S.

-LRB-

-RRB-

The

equilibrium

partition

function

and

base

pair

binding

probabilities

for

RNA

secondary

structure

Biopolymers

Milligan

G.W.

and

Cooper

M.C.

-LRB-

-RRB-

An

examination

of

procedures

for

determining

the

number

of

clusters

in

a

data

set

Psychometrika

Pedersen

J.S.

et

al.

-LRB-

-RRB-

Identification

and

classification

of

conserved

RNA

secondary

structures

in

the

human

genome

PLoS

Comput

Biol.

Reeder

J.

and

Giegerich

R.

-LRB-

-RRB-

Consensus

shapes

an

alternative

to

the

Sankoff

algorithm

for

RNA

consensus

structure

prediction

Bioinformatics

Rosenblad

M.A.

et

al.

-LRB-

-RRB-

SRPDB

signal

recognition

particle

database

Nucleic

Acids

Res.

Sankoff

D.

-LRB-

-RRB-

Simultaneous

solution

of

the

RNA

folding

alignment

and

protosequence

problems

SIAM

J.

Appl

Math.

Torarinsson

E.

et

al.

-LRB-

-RRB-

Thousand

of

corresponding

human

and

mouse

genomic

regions

unalignable

in

primary

sequence

contain

common

RNA

strucuture

Genome

Res.

Uzilov

A.V.

et

al.

-LRB-

-RRB-

Detection

of

non-coding

RNAs

on

the

basis

of

predicted

secondary

structure

formation

free

energy

change

BMC

Bioinformatics.

Washietl

S.

et

al.

-LRB-

-RRB-

Fast

and

reliable

prediction

of

noncoding

RNAs

Proc

Natl.

Acad

Sci

USA

Weinberg

Z.

and

Ruzzo

W.L.

-LRB-

-RRB-

Exploiting

conserved

structure

for

faster

annotation

of

non-coding

RNAs

without

loss

of

accuracy

Bioinformatics

-LRB-

Suppl

-RRB-

Weinberg

Z.

and

Ruzzo

W.L.

-LRB-

-RRB-

Faster

genome

annotation

of

non-coding

rna

families

without

loss

of

accuracy

In

Proceedings

of

the

Eighth

Annual

International

Conference

on

Computational

Molecular

Biology

-LRB-

RECOMB

-RRB-

ACM

Press

pp.

Westhof

E.

and

Michel

F.

-LRB-

-RRB-

In

Nagai

K.

and

Mattaj

I.W.

-LRB-

eds

-RRB-

RNAProtein

Interactions

Oxford

University

Press

Oxford

UK

pp.

Westhof

E.

et

al.

-LRB-

-RRB-

In

Bihop

M.

J.

and

Rawlings

C.

J.

-LRB-

eds

-RRB-

DNA-Protein

Sequence

Analysis

Oxford

University

Press

Oxford

UK

pp.

Workman

C.

and

Krogh

A.

-LRB-

-RRB-

No

evidence

that

mRNA

have

lower

folding

free

energies

than

random

sequences

with

the

same

dinucleotide

distribution

Nucleic

Acids

Res.

Yao

Z.

et

al.

-LRB-

-RRB-

Cmfinder

a

covariance

model

based

RNA

motif

finding

algorithm

Bioinformatics

Zuker

M.

-LRB-

-RRB-

Mfold

web

server

for

nucleic

acid

folding

and

hybridization

prediction

Nucleic

Acids

Res.

O O O O O O O O O O O O O O O O O O O O O O B O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O