Single-nucleotide-resolution chemical mapping for structured RNA is being rapidly advanced by new chemistries, faster readouts, and coupling to computational algorithms. Recent tests have suggested that 2' -OH acylation data (SHAPE) can give near-zero error rates ( 0-4 \%) in modeling  RNA secondary structure. Here, we benchmark the method on six RNAs for which crystallographic data are available: tRNAphe and 5S rRNA from E. coli; the P4-P6 domain of the Tetrahymena group I ribozyme ; and ligand-bound domains from riboswitches for adenine, cyclic di-GMP, and glycine. SHAPE-directed modeling of these RNAs gave significant errors ( overall false negative rate  of 17\% ; false discovery rate  of 21\% ) , with at least one  error in five of the six cases. Variations of data processing or modeling do not mitigate these errors. Instead, as evaluated by bootstrapping, the information content of SHAPE data appears insufficient to define these RNAs' structures. Thus, SHAPE-directed RNA modeling is not always accurate , and helix-by-helix confidence estimates, as described herein, may be critical for interpreting results from this powerful methodology.
Single-nucleotide-resolution chemical mapping for structured RNA is being rapidly advanced by new chemistries, faster readouts, and coupling to computational algorithms. Recent tests have shown that selective 2' -hydroxyl acylation by primer extension (SHAPE) can give near-zero error rates ( 0-2 \%) in modeling the helices of RNA secondary structure. Here, we benchmark the method using six molecules for which crystallographic data are available: tRNA(phe) and 5S rRNA from Escherichia coli, the P4-P6 domain of the Tetrahymena group I ribozyme , and ligand-bound domains from riboswitches for adenine, cyclic di-GMP, and glycine. SHAPE-directed modeling of these highly structured RNAs gave an overall false negative rate (FNR) of 17\% and a false discovery rate (FDR) of 21\% , with at least one helix prediction error in five of the six cases. Extensive variations of data processing , normalization, and modeling parameters did not significantly mitigate modeling errors. Only one varation, filtering out data collected with deoxyinosine triphosphate during primer extension, gave a modest improvement (FNR = 12\%, and FDR = 14\%). The residual structure modeling errors are explained by the insufficient information content of these RNAs' SHAPE data, as evaluated by a nonparametric bootstrapping analysis. Beyond these benchmark cases, bootstrapping suggests a low level of confidence (<50\%) in the majority of helices in a previously proposed SHAPE-directed model for the HIV-1 RNA genome. Thus, SHAPE-directed RNA modeling is not always unambiguous , and helix-by-helix confidence estimates, as described herein, may be critical for interpreting results from this powerful methodology.
R
suggested that
shown that selective
R
-OH acylation data
-hydroxyl acylation by primer extension
R
0-4
0-2
A
the helices of
R
on six RNAs
using six molecules
R
tRNAphe
tRNA(phe)
R
E. coli;
Escherichia coli,
R
;
,
R
RNAs gave significant errors (
highly structured RNAs gave an
A
(FNR)
R
;
and a
A
(FDR)
D
)
A
helix prediction
R
Variations
Extensive variations
R
or modeling do not mitigate these errors. Instead, as evaluated by bootstrapping, the
, normalization, and modeling parameters did not significantly mitigate modeling errors. Only one varation, filtering out data collected with deoxyinosine triphosphate during primer extension, gave a modest improvement (FNR = 12\%, and FDR = 14\%). The residual structure modeling errors are explained by the insufficient
D
SHAPE data appears insufficient to define
R
structures.
SHAPE data, as evaluated by a nonparametric bootstrapping analysis. Beyond these benchmark cases, bootstrapping suggests a low level of confidence (<50\%) in the majority of helices in a previously proposed SHAPE-directed model for the HIV-1 RNA genome.
R
accurate
unambiguous