  What factors affect a scientist's choice of research problem? Qualitative
research in the history, philosophy, and sociology of science suggests that
this choice is shaped by an "essential tension" between the professional demand
for productivity and a conflicting drive toward risky innovation. We examine
this tension empirically in the context of biomedical chemistry. We use complex
networks to represent the evolving state of scientific knowledge, as expressed
in publications. We then define research strategies relative to these networks.
Scientists can introduce novel chemicals or chemical relationships--or delve
deeper into known ones. They can consolidate existing knowledge clusters, or
bridge distant ones. Analyzing such choices in aggregate, we find that the
distribution of strategies remains remarkably stable, even as chemical
knowledge grows dramatically. High-risk strategies, which explore new chemical
relationships, are less prevalent in the literature, reflecting a growing focus
on established knowledge at the expense of new opportunities. Research
following a risky strategy is more likely to be ignored but also more likely to
achieve high impact and recognition. While the outcome of a risky strategy has
a higher expected reward than the outcome of a conservative strategy, the
additional reward is insufficient to compensate for the additional risk. By
studying the winners of 137 different prizes in biomedicine and chemistry, we
show that the occasional "gamble" for extraordinary impact is the most
plausible explanation for observed levels of risk-taking. Our empirical
demonstration and unpacking of the "essential tension" suggests policy
interventions that may foster more innovative research.

Physics and Society;Digital Libraries;Social and Information Networks;Applications