text: Maximum entropy methods are at the core of a new view of scientific inference, allowing analysis and interpretation of large and sometimes noisy data. Surprisal analysis extends principles of maximal entropy and of thermodynamics, where both equilibrium thermodynamics and statistical mechanics are assumed to be inferential processes. This enables surprisal analysis to be an effective method of information quantification and compaction and of providing an unbiased characterization of systems. Surprisal analysis is particularly useful to characterize and understand dynamics in small systems, where energy fluxes that are otherwise negligible in large systems heavily influence system behavior.
  Foremost, surprisal analysis identifies the state of a system when it reaches its maximal entropy, or thermodynamic equilibrium. This is known as balance state of the system because once a system reaches its maximal entropy, it can no longer initiate or participate in spontaneous processes. Following the determination of the balanced state, surprisal analysis then characterizes all the states in which the system deviates away from the balance state. These deviations are caused by constraints; these constraints on the system prevent the system from reaching its maximal entropy. Surprisal analysis is applied to both identify and characterize these constraints. In terms of the constraints, the probability  of an event  is quantified by
  Here  is the probability of the event  in the balanced state. It is usually called the “prior probability” because it is the probability of an event  prior to any constraints. The surprisal itself is defined as
  The surprisal equals the sum over the constraints and is a measure of the deviation from the balanced state. These deviations are ranked on the degree of deviation from the balance state and ordered on the most to least influential to the system. This ranking is provided through the use of Lagrange multipliers. The most important constraint and usually the constraint sufficient to characterize a system exhibit the largest Lagrange multiplier. The multiplier for constraint  is denoted above as ; larger multipliers identify more influential constraints. The event variable  is the value of the constraint  for the event . Using the method of Lagrange multipliers requires that the prior probability  and the nature of the constraints be experimentally identified. A numerical algorithm for determining Lagrange multipliers has been introduced by Agmon et al. Recently, singular value decomposition and principal component analysis of the surprisal was utilized to identify constraints on biological systems, extending surprisal analysis to better understanding biological dynamics as shown in the figure.