Probabilistic Programming for Anomaly Detection
Some of the leading challenges in anomaly
detection are:<p>(1) Goal.  What is the goal?  One goal is
for near <i>real-time</i> monitoring for
<i>health and wellness</i> and, then, detection
ASAP. This goal can be appropriate in
monitoring on-line systems, say, server
farms, digital communications networks,
process plants.  For much of medicine,
there is more time for detection.<p>(2) Old Data.  What is assumed about the
input data, that is, the <i>old</i> or
<i>historical</i> data?  One leading assumption
is that the data is a second order
stationary stochastic process.  Another
leading assumption is that the data is
independent, identically distributed.<p>(3) Anomalies.  There are essentially two
important cases about the anomalies.
First, we have seen the anomalies before,
have some good data characterizing them,
and now are just looking for those old
anomalies in new data.  An example here
is looking for <i>signatures</i> of malware.<p>Second, we have not seen the anomalies
before and are trying to detect them for
the first time, that is, the <i>zero-day</i>
case.<p>(4) Real-Time Assumption.  If doing
real-time monitoring, likely want to
assume that the real-time data is
probabilistically the same as the <i>old</i>,
<i>historical</i> data.<p>(5) False Alarm Rate.  Once the <i>context</i>
of the work as in (1)-(4) is clear,
likely the most important goal is to be
able to select a desired false alarm rate
and get that rate in practice.<p>(6) Detection Rate.  The next major
challenge is, for whatever false alarm
rate are willing to tolerate, how to get
the highest possible detection rate?  Of
course, we might like to achieve the
highest possible detection rate in the
sense of the classic Neyman-Pearson
result, e.g., in<p><a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=11440459" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=11440459</a><p>(7) Nominal versus Interval Data.  Another
challenge is how to handle data that is
<i>nominal</i> (i.e., with just a few values
known in advance) or <i>interval</i>, i.e.,
essentially any real numbers.<p>(8) Multi-Dimensional Data.  Another
challenge is how to get a higher detection
rate by jointly exploiting data on several
variables.<p>(9) Distribution-Free.  Another challenge
is how to handle situations where there is
little or no chance of knowing the joint
probability distribution of data on
several variables.  E.g., it is easy to
get input data on 20 variables, but
getting data enough on 20 variables to
have an accurate description of the joint
probability distribution stands to be
challenging, say, 1000^20 data points or
some such.  So, can want techniques that
are <i>distribution-free</i>, that is, that
make no assumptions about probability
distributions.<p>Anomaly detection is often close to
<i>hypothesis testing</i> in statistics, and
that is a large and mature field with a
lot of good work.