  We present an adapted single-shot convolutional neural network (YOLOv2) for
the real-time localization and classification of particles in optical
microscopy. As compared to previous works, we focus on the real-time detection
capabilities of the system to allow for manipulation of microscopic objects in
large heterogeneous ensembles with the help of feedback control. The network is
capable of localizing and classifying several hundreds of microscopic objects
even at very low signal-to-noise ratios for images as large as 416x416 pixels
with an inference time of about 10 ms. We demonstrate the real-time detection
performance by manipulating active particles propelled by laser-induced
self-thermophoresis. In order to make our framework readily available for
others, we provide all scripts and source code. The network is implemented in
Python/Keras using the TensorFlow backend. A C library supporting GPUs is
provided for the real-time inference.

Soft Condensed Matter;Machine Learning;Image and Video Processing;Optics