  Most publicly available image quality databases have been created under
highly controlled conditions by introducing graded simulated distortions onto
high-quality photographs. However, images captured using typical real-world
mobile camera devices are usually afflicted by complex mixtures of multiple
distortions, which are not necessarily well-modeled by the synthetic
distortions found in existing databases. The originators of existing legacy
databases usually conducted human psychometric studies to obtain statistically
meaningful sets of human opinion scores on images in a stringently controlled
visual environment, resulting in small data collections relative to other kinds
of image analysis databases. Towards overcoming these limitations, we designed
and created a new database that we call the LIVE In the Wild Image Quality
Challenge Database, which contains widely diverse authentic image distortions
on a large number of images captured using a representative variety of modern
mobile devices. We also designed and implemented a new online crowdsourcing
system, which we have used to conduct a very large-scale, multi-month image
quality assessment subjective study. Our database consists of over 350000
opinion scores on 1162 images evaluated by over 7000 unique human observers.
Despite the lack of control over the experimental environments of the numerous
study participants, we demonstrate excellent internal consistency of the
subjective dataset. We also evaluate several top-performing blind Image Quality
Assessment algorithms on it and present insights on how mixtures of distortions
challenge both end users as well as automatic perceptual quality prediction
models.

Computer Vision and Pattern Recognition