| Usually a person recognizes the content of a picture or video in the first 500 milliseconds of looking at it. Visual distortions can hinder this process, leading to annoyance when videos are consumed for entertainment, or, more seriously, leading to interpretation mistakes in professional applications such as video-based security or image-based medical diagnostics. The holy grail in imaging technology is to create systems that ensure their output to have optimal visual quality, as appreciated by their users. I propose to use semantics to lay the theoretical foundations for a new generation of visual quality optimization tools. So far visual quality optimization has been based on algorithms that assume that modeling the visibility of distortions is sufficient to quantify their annoyance. My expertise in the use of Machine Learning tools to emulate perceived quality based on distortion visibility led me to the conviction that this approach is somewhat simplistic, since it neglects the influence that content and semantics have on quality preferences. When distortions hinder the content recognition process, the accuracy of the visual analysis decreases, thus causing annoyance. This decrease can be critical, depending on the semantics of the content depicted in the image. Hence, the new challenge is to model visual quality appreciation with a semantic-aware approach. In this project I propose to (1) study how distortions hinder content recognition through empirical research, (2) capture the relationship between specific semantic categories and visual annoyance and (3) produce a semantic-aware quality assessment tool with the aid of machine learning. The resulting theoretical insights and practical tools will improve systems that assess, preserve and repair the visual quality of digital media. To support the utilization of this research, the image/video databases and related psychophysical data collected in the project will be made publicly available. |
