Fast and Reliable Emotions Detection Adapted for Driver Monitoring and Online Psychotherapy Sessions

Abstract

In this paper, we present a solution for human face monitoring, which can be used in multiple scenarios. The presented solution monitors how a person felt throughout the whole therapy session, what was relaxing to talk about, what made him or her angry or disgusted. Thus, psychotherapists may acquire more data about their patients, in addition to what they already collected. Another use case is monitoring a car driver, based on their emotions and blinking patterns, to ensure that the driver is in a suitable state. This paper presents a method to assess the feelings a person has, in the domain of the five primary emotions: happiness, sadness, surprise, anger, and disgust. Besides emotions, our model is capable of monitoring how tired a person is, by monitoring their eyes and blinking patterns. To ensure that a high detection rate is performed, a machine learning approach based on a convolutional network was employed, backed up by a solid training phase performed onto a considerable set of tagged visual information. The proposed method compares favorably against other state-of-the-art emotion detection solutions for the proposed scenario and its performance is validated using a bespoke online psychotherapy image dataset acquired using low-end webcams with CMOS sensors. The results proved that the proposed solution is both fast and dependable, thus being currently used with strong results in a real-world platform for supporting psychologists in their remote counseling real-time sessions.