Emotions are an essential constituent of well-being. They can be recognized using contact-free sensors such as cameras, based on facial expressions and physiological parameters, such as changes in temperature. We conducted an early evaluation of emotion recognition from RGB cameras using two datasets and high-lighted challenges such as subject-specific relationship between facial expressions and emotions, as well as inconsistent expressions during the same emotional state. Additionally we confirmed the feasibility of measuring subtle changes in temperature between facial regions correlating to different emotional states, using a thermal camera. Finally we proposed ideas for future improvements relating to transfer learning and cross-dataset data curation, which could allow for improvements in performance leading towards practical implementation of well-being monitoring.