SEWA Project

The Automatic Sentiment Analysis in the Wild (SEWA) is a EC H2020 funded project. The main aim of SEWA is to deploy and capitalise on existing state-of-the-art methodologies, models and algorithms for machine analysis of facial, vocal and verbal behaviour, and then adjust and combine them to realise naturalistic human-centric human-computer interaction (HCI) and computer-mediated face-to-face interaction (FF-HCI).

This will involve development of computer vision, speech processing and machine learning tools for automated understanding of human interactive behaviour in naturalistic contexts. The envisioned technology will be based on findings in cognitive sciences and it will represent a set of audio and visual spatiotemporal methods for automatic analysis of human spontaneous (as opposed to posed and exaggerated) patterns of behavioural cues including continuous and discrete analysis of sentiment, liking and empathy.

SEWA will draw on expertise from several disciplines as illustrated in the table below:

Expertise ICL UP RealEyes PlayGen
Image processing Yes Yes
Speech recognition Yes
Audio processing Yes Yes
Online learning Yes Yes Yes
Unsupervised learning Yes Yes
User studies Yes Yes Yes Yes
Multimodal learning Yes Yes
Multimodal recommendation Yes Yes
Social signals and social games Yes Yes
Audio-visual database design Yes
Ethical studies Yes Yes

Why SEWA?

Technologies that can robustly and accurately analyse human facial, vocal and verbal behaviour and interactions in the wild, as observed by omnipresent webcams in digital devices, would have profound impact on both basic sciences and the industrial sector...

Read more

How SEWA works?

SEWA uses state-of-the-art methodologies, models and algorithms for machine analysis of facial, vocal and verbal behaviour, and then adjust them to realise naturalistic human-centric human-computer interaction (HCI) and computer-mediated face-to-face interaction...

Read more

Where is SEWA applied?

As a proof of concept, and with the focus on novel HCI and FF-HCI applications, SEWA technology will be applied to machine inference of sentiment/ liking ratings in response to multimedia content (movie trailers, product adverts, etc.) watched by people in the wild...

Read more