[Reading] Using Human Physiology to Evaluate Subtle Expressivity of a Virtual Quizmaster in a Mathematical Game

http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6WGR-4F4WYNR-1&_coverDate=02%2F01%2F2005&_alid=516189552&_rdoc=1&_fmt=&_orig=search&_qd=1&_cdi=6829&_sort=d&view=c&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=c9c7e4550c1e39c2d84e639d3e50adcd

Helmut Prendinger and Junichiro Mori and Mitsuru Ishizuka
year 2003.

Abstraction: The aim of the experimental study described in this article is to investigate the effect of a life-like character with subtle expressivity on the affective state of users. The character acts as a quizmaster in the context of a mathematical game. This application was chosen as a simple, and for the sake of the experiment, highly controllable, instance of human–computer interfaces and software. Subtle expressivity refers to the character's affective response to the user's performance by emulating multimodal human–human communicative behavior such as different body gestures and varying linguistic style. The impact of em-pathic behavior, which is a special form of affective response, is examined by deliberately frustrating the user during the game progress. There are two novel aspects in this investigation. First, we employ an animated interface agent to address the affective state of users rather than a text-based interface, which has been used in related research. Second, while previous empirical studies rely on questionnaires to evaluate the effect of life-like characters, we utilize physiological information of users (in addition to questionnaire data) in order to precisely associate the occurrence of interface events with users’ autonomic nervous system activity. The results of our study indicate that empathic character response can significantly decrease user stress and that affective behavior may have a positive effect on users’ perception of the difficulty of a task.

Keyword: Life-like characters; Affective behavior; Empathy; Physiological user information; Evaluation

==== After Read ====
The writers use physiological signal to evaluate user interface and interaction between human and computer game. Their primary hypothesis is that if a life-like character provides affective feedback to the user, it can effectively reduce user frustration and stress. They use bio-sensors including GSR and BVP. Another short questionnaire is requested. The feature they get from the sensor signal is mean. The easy game is summing up the given five numbers. The result shows that the hypothesis is held expect the relation between game score and empathy.

The feature they get is very simple. They use a few sensors to show the result is good. I think the write wants to tell us that we can use just a few sensors the justify the helpful of well-design user interface. Some more application can be derived from the work.

[Reading] XPOD - A Human Activity and Emotion Awake Mobile Music Player

http://ebiquity.umbc.edu/paper/html/id/280/

XPod - a human activity and emotion aware mobile music player

Authors: Sandor Dornbush, Kevin Fisher, Kyle McKay, Alex Prikhodko, and Zary Segall

Book Title: Proceedings of the International Conference on Mobile Technology, Applications and Systems

Date: November 17, 2005

Abstract: In this paper, we consider the notion of collecting human emotion and activity information from the user, and explore how this information could be used to improve the user experience with mobile music players. This paper proposes a mobile MP3 player, XPod, which is able to automate the process of selecting the song best suited to the emotion and the current activity of the user. The XPod concept is based on the idea of automating much of the interaction between the music player and its user. The XPod project introduces a "smart" music player that learns its user's preferences, emotions and activity, and tailors its music selections accordingly. The device is able to monitor a number of external variables to determine its user's levels of activity, motion and physical states to make an accurate model of the task its user is undertaking at the moment and predict the genre of music would be appropriate. The XPod relies on its user to train the player as to what music is preferred and under what conditions. After an initial training period, the XPod is able to use its internal algorithms to make an educated selection of the song that would best fit its user's emotion and situation. We use the data gathered from a streaming version of the BodyMedia SenseWear to detect different levels of user activity and emotion. After determining the state of the user the neural network engine compares the user's current state, time, and activity levels to past user song preferences matching the existing set of conditions and makes a musical selection. The XPod system was trained to play different music based on the user’s activity level. A simple pattern was used so the state dependant customization could be verified. XPod successfully learned the pattern of listening behavior exhibited by the test user. As the training proceeded the XPod learned the desired behavior and chose music to match the preferences of the test user. XPod automates the process of choosing music best suited for a user’s current activity. The success of the initial implementation of XPod concepts provides the basis for further exploration of human- and emotion-aware mobile music players.

==== After Reading ====
It is a prototype application. They collect acceleration, galvanic skin response(GSR), skin temperature, heart flow, and near body temperature as input data. By using fully connected neural network, the system predict whether the user will skip this song. It looks like the product recently provide by Nike and Adidas.
But I think that the claim of song decision by emotion detecting is a little weak. Mostly they did is putting the bio-sensor signals into NN. They considered the changing of the signals. They did not explain how to consider the emotion and how they inference the result.
Moreover, I think some bio signals can help the recognition of activity, especially the `actively'.

[Reading] A Study in Users' Physiological Response to an Empathic Interface Agent

Helmut Prendinger, Christian Becker, and Mitsuru Ishizuka.
A study in users' physiological response to an empathic interface agent. [pdf]
International Journal of Humanoid Robotics, Vol. 3, No. 3, Sept. 2006, pp 371-391.
http://www.worldscinet.com/191/03/0303/S0219843606000801.html


Game: Skip-Bo
Emotion recognition: SC, EMG, & game sate(time) as input, 2-axes (arousal & valence) as output. (ref29)
Method: ANOVA
Subject: 32 (14m, 18f)
Hypothesis:

• If the virtual game opponent behaves ``naturally'' in that if follows its own goals and expresses associated positively or negatively valenced affective behaviors, users will be less aroused or stressed than when the agent does not do so.
• If the game opponent is oriented only toward its own goals and displays associated behaviors, users will be less aroused or stressed than when the agent does not express any emotion at all.
  

Agent's behavior:

• non-emotional
• self-centered emotional
• negative empathic
• positive empathic
  

Agent's abilities: (in PAD space)

• auditory speech
• facial
• body gesture

Result:

• The positive empathic condition was experienced as significantly more arousaling or stressful than the negative empathic condition.
  
• Users seemingly do not respond significantly different when empahtic agent behavior is absent.
  

Notes:

• EMG high, negative valence more
• global baseline -> for individual differences
  

[Reading] Emotion Analyzing Method Using Physiological State

Mera, Kazuya and Ichimura, Takumi

Book Series	Lecture Notes in Computer Science
Publisher	Springer Berlin / Heidelberg
ISSN	0302-9743
Subject	Computer Science
Volume	Volume 3214/2004
Book	Knowledge-Based Intelligent Information and Engineering Systems
DOI	10.1007/b100910
Copyright	2004
ISBN	978-3-540-23206-3
Pages	195-201

1. using uttrance in dialoge to fine the emotion (by ECG)
2. using the method proposed by R. Picard to recognize the emotion (extrating 20 features)
3. using NN to combine 1 + 2

Condiser 20 emotions which are classified into 6 emotion groups :

• Well-Being: joy, distress
• Fortunes-of-Others: happy-for, gloating, resentment, sorry-for
• Prospect-based: hpe, fear
• Confirmation: satisfaction, relief, fears-confirmed, disappointment
• Attribution: pride, admiration, shame, disliking
• Well-Being/Attribution: gratitude, anger, gratification, remorse
  

A method combine context and physiological signal processing to detect the clients emotion.

[Readings] Emotion recognition form physiological signals using wireless sensors for presence techonologies

Journal	Cognition, Technology & Work
Publisher	Springer London
ISSN	1435-5558 (Print) 1435-5566 (Online)
Subject	Computer Science and Engineering
Issue	Volume 6, Number 1 / February, 2004
Category	Original Article
DOI	10.1007/s10111-003-0143-x
Pages	4-14
Online Date	Thursday, February 19, 2004

Fatma Nasoz¹ , Kaye Alvarez², Christine L. Lisetti¹ and Neal Finkelstein

subject: 29
elicitation method: movie
method: kNN(71%), DFA(74%), MBP(83%)
elicited emotioin: sadness anger surprise fear frustration amusement (by 1995 Gross Levenson)
Measure: GSR, HR, tempreture
Data saving: 3-dimensional array, real number (see in page 12)

[readings] Multimodal Emotion Recognition & Expressivit Analysis

Multimedia and Expo, 2005. ICME 2005. IEEE International Conference on

Kollias, S. Karpouzis, K.


The paper present the framwork of multimodal emotion recognition and expressivity analysis. First, it introduces technologis that involve in emotion reocognition, such as speech, visaul, physiological signal processing, etc. Then, it mentions the important issues in emotional speech analysis, visual analysis, ECA expressivity, and physiological signal analysis in sections. Finally, some key problem identified for integration of sigle-mode emotion analysis techniques are listed.
I think this is a high level paper which shows up and lists the problem in the field of emotion recognition. The cited papers are distribute in three fields, which are speech, facial expression, and physiological signal. I will read more and part 3, the physiological signal process with emotion recognition.

[Readings]Evolving Pet Robot with Emotional Model

Kubota, N. Nojima, Y. Baba, N. Kojima, F. Fukuda, T.

This paper appears in: Evolutionary Computation, 2000. Proceedings of the 2000 Congress on
Publication Date: 2000
Volume: 2, On page(s): 1231-1237 vol.2
ISBN: 0-7803-6375-2
References Cited: 23
Digital Object Identifier: 10.1109/CEC.2000.870791

In daily, many pet robots have been existed. Writers show a pet robot with emotion model. They discribe the details in the paper. First, they have some sensors in the robot. People can interact with the robot via the sensors. The sensors' signal inputs feed into emotional model which is desing by themself. (The model include emotion, mood, and feelings.) Robots are express their emotion by actions such as moving, dancing, and sounding. Moreover, they use potential field to represent a trick. They try to teach the robot the ``trick''. This is the step of behavioral learning. Finally, they use GA to optimal fuzzy controllers and path planning for mobile robots.

In this paper, everything the robot feels is defined by human. I think the writers model the robot's behavior with emotional concept. But all the thing about the model is predefine without animal phsychology. It really work, but it is pre-define. That is another side of emotional using. I am not focusing this part now.

[Readings] Aritficail Emotion and Its Recognition, Modeling and Applications: An Overview

This paper really an overview. The writers using many reference to discribe the whole picture of artificial emotion. They using the view of medical science. But in the book I recently read, it is just one part of theory in the whole emotion psychology. Moreover, they discribe differenct emotion recognition methods. In the emotion modeling, analytic and symbolic method and botom-up method are both mentioned.

I think this is really an overview paper and no technical details inside. What I should do is to read the reference and to find the recognition method.

By the way, it is really strange that the papar is writen in simplify Chinese and published in Proceeding of the 5th World Cngress on Intelligent Control and Automatin. The conference was held in China. I think this is the reason why it is writen in Chinese.