Computer Science Honours 2011
Auditory interaction
Vision‐based interaction l d l
Multimodal interaction
Ubiquitous computing
d l l l
Augmented Reality / Virtual Reality
Mobile computing
Not covered: olfactory interfaces, taste f
interfaces
d l b d l
Unidirectional or bidirectional communication
Used often in daily life
ll d
Visually impaired
Eyes‐busy interfaces (GPS, SMS reader)
I i i
Interactive voice response systems
G ( dt k t ti )
Games (soundtrack to convey emotion)
Medical monitoring data (hearing patterns in sound)
Ambient displays (waterfall for network traffic)
Ambient displays (waterfall for network traffic)
What is it?
input to the computer is through a camera
image processed to recognise input
What kind of input?
motion
gesture recognition
laser pointer
sign language
pointing gestures f l
facial expression
recognition
lti l d liti t d t t t
multiple modalities presented to computer at a time
two or more inputs to an interfacep
example: driving a car…
tactile is al
visual
audio
scent?
assumed to make interaction more intuitive
can you think of any task that we do in real world where only interact with one modality?
where only interact with one modality?
l d l
First multimodal system of its kind
Combine voice and Combine voice and
gesture inputs to provide a “natural” user
modality.
“Put‐That‐There”
Nearly all multimodal
Nearly all multimodal systems since have used same combination of voice and gesture
Computing embedded physically and psychologically into movements,
interactions.
Example: electric motors
f
Computing woven into the backdrop of human interactions
“We are trying to conceive a new way of
h k b h k
thinking about computers, one that takes into account the human world and allows the
h l h h
computers themselves to vanish into the
background”
http://mediacup teco edu http://mediacup.teco.edu
http://www.intel.com/pressroom/archive/releases/20070403corp.htm
http://www.sonycsl.co.jp/person/rekimoto/as/
AR: computer generated world + real world
VR: computer generated world
Computer
Generated Graphics Virtual
World
Display
Graphics
Camera Display
Camera
Real World
h d d
First head mounted display
High resolution screen
High resolution screen to provide ambient
information to pilot p
“Grandfather of virtual
reality”
needs the least explanation needs the least explanation
using technology when moving.
typically cell phone PDA based computing
typically cell phone, PDA based computing
includes less known fields like wearable computing
computing
t f ti b “i h f t d d”
part of computing by “inch, foot and yard”
project by Xerox PARC
aims to enrich our computing environment by p g y emphasizing context sensitivity, casual
interaction and the spatial arrangement of computers
computers
Three characteristics differentiate a tab and its applications from traditional desktop:
P bili ll f f l i h
Portability: very small form factor, low‐weight
Communication: low‐latency interaction between users and system y
Context‐sensitive operation
lf d d l d
Olfactory: generate and deliver odours to user
Taste interface?
Haptics biting on different surfaces
Haptics: biting on different surfaces
Sound, vibration, chemical taste
Phili i j k Philips emotion jacket
For tomorrow:
d l f l h
Find an example of an actual project that
investigates non traditional inputs.
