2.1 Intra-Hand Inputs
2.1.2 Design Factors in Current Sensing Approaches
al. [56] demonstrated how the combination of hand tracking by vision sensing and touch track- ing by the touch sensing on the ring device can be promising for supporting high-precision and low-fatigue interaction by complementing each other. In this manner, future input system de- sign should consider the sensing technique combination to better support the gesture sensing algorithm and reduce the error [86].
Eye-tracking: Eye-tracking system is being employed in the latest HMD devices thanks to the current high-fidelity cameras. For example, Microsoft HoloLens21 and HTC Vive pro eye2 are utilizing this built-on system for object selection [90] or manipulation of eye movements and blink for VR avatar3. These applications leverage the characteristics of the eye, which is always available and very rapid. However, eye-tracking is vulnerable to disturbances in mobile contexts [91] [Accessibility] and enables very limited interactions – blink and gaze[Space]. To deal with this other input techniques such as dwell [92, 93] or other input modalities such as a clicker, speech, or hand-tracking should be combined [74].
Head gaze: Head gaze is used in most HMD devices for its simplicity and higher stability than eye-tracking. Similar to eye tracking, these devices support always available and easy to use inputs. However, for the same reason of eye-tracking, it is vulnerable to mobile context [Accessibility] and has a limited input space[Space].
Hand tracking: High-end HMDs utilize hand movements, poses, or gestures as an input.
As this input modality can detect the movement of the hand and high DoF of the finger motions through the camera attached to the device, it can provide suitable input set for diverse applica- tions, such as text entry [94], games [54], and 3D modeling [28] [Space]. However, long-term use of mid-air gesture causes fatigue (i.e., gorilla arm syndrome) [95] and the gestures or finger motions may be occluded by other body regions if the hand is not well positioned [39, 84][Accessibility].
Moreover, in a social situation, mid-air gestures may attract a lot of undue attention from the public [Social].
Speech: Advances in speech recognition technology enables hands-free use of wearables[Accessibility].
In particular, by acquiring the intention of the user from the command, it can provide much sim- ple and rapid use of applications for certain tasks such as make an alarm or play specific music which conventional touch input requires several steps of manipulation [Manipulation]. However, simple tasks involving continuous actions, such as scrolling or zooming in/out, may take longer
1https://www.microsoft.com/en-us/hololens/hardware
2https://www.vive.com/eu/product/vive-pro-eye/overview/
3https://www.vive.com/eu/product/vive-pro-eye/features/
than touch actions as the speech control is typically discrete [Manipulation]. Moreover, it is still difficult to use in public because of its social acceptance [Social] and privacy issues [10].
Controller: A controller is generally used by VR devices in a static and pre-set environ- ment [96]. Their precise orientation and positional tracking through the camera (on HMD or externally pre-installed) and built-in motion sensor are capable to make rapid and accurate 3Dmotion inputs, and by being combined with multiple buttons and touchpad on the controllers, they provide much wider input space for diverse applications such as games, 3D modeling, and teleconferences [Space]. However, their usage is limited to a specific environment and it is cum- bersome to bring the controllers around [Accessibility].
Motion based control: Built-in motion sensors installed in most wearable devices enable body motion tracking that support head-gaze interface on the HMD or hand gesture detection in smart-watches. In particular, its high sampling rate and low-power support increase the use of motion control in smartwatches for the activation gesture (Raise arm), wrist gestures (flick in/out), and finger gestures (flick or tap). Motions running on the one-hand only provide higher accessibility than the touch interface (which requires two hands [3]) [Accessibility] and rapid inputs [Manipulation]. In this case, the input set design is particularly important as the large- scale motion gestures can lower the social acceptability and inconvenient gestures can limit the long-term use [Social].
There is no technology that completely satisfies all design factors. Rather, these interfaces are often combined each other. For example, in HMD, head gaze (or eye-tracking) is used for pointing while hand tracking or controller is used for selecting and dragging. Likewise, a smart watch uses a touch interface for main input modality while simple inputs are done by motion control, and text inputs are done by speech. These multimodal inputs complement one another to enables much wider input space and convenient uses.
In addition, this holistic view of current approaches arises further discussions. The decoupling trends between input location and the output feedback location [42] are becoming more dominant due to the small input surface of the wearable devices and their scattering around the body.
Body area network [7] and image recognition technology [97] also support this trend by fully
utilizing the sensor data from each body region or detect the body motions from the camera and thus this input trend can provide a wider input space compared to conventional touch inputs.
These can be even more extended with the development of new sensors or techniques. However, such a decoupling may suffer from difficulties to make an easy-to-understand mapping between user action and the input [98].
On the other hand, although wearables are a new form-factor of computing that is always on the body, there is a lack of considerations for the use of the human body itself in the current input modalities [99]. In the research community, there have been long attempts to utilize the body as an input and output platform [45]. These extra input dimensions can open up new interaction opportunities by transforming whole body areas into a touch-enabled surface. In this case, the proximate body area from wearables can be utilized as the input surface, which makes it easier to build easy-to-understand mappings. These areas may include hand, back of the hand, arm, or face which proximate to the smartwatches or smartglass.