April 2006 101
E N T E R T A I N M E N T C O M P U T I N G
Mobile device user interfaces
play a crucial role in facilitating a
pleasant and rich user experience.
M
obile phones, rapidlybecoming ubiquitous, now claim about 2 billion users worldwide. Popu-larity is transforming them from simple voice-communica-tion devices to advanced-communica-tion devices that provide voice, text,
andvideo messaging. Further, mobile phones now embody consumer-elec-tronics features such as built-in cam-eras, MP3 players, and video capture and play. Increasingly, computing func-tions such as e-mail, Web access, and spreadsheets have been added to the mobile phones’ repertoire.
The next generation of mobile sys-tems will operate on a core Internet Protocol network that supports all popular operating systems and soft-ware applications. Their high data rates and low latencies, together with their increasing capabilities, will help mobile phones facilitate the delivery of rich multimedia services such as video telephony; streaming news, sports, and movies; and multimedia messaging, Web browsing, and games. Statistics from the first quarter of 2005 show the use of mobile content
and applications in the US grew across all segments, with mobile games attracting 6.23 million users, photo messages 12.24 million, ring tone downloads 23.09 million, and text messaging 65.68 million. Analysts ex-pect this enormous growth to continue in the near future, with the worldwide mobile content market reaching $59 billion by 2009, according to Juniper Research (www.juniperresearch.com).
User interfaces for mobile devices play a crucial role in facilitating a pleasant and rich user experience, which makes the design of simple yet effective interfaces for mobile content a top priority.
MOBILE SYSTEMS LIMITATIONS
Several inherent limitations of mobile systems must be taken into account when developing and deploying con-tent and user interfaces.
The mobility constraints can be broadly classified as technology- or user-related. Technology-related con-straints can be further classified into communications- and device-related. Major communications-related limita-tions are higher error rates, more
dis-connections and dropped calls, higher noise, and relatively lower bandwidths.
Device-related constraints include limited resources such as modest pro-cessing power, relatively small mem-ory capacity, smaller display size, smaller-sized and fewer buttons, and limited battery power. Despite tech-nological advances, the need for mobile units to be small, light, and cheap will cause the device-related constraints to persist.
User-related constraints include lim-ited attention spans during mobility, changing locations and contexts, users’ idiosyncrasies, and personal preferences.
Mobile content UIs
The user interface must be designed to facilitate the widespread and increased use of mobile content. Current UI approaches in the com-puting domain—such as windows, icons, menus, and pointing (WIMP)— are inadequate and inappropriate for mobile applications.
Mobile device constraints pose sev-eral UI design challenges. UIs for emerging mobile devices consist of hardware and software combinations such as the following:
• additional buttons for text input and special functional keys, • touch-sensitive screens, • audio/visual interfaces, and • haptic interfaces to provide
virtual-reality experiences.
Generally, the UI should be intuitive and easy to use, as the immense popu-larity of Apple’s iPods and iTunes has shown. Simpler interfaces also conserve mobile devices’ precious battery power. For several mobile services, access requires a username and password, and possibly the user’s name, mailing address, and credit card information. For a typical application, navigating the different menus and entering the required information on a mobile phone keypad requires from 60 to 80 clicks. User profile information for per-sonalized services requires even more user input. Optimizing the menus and
User Interfaces
for Mobile Content
S.R. Subramanya
LGE Mobile Research
Byung K. Yi
102 Computer
play automatically adapts to the ver-tical or horizontal positioning of the handset. The Nokia 9300 provides a row of eight dedicated application buttons in addition to a QWERTY keyboard. The user interface includes a fully featured Web browser, a virtual mouse, and multiple windows. The user interface also provides a control panel for setting up and prioritizing network connections.
USER INTERFACES FOR 3G DEVICES
The emerging third-generation phones support many rich multimedia data applications and enhanced voice-related functions. Thus, their UIs must be designed for two broad aspects: data- and voice-related performance. Typical data applications include TV broadcasts, video streaming, music players, multimedia messaging, instant messaging, e-mail, Web brows-ing, mobile commerce, mobile multi-party gaming, mobile collaboration, and several location-based services.
Voice-related applications include voice-activated calls, avatar- and ges-ture-based displays during calls, and voice-memo-related functions. UIs for applications such as these, which pro-vide a unified experience, present a difficult design challenge.
User interfaces on the hardware front include enhanced keypads, spe-cial keys and joysticks for gaming, and interfaces to several integrated devices and to specialized external devices such as sensors and medical monitors. Given that, in the near future, mobile users are expected to produce consid-erable amounts of content, the user interface design must address two dis-tinct purposes: content production and consumption.
user navigations, and devising novel means of data input will result in a sig-nificant reduction in the number of clicks and battery power consumed.
Typical current-generation user interfaces
Most current-generation mobile phones have built-in cameras, MP3 players, and several personal orga-nizer applications—such as a calendar, scheduler, notepad, alarm clock, and calculator—with associated UIs. Some models support terrestrial digital mul-timedia broadcasting, enabling users to view TV programs while making telephone calls. As the number of built-in devices and applications expand on mobile devices, the provi-sion of appropriate UIs for these devices and applications becomes increasingly important.
For example, the Sony–Ericsson M600 uses an innovative design to provide a full QWERTY keyboard using no more keys than a standard mobile phone keypad. This eliminates the “multiple tappings” often required to input a single letter and facilitates easy messaging and e-mail. In addition to touchscreen and handwriting recognition, the M600 provides an interface that supports multiple tasks such as browsing the Internet while making a phone call. It also supports Word, Excel, and PowerPoint editors and an Adobe PDF viewer.
The LG SV360 has a one-million-polygon-per-second graphics accelera-tor chip that lets users play games with minute and realistic 3D graphics. The relatively larger 2.2-inch LCD screen provides a higher resolution for 3D games than most mobile phones can accommodate. The phone also has acceleration sensors that let users inter-act more closely and provide them with better controls and a more realistic feel, which can enhance game enjoyment.
The Panasonic P901iTV can receive terrestrial digital broadcasts and sup-ports services that combine mobile communications and broadcasts. For example, a user could visit a relevant Web site by clicking on the URL played during a broadcast. The
dis-UI design for content production
Content production tools should facilitate producing quality content on mobile devices as well as transferring content between mobile phones and other consumer electronics devices, PCs, and servers over several networks, such as Bluetooth, cellular, and WiFi. Also, tools for processing and manipulating the captured or received content must be developed. These tools should have simple user inter-faces that let users manipulate the mobile device’s resources, both hard-ware and softhard-ware, more effectively.
For example, the tools should enable the integrated use of resources such as the camera, scanner, voice recorder, optical character recognition, graphics, and animation generators. In addition, the UIs should support interoperabil-ity among the different resources and applications. The UI should, for exam-ple, easily import a scanned copy of a business card into a user’s contact list. The picture taken from the camera should also be easy to process and import or export onto relevant appli-cations, such as a photo-album-man-agement application.
UI design for content consumption
At the consumption end, UIs should cater to users’ special mobility needs. For example, given mobile users’ lim-ited attention spans, the UI should have pronounced auditory and visual cues, especially during hands-free operations.
Mobile users’ needs for content type and presentation modes change fre-quently during mobile operations. These users prefer to have control over several content presentation parameters based on time and place, situation, interest level, and cost con-siderations. Thus, their presentation parameters include the following:
• level of content detail, such as full-length or summaries;
• presentation quality, including res-olution, color versus gray scale, and frame rate; and
E N T E R T A I N M E N T C O M P U T I N G
User interface design
must address two
distinct purposes:
content production
• presentation-style elements such as sound effects, screen layouts, and visual effects.
Mobile users also want control over different proportions of media such as audio, video, animations, graphics, and text, based on mood, personal taste, mobility situations, and cost factors. The same content must be presented with different media combinations to suit different mobility situations. For example, the content might need to be rendered in audio-only form while dri-ving, it could be in audio and video while walking, and it could include text in addition to audio and video while stationary. Thus, user interfaces and techniques that adapt to different mobility situations and render the appropriate media combinations best suited to the current mobility scenario should be developed.
Design for interactivity
Unlike broadcast content, mobile content would increasingly be on-demand in most situations, with high interactivity. Consumers will have fre-quent and meaningful interactions with the content provider system for a variety of purposes, such as
• looking up and interacting with content and service directories; • querying for content of interest; • requesting content at certain times,
from various locations, and onto specific devices;
• sending delivery-control signals to invoke commands such as browse, play, stop, fast-forward, pause, and rewind;
• providing feedback on programs’ content and perceptual quality; • setting or updating user
prefer-ences; and
• following up on advertisements.
For mobile users, these interactions should be faster, with many fewer key-strokes and, sometimes, with voice commands.
Design for immersiveness
The availability of powerful
graph-ics coprocessors, digital signal proces-sors, and 3D displays can facilitate 3D visual effects. In addition, acceleration sensors, touch-screen interfaces, and haptic interfaces would enhance immersiveness in games and virtual-reality applications. User interfaces must be designed to take these devel-opments into account so that they can support premier communications ser-vices, mobile collaboration, gaming, and VR applications.
OTHER DESIGN CONSIDERATIONS
The Web’s many and varied content repositories, the emergence of mobile Web access, and the variety of loca-tion-based services that depend on it underscore the need for effective and efficient Web interfaces. Further, users increasingly treat mobile phones as inherently personal devices, as the more than 2,000 different phone models and the immense popularity of ring tones, ring-back tones, and wallpapers show. Designing and pro-viding customizable interfaces has thus become critical when catering to individual tastes and preferences.
The design of optimal navigation and uniformity across several menus, within the same application as well as across different applications, must also be addressed. Developers thus face the challenge of determining the sets of UI components, and their fea-tures and benefits, to satisfy different user needs. Further, developing con-tent consumption models and deter-mining how they relate to UI design will require serious study.
User interface design poses several challenges such as handling several on-device and external devices and supporting a variety of voice and mul-timedia data applications while being fast, simple to use, and customizable. Coping with these challenges will require several advances in the tools for UI design and development.
Currently, developers use the User Interface Markup Language (UIML) for static UI elements (structures) and their properties (styles). The VoiceXML standard has been
devel-oped for speech-based dialogs. The Synchronized Multimedia Integration Language (SMIL) provides a means for describing the spatial and timed appearance of complex multimedia content combinations. Further work in the development of platform-inde-pendent markup languages that sup-port the multimodal interactions expected in future mobile devices is still needed, however.
A
nalysts expect that third-genera-tion mobile phones and beyond will be conduits for enormous amounts of multimedia content. Several major issues along a variety of dimensions must be addressed in the design and provision of content to these mobile devices. Because they are crucial for enhancing the user experi-ence and enabling the effective con-sumption of mobile content, user interfaces are therefore among the major issues that need further study and work. ■S. R. Subramanyais a senior research scientist at LGE Mobile Research. Con-tact him at subra@lge.com.
Byung K. Yiis a senior executive vice president at LG Electronics. Contact him at bkyi@lge.com.
April 2006 103 Editor: Michael Macedonia,
