Memory

Design Implications

4.2.3 Memory

Memory involves recalling various kinds of knowledge that allow people to act appropri- ately. For example, it allows them to recognize someone’s face, remember someone’s name, recall when they last met them, and know what they said to them last.

It is not possible for us to remember everything that we see, hear, taste, smell, or touch, nor would we want to, as our brains would get overloaded. A filtering process is used to decide what information gets further processed and memorized. This filtering process, how- ever, is not without its problems. Often, we forget things that we would like to remember and conversely remember things that we would like to forget. For example, we may find it difficult to remember everyday things, like people’s names, or scientific knowledge such as mathematical formulae. On the other hand, we may effortlessly remember trivia or tunes that cycle endlessly through our heads.

How does this filtering process work? Initially, encoding takes place, determining which information is paid attention to in the environment and how it is interpreted. The extent to which it takes place affects people’s ability to recall that information later.

The more attention that is paid to something and the more it is processed in terms of thinking about it and comparing it with other knowledge, the more likely it is to be remembered. For example, when learning about a topic, it is much better to reflect on it, carry out exercises, have discussions with others about it, and write notes rather than pas- sively reading a book or watching a video about it. Thus, how information is interpreted when it is encountered greatly affects how it is represented in memory and how easy it is to retrieve subsequently.

Another factor that affects the extent to which information can be subsequently retrieved is the context in which it is encoded. One outcome is that sometimes it can be difficult for people to recall information that was encoded in a different context from the one in which they are at present. Consider the following scenario:

You are on a train and someone comes up to you and says hello. You don’t recognize this person for a few moments, but then you realize it is one of your neighbors. You are only used to seeing them in the hallway of your apartment building and seeing them out of context makes this person initially difficult to recognize.

Another well-known memory phenomenon is that people are much better at recognizing things than recalling things. Furthermore, certain kinds of information is easier to recognize than others. In particular, people are good at recognizing thousands of pictures even if they have only seen them briefly before. In contrast, people are not as good at remembering details about the things they photograph when visiting places, such as museums. It seems that they remember less about objects when they have photographed them than when they observe them with the naked eye (Henkel, 2014). The reason for this is that the study participants appeared to be focusing more on framing the photo and less on the details of the object being photographed. Consequently, people don’t process as much information about an object when taking photos of it compared with when they are actually looking at it; hence, they are unable to remember as much about it later.

Increasingly, people rely on the Internet and their smartphones to act as cognitive pros- theses. Smartphones with Internet access have become an indispensable extension of the mind. Sparrow et al. (2011) showed how expecting to have readily available Internet access reduces the need and hence the extent to which people attempt to remember the information itself, while enhancing their memory for knowing where to find it online. Many people will whip out a smartphone to find out who acted in a movie, the name of a book, or what year a pop song was first released, and so on. Besides search engines, there are a number of other cognitive prosthetic apps that instantly help people find out or remember something, such as Shazam.com, the popular music recognition app.

4.2.3.1 Personal Information Management

The number of documents written, images created, music files recorded, videoclips down- loaded, emails with attachments saved, URLs bookmarked, and so on, increases every day. A common practice is for people to store these files on a phone, on a computer, or in the cloud with a view to accessing them later. This is known as personal information management (PIM). The design challenge here is deciding which is the best way of helping users organize their content so that it can be easily searched, for example, via folders, albums, or lists. The solution should help users readily access specific items at a later date, for example, a par- ticular image, video, or document. This can be difficult, however, especially when there are thousands or hundreds of thousands of pieces of information available. How does someone find that photo they took of their dog spectacularly jumping into the sea to chase a seagull, which they believe was taken two or three years ago? It can take them ages to wade through the hundreds of folders they have catalogued by date, name, or tag. Do they start by homing in on folders for a given year, looking for events, places, or faces, or typing in a search term to find the specific photo?

ACTIVITY 4.2

Try to remember the birthdays of all the members of your family and closest friends. How many can you remember? Then try to describe the image/graphic of the latest app you downloaded.

Comment

It is likely that you remembered the image, the colors, and the name of the app you down- loaded much better than the birthdays of your family and friends—most people now rely on Facebook or other online app to remind them about such special dates. People are good at remembering visual cues about things, for example, the color of items, the location of objects (for example, a book being on the top shelf), and marks on an object (like a scratch on a watch, a chip on a cup, and so on). In contrast, people find other kinds of information per- sistently difficult to learn and remember, especially arbitrary material like phone numbers.

4 . 2 W h A T I S C O G N I T I O N ? 113

It can become frustrating if an item is not easy to locate, especially when users have to spend lots of time opening numerous folders when searching for a particular image or an old document, simply because they can’t remember what they called it or where they stored it.

How can we improve upon this cognitive process of remembering?

Naming is the most common means of encoding content, but trying to remember a name someone created some time back can be difficult, especially if they have tens of thousands of named files, images, videos, emails, and so forth. How might such a process be facilitated, considering individual memory abilities? Ofer Bergman and Steve Whittaker (2016) have proposed a model for helping people manage their “digital stuff” based on curation. The model involves three interdependent processes: how to decide what personal information to keep, how to organize that information when storing it, and which strategies to use to retrieve it later. The first stage can be assisted by the system they use. For example, email, texts, music, and photos are stored as default by many devices. Users have to decide whether to place these in folders or delete them. In contrast, when browsing the web, they have to make a conscious decision as to whether a site they are visiting is worth bookmarking as one they might want to revisit later.

A number of ways of adding metadata to documents have been developed, includ- ing time stamping, categorizing, tagging, and attribution (for example color, text, icon, sound, or image). Surprisingly, however, the majority of people still prefer the old- fashioned way of using folders for holding their files and other digital content. One reason is that folders provide a powerful metaphor (see Chapter  3, “Conceptualizing Interaction”) that people can readily understand—placing things that have something in common into a container.

A folder that is often seen on many users’ desktop is one simply labeled “stuff.” This is where documents, images, and so forth, that don’t have an obvious place to go are often placed but that people still want to keep somewhere. It has also been found that there is a strong preference for scanning across and within folders when looking for something rather than simply typing a term into a search engine (Ofer and Whittaker, 2016). Part of the problem with using search engines is that it can be difficult to recall the name of the file someone is seeking. This process requires more cognitive effort than navigating through a set of folders.

To help users with searching, a number of search and find tools, such as Apple’s Spot- light, now enable them to type a partial name or even the first letter of a file that it then searches for throughout the entire system, including the content inside documents, apps, games, emails, contacts, images, calendars, and applications. Figure 4.4 shows a partial list of files that Spotlight matched to the word cognition, categorized in terms of documents, mail and text messages, PDF documents, and so on.

4.2.3.2 Memory Load and Passwords

Phone, online, and mobile banking allow customers to carry out financial transactions, such as paying bills and checking the balance of their accounts, at their convenience. One of the problems confronting banks that provide these capabilities, however, is how to manage secu- rity concerns, especially preventing fraudulent transactions.

Figure 4.4 Apple’s Spotlight search tool

BOX 4.1

The Problem with the Magical Number Seven, Plus or Minus Two

Perhaps the best-known finding in psychology (certainly the one that nearly all students remember many years after they have finished their studies) is George Miller’s (1956) theory that seven, plus or minus two, chunks of information can be held in short-term memory at any one time. However, it is also one that has been misapplied in interaction design because several designers assume that it means they should design user interfaces only to have seven, plus or minus two, widgets on a screen, such as menus. In fact, however, this is a misapplication of the phenomenon, as explained here.

4 . 2 W h A T I S C O G N I T I O N ? 115

By short-term memory, Miller meant a memory store in which information was assumed to be processed when first perceived. By chunks of information, Miller meant a range of items such as numbers, letters, or words. According to Miller’s theory, therefore, people’s immediate mem- ory capacity is very limited. They are able to remember only a few words or numbers that they have heard or seen. If you are not familiar with this phenomenon, try the following exercise:

Read the first set of numbers here (or get someone to read them to you), cover it up, and then try to recall as many of the items as possible. Repeat this for the other sets.

• 3, 12, 6, 20, 9, 4, 0, 1, 19, 8, 97, 13, 84

• cat, house, paper, laugh, people, red, yes, number, shadow, broom, rain, plant, lamp, chocolate, radio, one, coin, jet

• t, k, s, y, r, q, x, p, a, z, l, b, m, e

How many did you correctly remember for each set? Between five and nine, as suggested by Miller’s theory?

Chunks of information can also be combined items that are meaningful. For example, it is possible to remember the same number of two-word phrases like hot chocolate, banana split, cream cracker, rock music, cheddar cheese, leather belt, laser printer, tree fern, fluffy duckling, or cold rain. When these are all jumbled up (that is, split belt, fern crackers, banana laser, printer cream, cheddar tree, rain duckling, or hot rock), however, it is much harder to remem- ber as many chunks. This is mainly because the first set contains all meaningful two-word phrases that have been heard before and that require less time to be processed in short-term memory, whereas the second set is made up of completely novel phrases that don’t exist in the real world. You need to spend time linking the two parts of the phrase together while trying to memorize them. This takes more time and effort to achieve. Of course, it is possible to do if you have time to spend rehearsing them, but if you are asked to do it having heard them only once in quick succession, it is most likely that you will remember only a few.

So, how might people’s ability to remember only 7 ± 2 chunks of information that they have just read or heard be usefully applied to interaction design? According to a survey by Bob Bailey (2000), several designers have been led to believe the following guidelines and have created interfaces based on them:

• Have only seven options on a menu.

• Display only seven icons on a menu bar.

• Never have more than seven bullets in a list.

• Place only seven tabs at the top of a website page.

• Place only seven items on a pull-down menu.

He points out how this is not how the principle should be applied. The reason is that these are all items that can be scanned and rescanned visually and hence do not have to be recalled from short-term memory. They don’t just flash up on the screen and disappear, requiring the user to remember them before deciding which one to select. If you were asked to find an item of food most people crave in the set of single words listed earlier, would you have any problem? No, you would just scan the list until you recognized the one (chocolate) that matched the task and then select it—just as people do when interacting with menus, lists, and tabs, regardless of whether they consist of three or 30 items. What users are required to do here is not remember as many items as possible, having only heard or seen them once in a sequence, but instead scan through a set of items until they recognize the one they want. This is a quite different task.

One solution has been to develop rigorous security measures whereby customers must provide multiple pieces of information before gaining access to their accounts. This is called multifactor authentication (MFA). The method requires a user to provide two or more pieces of evidence that only they know, such as the following:

• Their ZIP code or postal code

• Their mother’s maiden name

• Their birthplace

• The last school they attended

• The first school they attended

• A password of between five and ten letters

• A memorable address (not their home)

• A memorable date (not their birthday)

Many of these are relatively easy to remember and recall since they are familiar to the specific user. But consider the last two. How easy is it for someone to come up with such memorable information and then be able to recall it readily? Perhaps the customer can give the address and birthday of another member of their family as a memorable address and date. But what about the request for a password? Suppose a customer selects the word inter- action as a password—fairly easy to remember, yes? The problem is that banks do not ask for the full password because of the danger that someone in the vicinity might overhear or oversee. Instead, they ask the customer to provide specific letters or numbers from it, like the seventh followed by the fifth. Certainly, such information does not spring readily to mind.

Instead, it requires mentally counting each alphanumeric character of the password until the desired one is reached. How long does it take you to determine the seventh letter of the password interaction? How did you do it?

To make things harder, banks also randomize the questions they ask. Again, this is to pre- vent someone else who is nearby from memorizing the sequence of information. However, it also means that the customers themselves cannot learn the sequence of information required, meaning that they have to generate different information each time.

This requirement to remember and recall such information puts a big memory load on cus- tomers. Some people find such a procedure quite nerve-racking and are prone to forget certain pieces of information. As a coping strategy, they write down their details on a sheet of paper.

Having such an external representation at hand makes it much easier for them to read off the necessary information rather than having to recall it from memory. However, it also makes them vulnerable to the fraud the banks are trying to prevent should anyone else get ahold of that piece of paper! Software companies have also developed password managers to help reduce memory load. An example is LastPass (https://www.lastpass.com/), which is designed to remember all of your passwords, meaning that you only have to remember one master password.

ACTIVITY 4.3

How can banks overcome the problem of providing a secure system while making the mem- ory load easier for people wanting to use online and mobile phone banking?

4 . 2 W h A T I S C O G N I T I O N ? 117

Much research has been conducted into how to design technology to help people suffer- ing from memory loss (for instance those with Alzheimer’s disease). An early example was the SenseCam, which was originally developed by Microsoft Research Labs in Cambridge (UK) to enable people to remember everyday events. The device they developed was a wearable camera that intermittently took photos, without any user intervention, while it was worn (see Figure 4.5). The camera could be set to take pictures at particular times, for example, every 30 seconds, or based on what it sensed (for example, acceleration). The camera employed a fish-eye lens, enabling nearly everything in front of the wearer to be captured. The digital images for each day were stored, providing a record of the events that a person experienced.

Several studies were conducted on patients with various forms of memory loss using the device. For example, Steve Hodges et al. (2006) describe how a patient, Mrs. B, who had amnesia

Comment

Advances in computer vision and biometrics technology means that it is now possible to replace the need for passwords to be typed in each time. For example, facial and touch ID can be configured on newer smartphones to enable password-free mobile banking. Once these are set up, a user simply needs to put their face in front of their phone’s camera or their finger on the fingerprint sensor. These alternative approaches put the onus on the phone to recognize and authenticate the person rather than the person having to learn and remember a password.

BOX 4.2

Digital Forgetting

Much of the research on memory and interaction design has focused on developing cogni- tive aids that help people to remember, for example, reminders, to-do lists, and digital photo collections. However, there are times when we want to forget a memory. For example, when someone breaks up with their partner, it can be emotionally painful to be reminded of them through shared digital images, videos, and Facebook friends. How can technology be designed to help people forget such memories? How could social media, such as Facebook, be designed to support this process?

Corina Sas and Steve Whittaker (2013) suggest designing new ways of harvesting digital materials connected to a broken relationship through using various automatic methods, such as facial recognition, which dispose of them without the person needing to go through them personally and be confronted with painful memories. They also suggest that during a separa- tion, people could create a collage of their digital content connected to the ex, so as to trans- form them into something more abstract, thereby providing a means for closure and helping with the process of moving on.