UX Foundations: Interaction Design

Course by David Hogue.

Today’s Interaction Designer

Interaction Design was introduced by Bill Moggride and Bill Verplank in the 1980s.

Project types and deliverables

It’s about designing for the entire interconnected system, the device, the interface, the context, the environment, and the people.

Technical and software skills

Starting from pencil and paper or a whiteboard, to understand, define, and frame the problem.

To low-fidelity diagrams, models, and flows, that helps us identify potential directions, missing information, and paths the project may take.

After that, we need to translate the ideas, from concepts in papers, to pixels on screens.

Even more, by making pixels alive, we should put interactive prototypes in use, in the hands of the people who will be using the product to validate our ideas.

Interaction designers have knowledge and skills across several fields, but we often go deep in a few specific areas. This allows us to understand people and problems from different perspectives and give us the capabilities in specific areas to solve those problems.

Any knowledge in the area, at the end, helps us to understand People, Technology, and Design.

Resources and communities

There are tons of opportunities, learn, just learn.

A Model of Psychology and Interaction Design

Why psychology?

Understanding how people feel, think, understand the information, and make decisions, can help us generate better ideas and solutions and help us craft and deliver better experiences.

The core model is:

Discover (study and insights) [this course]
Craft (design and prototype) [this course]
Deliver (develop and measure)
Improve (iterate and optimize)

Discover

Identify (research about) the problem to be solved or need to be met.

By researching we need to:

Measure (objective, quantitative data)
Observe (subjective data that may be qualitative or quantitative)
Ask (self-report data that may be qualitative or quantitative)

Define, describe the problem or need and the desired outcome or goal.

Frame, place the solution and outcome in the larger context of the person and the product or service.

Craft

Design from an abducting reasoning, critical thinking, and problem-solving perspective.

Prototype, evaluate if our design decisions are working, and identify alternatives if not.

Iterate, designing and prototyping are iterative processes, we generate, evaluate, and adjust ideas repeatedly to identify the best solution.

Introducing the Interaction Design model

CONTEXT
└> GOALS
   ├> PRODUCTIVITY
   └> CREATIVITY & PLAY
      ├> PERCEPTION 
      ├> MOTIVATION
      ├> COGNITION
      └> EMOTION
         └> BEHAVIOR

Context

Refers to the circumstances, situation and environment in which a person is interacting with a product.

Who?
- Description
- Experience
- Expectations
- Individual or group
When and Where?
- Location
- Time
- Public vs private
- Focus
What and How?
- Objects, devices, and tools
- Purpose
- Conjunction
- Alternatives
- Efficacy

Goals

What the people are trying to achieve. The desired outcomes.

It’s important to define the type of activity. Productivity vs Play, what we may feel obligated to perform, or may feel desire to engage in creative play (the “I need to” vs “I want to”).

Describing the Goals and Desired Outcomes:

Objectives
- Outcomes
- Achievements
- End states
Importance
- Value
- Essential/necessary
- Optional/elective
Urgency
- Priority
- Critical/compelling
- Timing

Introductory Psychology

Sensation

Most important senses interaction focuses on are: vision, hearing, touch, and proprioception.

What the user sees and how, keeping in mind possible user visual deficiencies, what the user hears when something happens, nothing or some sound signals, how the user squeezes or drags an element, the device vibrates or not, how the elements on the UI are distributed and perceived.

Perception

Is the fact of processing and recognizing the information we get through the senses.

Environmental conditions can change the way we perceive things, but we still manage to do that with great consistency due to the brain “magic”.

Shape, lightness, color, distance, size, location, and timbre are some of these conditions.

Gestalt principles

Starting in 1910, Max Wertheimer, a German psychologist, noticed that a series of blinking lights creates the illusion of motion. The Phi Effect.

Gestalt means “shape and form”.
We perceive the world as meaningful and complete objects, not a series of independent parts.

Gestalt Laws

Figure-ground: how we organize our perceptions in terms of foreground objects, the clearly defined ones, and background objects, which may be unbounded or vaguely defined.
Simplicity: how we organize our perceptions into the simplest possible experience.
Proximity: objects near one to other in space or time, are perceived as being a group.
Similarity: objects with similar characteristics, such as form, color, size, and brightness, are perceived as belonging together.
Common fate: how objects moving together are perceived as belonging together.
And others like closure, continuity, and symmetry.

Affordances

Physical characteristics of objects make possible what we are able to do.
We recognize when we are able to interact.
Affordances are relative to the individual and their ability to interact.
It doesn’t require conscious attention and thought.
It arises from our intrinsic understanding of our physical relationship with the world around us.

Don Norman introduced the concept of the perceived affordance. When people perceive similarity between a digital representation and an actual physical object.

We might use common depth cues, like:

Shadows
Perspective
Overlap / Layers
Blur
Scale

Motivation

We need to understand both the user’s goals and needs. As we know what actually drives people to do things, we can describe context and goals more efficiently:

People
Situation
Needs
Objectives
Importance
Urgency

Theories of Motivation

Locus of control

External - Extrinsic motivation: when people’s behavior is influenced by external factors, usually things they need/are obligated to, or ends in rewards or earnings.
Internal - Intrinsic motivation: when people do something because they want to, because it’s personally rewarding.

David McClelland described motivation as

Achievement: problem solving and learning.
Affiliation: social relationships.
Power: influencing others.

Victor Vroom - Expectancy theory

Expectancy: “If I put forth the effort necessary, then I will perform well”.
Instrumentality: “If I perform well, then I will receive an outcome”.
Valence: “If I value the outcome, then I will be motivated to put forth the effort necessary”.

ERG theory

Existence: basic survival and safety.
Relatedness: establishing and maintaining interpersonal relationships.
Growth: intrinsic desire for personal development and self-esteem.

Attention and Memory

Attention

Attention is a limited resource, we cannot pay attention to everything, so we focus only on what we think is important.

As the designers, we need to provide structure, hierarchy and guide attention to what is truly important and necessary.

Memory

Our ability to store, recall, and recognize information.
We expect it to work and only notice when it fails.
We depend on our devices to store information for us to extend our memory.

Three stages of memory

Sensory store: Very brief snapshot of incoming sensory information; most of it will be lost.
Working: Short-duration and limited capacity memory for active thinking and processing.
Long-term: Long duration and very large capacity, this is where we store information for the future.

Declarative:
- Things we know (knowledge).
- Semantic - facts.
- Episodic - events.
- Easy to state and describe information we know.
Procedural
- Things we can do (skills).
- Easier to show how to perform an action or task than to describe how to do it.

Encoding information

Schemas

Memory structures that organize information for meaning and associations.

Assimilation

New information similar to what we already know gets added to existing schema.

Accommodation

New information unlike anything we already know requires new schemas.

How we retrieve information

Recall

What we usually think of as memory: we store information and retrieve it when we need it.

Recognition

It’s making a selection from a set of choices by comparing what we perceive with what we remember.

Recognition is easier than recall

Show information and functionality when it is needed.
Put information and functionality where people expect to find it.
Present information in meaningful ways.

Visual memory is powerful

Memory for images is better than our memory for words.
Images can represent complex concepts, easier and faster than words.

Pattern recognition

The ability to see patterns and to make connections between similar and related things.

Reasoning and logic

Problem-solving

It’s identifying a solution and defining the steps necessary to achieve it.

Methods:

Trial-and-error
Difference reduction (comparing)
Subgoal decomposition
Analogy
Transformation
Algorithms and heuristics

Decision-making

It’s making the best choice from a set of options for a given context or situation.

Methods:

Rational
Optimal
Satisficing
Pain vs. Value

Our knowledge, experience, understanding, and expectations all contribute to deciding what we will do.

Mental Models

They are our thoughts and expectations about how things work in the real world. They influence our behavior, decisions, and attitudes.

Concept formation

Taking information, breaking it into meaningful pieces, and sorting them into general rules.
Associations, schemas, and concepts come to form mental models.

Types of Models

Mental model

Prior experiences and expectations of how a product should work in the user’s mind.

Concept model

Prior experiences, plans, and design intents for how a product should work in the creator’s mind.

System model

How the product actually works.

Good mental models simplify

Exposing the system nearly always complicates the product experience.
We must understand how something really works (system model) so that we can design a product (concept model) that is easy for the people to understand and use (mental model).

The effort of thinking

Cognitive Load

It refers to how much effort we put into understanding information, making decisions, and solving problems.

Cognitive Friction

It occurs when people have to put in more than they should and are therefore slowed down.

Information overload

Attention and working memory are limited.
Exceeding our cognitive capacity can lead to forgetting, misunderstanding, and confusion.
Products should do as much of the work as possible for the people who use them.

Not all cognitive load is bad, but it can cause cognitive friction.

Reducing Cognitive Load

Base smart defaults on context, usage patterns, and past behavior.
Only use smart defaults when they are likely to be correct.
Make smart defaults easy to change.

Reducing Cognitive Friction

Understand the mental models.
Provide meaningful information.
Keep people focused on the outcome or goal.
Simplify the interactions.
Provide help and instructions.

Theories of Emotion

Emotions are subjective and difficult to define and measure, they may vary by individual context. The absence of negative experiences is not enough to create positive emotions.

Designing for positive experiences

Increase the frequency and duration of positive emotional experiences. Decrease or avoid the negative emotional experiences.

Classifying emotions

Paul Ekman classifies by six basic and universal emotions:

Anger
Disgust
Fear
Sadness
Happiness
Surprise

Designing for delight

Surprise only works once.

A continuous stream of surprising new things is unsustainable.

Levels of Delight (Tollady and Rowe)

Surface delight comes from the characteristics of the product we perceive and experience.
Deep delight emerges when products are frictionless, effortless, easy, and meaningful.
Strive to create products that are deeply delightful.

Creating Delight (Dana Chisnell)

Pleasure arises from humor, uniqueness, personality, beauty, simplicity, and the ability to anticipate needs.
Flow arises from ease of use, efficiency, immersiveness, automation, and when people feel empowered.
Meaning creates delight with purpose, value, connectedness, relevance, and personal engagement with compassion and care.

Hierarchy of product quality (Aaron Walter)

Functional
Reliable & Dependable
Ease of Use
Pleasurable (Delight)

Making something functional and reliable is not enough to be “ready to use”. We should think about its ease of use and the rewarding/pleasurable part before launching it.

Incorporating Delight

Delight is many things, and it must be part of the design process.
It cannot be added at the end.
Delight (and positive emotional experiences in general) should be on par with functionality, reliability, and usability.

Empathy

”Empathy is the ability to understand and identify with another person’s context, emotions, goals, and motivations.” - Whitney Quesemberry

You are not designing for yourself

Do not assume that people have the same needs or will use a product the same way we would.
We know much more about the product, the technology, and the business needs, therefore we are biased.
It is easy to fall into the trap of designing for ourselves.

Fostering Empathy

Spend time with the people who use your product.
Develop good listening and observation skills.
Collect objective data about how your product is actually being used.
Remove yourself from the product decisions.
Practice with colleagues and help each other improve.
Employ user-centered design methods.

Learning Behaviors

Behavior and Interaction Design

Focus and the signals and cues that indicate certain behaviors may be performed.
Internal states, motivations, and emotions are difficult to detect and measure, so we observe behavior to try to understand needs and goals.

Multiple Theories of Behavior

We’ll focus on behaviorist theories, because interactions are observable and measurable.
These theories can provide models for predicting what people may do and why they interact with a product.

Basic Terminology

Stimulus is a signal.
Response is a behavior.
Consequence is an outcome.
Conditioning is learning.
Association is connecting.

Three Theories of Learned Behavior

Classical conditioning: we learn to associate experiences that occur together in space and/or time. Operant (or Instrumental) conditioning: we learn to behave to seek reward and avoid punishment. Observational (or Social) learning: we learn by observing others then imitating them.

Classical Conditioning

Start with a behavior that already occurs in response to a signal.
Pair a new signal with the existing signal.
Over time and with repeated experience, the new signal becomes associated with the original signal, and the new signal elicits the same behavior.

Conditional Emotional Responses

Some emotional reactions are learned responses to neutral signals. These are called Conditional Emotional Responses.
Neutral signals paired with stimuli that cause an emotion can elicit that emotion once the neutral signal and original stimulus are associated.

Operant Conditioning

A stimulus (or signal) indicates that a behavior may be performed.
Once the behavior is performed, an outcome (or consequence) results.

Reinforcement vs Punishment

Outcomes that are good, desirable, and rewarding reinforce the behavior and increase the frequency of that behavior.
Outcomes that are bar or undesirable punish the behavior and decrease the frequency of that behavior.

Frequency of Reinforcement

New behaviors can be learned quickly if they are reinforced immediately after every occurrence.
In real life, most reinforcement is not this predictable and consistent.
Reinforcement is still effective even if it is not delivered for every behavior.

Reinforcement can be scheduled as something happens a certain amount of times or the amount of time since the last time something happens.

I.e.:

Reinforce some behavior every ten times it occurs, or about two hours since the last time behavior has been reinforced.

Rewards have different values

Nor all rewards have the same value for everyone.
What reinforces behavior in one person may not reinforce the same behavior in another.

Selecting Rewards

Attention, recognition, and praise are social reinforcers.
Money, credits, and points are financial reinforcers.

Extinction

Behaviors which are no longer reinforced then fade.
The more frequent and regular the reinforcement, the faster the behavior will fade when reinforcement is withdrawn.

For Albert Bandura, Learning:

Is a cognitive process
Occurs in social context.
May occur even without performing the behaviors or being directly reinforced.

In addition to behaviors, we also learn values and attitudes.

Modeling and Imitation

When we observe other people behaving, they serve as models that we can then imitate.
Seeing other people being reinforced makes us more likely to perform that behavior.
Seeing other people being punished makes us less likely to perform that behavior.

Learning behavior is an active process

We may learn from watching others or the environment around us and not show any changes in behavior.
Observation, cognition, and context all influence what we learn and when we choose to exhibit a behavior.

Three Types of Modeling (Bandura)

Live models: a person demonstrates the desired behavior, we observe directly, then we imitate their behavior.
Direct instruction: a person describes the desired behavior in detail and provides instructions to us about how to do it, then we perform the behavior as described.
Symbolic instruction: modeling occurs through media, webs, written and/or illustrated information, video, and audio. We view the media then perform the behavior as instructed.

Using learned behavior

Past experiences influence future behavior

What we learned in the past influences how we behave now and how we might behave in the future.
We look for ways to apply what we already know to new situations.
Transfer refers to our ability to take information and behaviors learned in one context and apply it in another.

Generalization

We generalize our past experiences and transfer what we have learned to similar situations.
If it even remotely looks like something we have done before, we will interact with it using those behaviors.
When experiences are identifiably analogous, people will generalize and transfer their knowledge more efficiently.
The less similar the situation appears, the more cognitive effort it takes to determine if a past learned behavior is relevant and should be applied.
Facilitate generalization by making it easier for people to identify analogous situations.

Discrimination learning

People also learn how to distinguish among situations or problems that may appear similar but are actually different.
Do not design a product that misleads people to predict it operates in a way it does not, or makes it look non-interactive.

The Interaction Design Model

The behavior on the interaction design model is the actual interaction with the product, and it has its own cycle.

... -->
--> PERCEIVE -->
--> PREDICT -->
--> INTERACT -->
--> FEEDBACK -->
--> LEARN -->
--> PRACTICE / REMEMBER -->
--> ...

Perceive

People are less likely to interact if they do not perceive the opportunity to interact, even if they need or want to interact.

People must understand there are things they can do.

Good design solutions increase the perceivability of opportunities to interact as well as inform people about the appropriate or correct ways to interact.

We bring our past experiences and learning to every new experience.

Perceivability

More than just a visual signal and cues.
All content and interactions must be accessible for all people.
Gestures and voice interaction often lack perceivable signals and cues, so people often do not know what they can do or say

Multiple possible interactions

MAny products offer many functions and services, but usually need only one a few per use.
We quickly process our options and select the most salient and relevant, and then ignore the rest.

Predict

Confidence

If people are confident that their interaction with a product will help them achieve their goals, then the probability of interaction increases.

Deciding among options

When there are different ways to complete a task, people decide based on:

Context
Goal
Prior experience
Perceivability
Confidence

Feedback

Three stages of feedback

Interaction acknowledged

An immediate response to a person’s interaction and request. The product acknowledges that they have done something.

Techniques:

Hovers, rollovers, and down states.
Transitions for size, rotation, and location.
Modal layers and dialogs appear or disappear.
Sound and vibration.

Request in progress

Interim feedback to indicate that the product is still working on the request and provides progress information.

Techniques:

Deactivate states with status messages.
Progress indicators for time, amount, or simply ongoing.

Results of interaction

The final feedback is the actual result or output of the product in response to a person’s request.

Final feedback forms and variations:

Navigate to a new page or screen.
Modal layers and dialogs.
Update content on the current page.
Control media playback.
Control game play.
Error message.

Ambient feedback

Subtle, continuous change to the interface that provides useful information at a glance.
It usually conveys information about a single variable in a simple way.

Anticipatory feedback

Information is provided before a person interacts to help them understate what the outcomes may be.
Improves the accuracy of predictions.
Helps people avoid interactions that are not likely to help them achieve their goals.

Making meaning of feedback

We put cognitive effort into making sense of feedback
Is the outcome meaningful?
Does the outcome match the prediction?
Does the outcome help the person get closer to the goal?

Value of outcomes

Meaningful outcomes that match predictions and move people toward their goals are more useful and valuable.
The probability of continued interaction with the product increases.

Feedback bring emotions

Useful outcomes and meaningful feedback create positive experiences, instill trust, and may even delight people.

Learn

Practice to learn

We observe our own behavior and the outcomes that occur.
Repeated interactions strengthen the associations and improve the accuracy of predictions.
The more we practice, the faster and more accurate our interactions with the product become.

Intuitive Design

Even when products are “intuitive”, they may require multiple uses and practice before people learn and remember how to interact.

Remember

Memory and remembering

Sensory Store: during the perceive phase we process all incoming information and identify the important elements.
Working Memory: during the feedback and learning phases we process information for meaning and associations.
Long-term Memory: during the remember phase we store learned information (and skills) for future use.

Transfer

We recognize products we have used before, may remember how to use them, and recall and ew-use what we learned.
We notice similarities between prior experiences and new products, transfer what we learned from earlier product experiences, and apply those skills and knowledge to the new products.

Sometimes, we can mismatch, confuse, or even forget what we have learned.

Interaction Design

Can be approached as:

Dimensions of focus
Models of methods and processes
Guiding principles

Five Dimensions of IxD

By Gillian Crampton Smith and Kevin Silver.

Words (content): should be easy to understand and written to effectively communicate information.
Visual representations: diagrams, illustrations, graphics, and even typographic styles and media to clarify data and concepts.
Physical objects or space: physical hardware, such as keyboard, mouse, or smartphone, and the space necessary to use it effectively.
Time: the time during which people interact with the product and how they might measure their progress.
Behavior: actions, reactions, and emotions people have when interacting with a product.

Design Thinking

Exploratory and iterative methods are effective when we are facing problems that are vague or not understood.

We need to:

Understand people and problems.
Change assumptions.
Generate solutions.

Different methods helps us to:

Diverge: is the process of generating multiple ideas.
Converge: is the process of choosing the best of those ideas.

Design Thinking methods

Observe and empathize.
Explore and create.
Build and deliver.
Iterate and improve.

Double Diamond

Problem definition:
Problem --> Discover --> Define

Design brief:
Solution --> Develop --> Deliver

Core Design Principles

Understand people and their context.
Define a product’s purpose based on people’s needs and goals.
Craft experiences for efficient flow.
Deliver meaningful, relevant, and valuable experiences.
Ensure products are easy to use, learn, and remember.
Always respect and protect the people who use our products and services.

Levels of design

Structure

It focuses on the overall organization and architecture of the product. We identify and define the systematic structure of content and functionality.

Scent of Information

It’s derived from information foraging theory, this is how people can predict what they will find if they follow a path or flow through a product.

Three levels of Structure

Product: overall architecture defines all of their sections, the purpose of each, and the navigation and connections among them.
Screen: what belongs on each page or screen and where should it appear? What appears on all multiple screens? Placement can communicate importance, relationships, and functionality.
Component: define the content and interactivity of individual elements on the screen. We are not yet focused on final appearance, because we first need to validate that the structure and flow are correct.

Flow

Here we identify and define all of the steps people must take and the states of the product as they work toward their goal.

Three levels of Flow

Multi-channel ecosystem: many products and services are used in conjunction with others. How does the product fit into a larger ecosystem? What are the paths through the product and the states of the product?
Product: identify the paths through the product and all of the steps and actions necessary to complete tasks. There is at least one path for each function. There may be more than one path to achieve the same goal.
Components: identify and describe the various states of interactivity screens and components. Define the conditions that produce each and the outcomes that may result.

Simple flows

Simple tasks tend to be brief, direct, and linear.

Complex flows

Complex tasks tend to have multiple decision points, branching, and repeat actions, and have multiple states contingent upon context and prior interactions or choices.

Nested flows

Flows may also be nested within other flows.
There may be sub-steps, multiple paths, interactions across products and channels, and multi-session interactions.
Design to effectively communicate place, state, and progress.

Interface

At this level, we focus on appearance and behavior, but this is also the stage where we add the polish, nuance, and refinement that can differentiate a product.

More than Brand and Identity

Appearance and behavior can:

Clarify structure and flow.
Clarify and enhance content
Guide through a path
Enhance feedback
Help people identify opportunities to transfer and apply their prior experiences.

Appearance, Meaning, and value

Good design enhances the value and quality of a product.
Bad design detracts from or even interferes with a product experience.
People judge the quality of a product higher and think it works better when it is attractive.

Patterns

Design patterns

Design patterns are recurring, re-usable, optimal solutions to common design problems.

There are DP for structure, flow, and interface.

Good DP are:

Familiar, recognizable, and easy to apply correctly.
Optimal, effective, and efficient.
Relevant in the context and to the goals.
Effortless and easy to use with minimal work.

Good design patterns improve consistency and cohesiveness, which makes products easier to use, learn, and remember.

Anti-patterns

Anti-patterns are recurring solutions that may be common and familiar, but they are non-optimal in practice.

They are non-optimal, but it doesn’t mean they are bad or worse than a pattern. They are just exceptions.

Dark patterns

Dark patterns intentionally mislead people into making choices and performing actions they would not have otherwise chosen to do and which are not in their best interests.

Harry Brignull studied and identified multiple types of dark patterns, such as:

Bait and switch
Disguised ads
Forced continuity
Friend spam
Hidden costs
Misdirection
Price comparison prevention
Privacy Zuckering
Roach motel
Sneak into basket
Trick questions

Dark patterns are associated with violations of privacy, theft of money and information, and the installation of malware.

Features that can counter a dark pattern from a product:

Perceivable / Findable.
Honest / Transparent.
Notify / Clarify.
Comparison / Confirmation.
Reversible / Usable.

Navigation is the movement from place to place or state to state within a product.

Information architecture:

Information architecture as a field and practice is attributed to Richard Saul Wurman.
It is typically involved in creating the structure of a navigation system.
Good information architecture is often considered essential for good interaction design.

Mental models we have about:

How information should be organized.
Flows and the sequence of steps it will take to perform an action.
Tasks organized into a logical sequence of steps (that will be easier to complete).

Types of navigation systems:

Hierarchical: used when there are multiple levels, or hierarchies, in the structure of the content and functionality.
Hub and spoke: there is a central hub from which to explore and browse. There are often several independent sections, and each may have different content or functions.
Adaptive: navigation adjusts dynamically to the context, content, and how it is being used.
- Individual access privileges.
- Suggested actions based on context.
- Valid actions based on section or content.
Steam: used when there is a steady stream of new content or posts, presented in reverse chronological order, may be algorithms to promote sponsored or popular content.
Search sort, and filter: effective when there are vast amounts of content, whether or not it is well-organized.
Interactive or gestural: used when navigation elements are revealed after an interaction, often lack signals or cues, so they may be difficult to discover. We shall use hints and cues to indicate that actions are available.
Content link: links in the content of a page or screen are a form of contextually relevant navigation.
Hybrid: when mixing different types of navigation into a single system.

Findability vs Discoverability

Findability is when a person knows in advance that something is or should be within the product. Discoverability is when a person does not know in advance that something exists and is available within that product.

Content

Content strategy

It guides the creation, publication, and maintenance of meaningful, relevant content.
Content creation is an essential part of the design process.
Content takes many forms.

Design with text

Readability: is the ease with which a reader can understand written text. We need to use appropriate vocabulary and syntax based on the context of the product.
Scanning: organize text to facilitate scanning to help people focus their attention and find what they need.
Typography: optimize font size, line height, and line length. Also text styles are used to differentiate among text types, headers, labels, body content, and links.
Interactivity: expose only the necessary information to keep the product focused and simpler. Extensive content may benefit from interactive layouts that conceal and reveal supplemental information as needed.

Microcopy

Small pieces of text found throughout the interface, important for clarity and guidance. Such as buttons, labels, status and progress indicators.

Localization

Language.
Numeric, date, and time formats.
Use of currency.
Keyboard usage.
Collation and sorting.
Symbols, icons, and colors.
Culturally appropriate text and graphics.
Varying legal requirements, among others.

Imagery

Images have high information density. They can convey much in a small amount of space.
Enable pan and zoom for exploration of large, highly detailed images.

Interactive images and media

Highly interactive content experiences can be very engaging.
Interactions should support and enhance the information and improve comprehension.
Design interactive media with the same principles used to design products.

Media controls

Auto-play should be used only when requested and expected.
When not requested, auto-play distracts and interrupts.
Removing control over auto-playing media is an undesirable and negative experience.

Inputs

Bi-directional data flow

Interaction is bi-directional. People add and receive information.
Data can be entered many ways through input devices and sensors.
Data may be gathered without explicit interaction. (Such as GPS location or ambient brightness).

Interaction gap

The more distance or time between the point of interaction and the result or reaction, the more difficult it may be to control.
Experiences where people “interact here for outcomes over there” may be more difficult to predict and learn.

Direct interaction

It’s easier to learn, control, and predict.
Strive to make interactions as direct and immediate as possible.

Keyboards

Typing has been around for 150 years and is a familiar interaction.
On-screen keyboards replicate the familiar layout of keys, but the interactions are more direct, because the interaction gap is reduced.

Mice and trackballs

Distance from and orientation relative to the screen may differ for these input devices.
Use and interactions have become more complex, because people need more control over the objects on their screens.
Both require learning and practice to become proficient.

Pen tablets

The surface of the tablet is a mirror of the coordinate system on the screen.
Use and interaction can be more difficult to learn and master, because it requires precise spatial and motor control.

Clicking and tapping

The most familiar and frequent interactions with easily recognized outcomes.
Tapping on screens is more direct than using a mouse.

Signals and cues

Anything may be tapped on a touchscreen, so we need to provide signals about which elements are actually interactive.
Depth cues, such as shadows, layers, and textures, can make on-screen objects look like they can be touched.

Target size

Small and/or faraway targets are harder to reach and tap quickly and accurately. Instead, use larger and/or near targets.

Gestures

Gestures leave no trace.
Accidental gestures may go unnoticed and may be difficult to undo.
Hidden, complex, and arbitrary gestures are difficult to learn and remember.
Use gestures consistently and in familiar ways whenever possible.

Common gestures:

Tap
Double-tap
Long or hard press
Scroll
Pan
Drag
Swipe
Flick
Pinch
Spear
Rotate
Multi-touch

Device gestures:

Orient
Shake
Flip

Signals and cues for gestures

Buttons and icons are perceivable, but gestures are invisible.
Previews, hints, and motion can be used to indicate that gestural interaction is possible.
Demos and instructions can show how to gesture.

Arbitrary and abstract gestures

Multi-touch gestures may be powerful, but they can be difficult to discover.
Multi-touch gestures vary across brands and products, which makes learning, generalization, and transfer of experience even more difficult.

Voice

Voice recognition and speech synthesis have been available for decades. It is still evolving and improving. People are very good at understanding context, reference, intent, and non-verbal communication, computers are not -yet-.

We need to design to help people learn what to say and how to say it while we wait for technology to catch up to our expectations.

Principles for Voice Interaction

Reveal: when voice interaction is available and ready.
Guide: what people can say and the actions the product can perform for them.
Repeat: repeat the request and provide options and examples when asking for clarification.
Tell: state the request when delivering results or performing actions on their behalf.
Respect: be polite, friendly, concise, considerate, and unobstructive.

Sensors

Sensors gather data from the environment around the device.

They are:

Camera
Microphone
Photometer
Accelerometer
Gyroscope
Compass
Thermometer
Barometer
Altimeter
Magnetometer
Proximity

And more.

Uses for Sensor Data

Context may be better understood with location and environmental data.
Monitor health and fitness.
Learn about nearby things with audio and image recognition.
Automatic device adjustments for improved usability.

Privacy and Security about on sensors

Data collected is often personal and may be sufficient to identify the person.
Be transparent about what is collected and how it is used.
Offer clear and findable privacy and security controls.

Microinteractions

Microinteractions are the small, nuanced reactions of a product that provide information and feedback.

Dan Saffer studied microinteractions extensively and identified four parts:

Triggers: signal or cue that something has happened or will happen.
Rules: determine the conditions under which microinteractions occur and define what will happen.
Feedback: verifies that the person has interacted with the product or that specific conditions have been met.
Loops and modes: some interactions repeat (loop) until the person stops it, others change the way a device works (mode) under certain conditions.

Motion

Our brains are wired to be sensitive to motion and change. It draws and guides our attention through the product.

Motion can also distract, stealing people’s attention is unwanted and unwelcome. Use motion judiciously and with purpose.

Motion has meaning

Time: animations and motion may communicate progress, transition, and status.
Space: motion can help people understand spatial location and hierarchy.
Relationship: connections, relationships, and associations can be clarified and highlighted.
Change: shape, size, appearance, and/or location may transform in proportion to changes in value or meaning.
Opportunity: indicate opportunities to interact, often with gestures, by mimicking the gesture with similar motion of the interactive elements on the screen.

Smooth Motion

Abrupt changes feel mechanical and may be disruptive.

Easing is used to create smooth motion with acceleration, deceleration, and other physical effects.

Motion that embodies characteristics of the natural world can be more appealing.

Sound

Sound is an important part of product design, but it can also distract or even annoy. It can enhance a product by providing information and contributing to positive emotions.

Sounds as icons

Auditory icons are sounds that correspond to something in the real world.

Earcons are arbitrary sounds we learn to associate with actions.

Sound has value and purpose

Direct attention: attract and guide a person’s attention to outcomes they requested or which require their response.
Provide feedback and information: provide feedback and information about the status and progress of interactions and processes.
Support brand and identity: sounds may be identifiable and associated with a product’s brand and identity.

Haptics

It’s how people see and manipulate objects using their senses of touch and proprioception.

Haptic feedback

Vibration: making the device vibrate may create different sensations.
Force: it’s used by mechanical systems to “push back” against interactions to simulate natural conditions of friction and resistance.
Texture: surface textures may be simulated with variable friction, and advanced materials may even make physical changes.

Vibration

Taps on the screens: commonly acknowledged with rapid vibrations.
Hard or long press: indicate when a press has been long or forceful enough to trigger the interaction.
Alarms and notifications: sounds and vibration are commonly used in conjunction and increase effectiveness in environments with variable noise levels
Gestural feedback: may be used to provide feedback during a gesture and at the end to indicate successful completion.

Some haptic feedback is real

Not all is simulated, physical buttons and switches provide real haptic feedback.

Error handling

No matter how much we plan, things go wrong. Errors cause frustration, and error messages are important feedback and guidance.

The best error handling is error prevention

Effective error messages

Use natural language and never blame the person.
Clearly describe what went wrong.
Briefly explain why it went wrong.
Recommend how to fix or resolve the error.

Make error messages noticeable and place them as near the error as possible.

There may be two messages:

A generic message that an error has occurred, usually, in an easily noticed location.
A specific message about the error near where the error occurred.

Presentation of Error Messages

Provide error messages as soon as possible.
Use color to draw attention and indicate urgency.
Subtle transitions can be used to remove an error message after a correction has been made.

Mistakes

When mistakes or errors are made, sometimes the best solution is undo.

Error vs Mistake

An error occurs when an action or process cannot be completed correctly.

A mistake is an incorrect choice or information that does not prevent the process from being completed, but the end result is not anticipated or desired.

Detecting mistakes

Good feedback confirms that the action was performed and reports the results or outcomes.
Important actions should provide the opportunity to make corrections or be canceled.

Usability and accessibility

Usability is the ease of use of the product or service.

Utility is the usefulness and value of the product or service.

Evaluating Usability

Sufficient motivation or need can override poor usability.
Just because someone completes a task, does not mean it was easy to do.
We need to observe people using a product and interview them about their experience to better evaluate usability.

Usability principles

Human-Computer Interaction (HCI)

Learnability: how easily can a new person learn to use the product?
Flexibility: how many ways can a person interact with the product?
Robustness: how well does the product help people when errors occur?

By Dix, Finlay, Abowl, and Beale.

Nielsen and Shneiderman

Learnability: how easily can a new person learn to use the product?
Efficiency: how quickly and accurately can a person perform tasks?
Memorability: how well do people remember a product if they have not used it recently?
Errors: how many errors occur, and how well do people recover from them?
Satisfaction: how much do people enjoy using a product, and are they satisfied with the results?

ISO 9241 - Five Principles

Learnability: how easily can a new person learn to use the product?
Understandability: how well can people understand what they are seeing?
Operability: how much control does the person have within the product?
Attractiveness: how visually appealing is the product?
Usability Compliance: does the product adhere to documented standards?

Eight Golden Rules of Interface Design

Strive for consistency.
Enable shortcuts for frequent actions.
Offer informative feedback.
Design dialogs to yield closure.
Offer simple error handling.
Permit easy reversal of actions.
Support internal locus of control.
Reduce short-term memory load.

By Ben Shneiderman, PhD.

Accessibility

All people shall be able to perceive, understand, navigate, and interact with a product, regardless of their abilities, and they can contribute equally.

Accessibility is often a legal requirement.

Basic accessibility

Text should be structured for screen readers.
Imagery should have descriptions.
Videos should be captioned.
Interfaces should be navigable and usable by multiple methods, including keyboard, mouse, touch, and voice.