COMPARING RESEARCH TOOLS

Although certain research tools are particularly valuable at specific points in
the design process, it is also possible to generalize about situations for using
individual research tools. The flow chart presented in Figure 2.1 is a quick
guide to the research issues that, in general, might lead you to select one
research tool instead of another.
The reason that some research tools are more appropriate in some situations
than others is the strengths and weaknesses of the tools themselves.
Each of the tools described in the next few paragraphs will be presented in
depth in the chapters that follow.
Written surveys have been an integral part of the design research process
for decades, and for good reason—they efficiently and effectively (and relatively
inexpensively) gather a lot of useful information. Surveys have credibility
and legitimacy with client groups, making them a valuable resource. When
you are using a written survey, you collect the same information from each
respondent, whether they are from the same client group or even the same project, and that consistency means that you can compare material you collect
in one situation to data gathered in another.
You will wait a long time for data if you do not effectively manage the survey
distribution and collection process. Reminders are key, and it is important
to preview the data gathered before the final data collection deadline to make
sure that nothing is amiss with the process.
Written surveys are not a panacea, however. There are certain situations
where people just can’t answer questions that designers would like to pose.
Nondesigners are notoriously bad at describing “ideal” sorts of environments,
for example. Lacking a design vocabulary, nondesigner respondents can only
rehash spaces they have seen or visited in the past. People’s expectations
about future places are strongly influenced by their past place experiences.
In some instances, respondents don’t have the information required to
answer a question. For example, people might have been doing certain tasks
for so long that they have forgotten what they actually do in a particular
situation.
The problem with asking people questions that they can’t answer well is
that respondents (who generally want to help researchers get the information
that they need) will answer questions posed and won’t volunteer that they
don’t know, can’t recall, or can’t respond. In these situations, a survey is better
replaced by an observation technique that allows the designer to watch the
activity to understand the actual situation.
For example, asking an office worker how many informal conversations he
or she has in his or her workstation each day is problematic. The office worker
might not really know, since these discussions are unplanned and often very
short or seemingly unimportant, or the person being interviewed or surveyed
may know that these conversations can annoy nearby workers and therefore
not want to acknowledge them. However, gaining insight into this question is
important input into the development of a future workspace. In this situation, a
researcher’s ability to observe the frequency of this behavior would be useful.
When you ask survey questions, you also can’t probe intriguing responses
or resolve apparent inconsistencies in answers—particularly if survey respondents
are anonymous. Following written surveying with observations or discussions
(single interviews or group talks) can help rectify this situation.
Written surveys can also seem inappropriate in some circumstances. If a
one-off single family home is being created, a survey of future residents may
seem inappropriately “mechanical.” It can also be impossible to survey some
user groups—patients in some healthcare settings may not be willing or able
to complete surveys, for example.
Developing a survey, particularly one with multiple-choice questions,
requires enough information about the situation being studied to write effective
questions. A lot of knowledge of what nurses might be doing in a nurses’
station is needed to develop a multiple-choice question about their activities
there that provides appropriate response options.
Most surveys are now administered electronically and data that are collected
flow directly into an electronic spreadsheet, but if you are distributing
a survey in paper form, entering and analyzing the data collected can be time
consuming.
Real-time, real-place sampling of the experiences of stakeholders is little
used, but is often an effective research tool. It is known as the experience sampling
method, or ESM. Study participants receive pagers or are contacted in
some other way at randomly selected times while in a space to be redesigned.
Each time participants receive a page, they are asked to answer customized
questions about the specific activity they are engaged in, their psychological
state, aspects of the activity, or other topics of interest. This technique
has become increasingly viable as small handheld terminals for relaying the
answers to these questions have become more available and economical. The
resulting data sets are comprehensive and very rich in useful information.
Speaking with people face to face is another valuable way to collect information.
Not only do open discussions help resolve many of the cloudy issues
raised by survey responses, they allow the design researcher to explore topics
of interest in more detail. Follow-up questions are possible that are directly
related to the answers provided. This is particularly important if unexpected
but pertinent information is shared and is not possible when a written survey
is used. Within very broad parameters, it is possible to evolve interview questions
being asked as the research process unfolds. When you need to collect
information but don’t have much knowledge related to a project, speaking face
to face can be a viable option.
In this scenario, interactions between the interviewer and an interviewee,
or interviewees if group discussions are held, often result in fortuitous
exchanges that collect crucial information that directly influences the ultimate
design solution. Interchanges among participants in a discussion group are a
good way to establish overall design priorities. Regularly, during group discussions,
interviewees ask each other questions or supply clarifying bits of information
that greatly enrich the research process. These interactions are the
primary methodological reason for having group discussions. Design charettes
are a specific sort of discussion group.
Common research practice demands that certain interviews be conducted.
If a residence is being constructed and the ultimate residents of a home are
known, they should be interviewed, as should executives and other stakeholders
at firms where workplaces are being designed, for example. These executives
are an important source of information about an organization’s future, for
example.
Outside experts are sometimes interviewed in the course of a design
project. These interviews are particularly useful when the research schedule
needs to be compressed because of time constraints and appropriate
experts are available for interviews. There are many types of experts (place
type, social science subject, etc.). The criteria used to select the person(s) interviewed are determined by the research project objectives. For example,
an interview with a neonatal intensive care nurse with years of professional
experience can be substituted for interviews with a range of stakeholder
nurses and for observations in situ if a new neonatal intensive care department
is being planned. The expert is a good alternative when and if there
are no current stakeholders to interview or if concerns about infection preclude
observation, for example. It is always dangerous, however, to rely
solely on information from a single expert, no matter how strong the expert’s
credentials.
Collecting information using interviews can be time consuming, particularly
when the hours required to select interviewees and actually schedule discussions
are considered. Since multiple users participate in discussion groups,
they are a little more efficient, but it takes longer for the interviewer to coordinate
the schedules of all participants in a group.
Interviewing itself can also take longer than might initially be anticipated
because transition time between interviews is always required, and those
transition times may be quite long if the interviewer needs to travel between
interviews, even within the same building.
It can be difficult for individuals to frankly discuss information that they
find sensitive, and researchers need to consider the types of topics they need
to investigate before using interviews as a research tool. Some topics, such
as sexual issues, are clearly more difficult for most people to discuss with an
interviewer, but other topics might be more subtly tough, such as those related
to financial or human resource performance.
Group activities have many of the advantages and disadvantages of
group interviews, but they often relax participants, so they can be a good
way to begin a group discussion session. Group activities involve everyone
present in the conversation, and once people participate, they are much
more likely to do so again. They include creating collages that address some
design-related issues, dramatizing particular situations of interest, and other
similar participatory exchanges. They can be used to resolve appropriate
use of limited resources, as well as to assess particular design options or
preferences.
When people are unable to respond to questions on topics of interest
either orally or in writing, observations can be key. Observations can also
be useful when there is clearly a “correct” answer to a particular question
or a clear motivation to distort the data collection process. In North America
today, people at firms undergoing downsizing may be reticent to admit that
there are serious issues with their current work environments, but observations
of body language, for example, may clearly delineate those issues.
Space syntax tools are a specialized form of very structured observation,
at the level at which we will discuss them here. Observation is also useful
when users don’t have experience talking about an issue. They may lack
the vocabulary to do so effectively. Sometimes people have been doing something for so long, they no longer really know what they do—imagine
your grandmother preparing for a holiday feast—and for those situations,
observations can again be useful. When you’re observing, it’s also possible to
note signs of wear in an environment or physical modifications to the space
that can help guide future design decisions. For example, a chair that’s pristine
probably is sat in only occasionally. Other research methods can learn
why that might be the case. If conference room windows have paper stuck
over them, that might indicate a lack of display space inside or a need for
more privacy during meetings.
Behavior mapping is an organized way to collect information about how
people use a space for example, how they move through a space, and where
they linger. It is a specialized form of observation and discussed in depth in
Chapter 6. At a fundamental level, it involves literally recording on a floor plan
of a site the travel and place-use behaviors of a project-relevant sample of the
people who use a space.
People can be observed and then interviewed immediately when space
simulations are used, and the combination of interviews and observations in
these realistic settings provides important information about the probable real
experience of being in a proposed space. Sometimes, surveys are also used in
combination with simulations. Constructing simulated environments is expensive,
however, and care must be taken to collect information from a representative
sample of potential space users.
Reviews of previous research are a good way to begin any new project.
Material can be effectively accessed through the Internet or other publicly
available sources when the parameters used to determine research quality,
which are fully presented in this chapter and in Chapter 4, are followed. Using
the correct search terms is key to effective literature reviews. In the chapters
that follow, we help identify effective search terms.
Literature reviews provide a strong foundation for a research program and
indicate topics that do not need to be investigated for a specific project because
generalizable findings have already been determined.
Content analysis of visual and verbal information is a valuable way to
uncover implicit, tacit information, although it is also a good source of explicit
material. It involves reviewing written and visual records to identify often
unstated but significant issues of importance to stakeholders. Obtaining
the materials to be studied can be difficult. Content analysis is discussed in
Chapter 5.
Geographic information systems (GIS) store a variety of information that
can be used by designers. They present data about the exterior, surrounding
environments of structures being built—and those environments may be
natural, demographic, and economic, among others. These materials can be
used to site structures so that views to natural vistas are maximized and stress
on natural resources is minimized. GIS is also useful for selecting sites that provide user-desired nearby amenities. GIS tools are becoming more advanced
and common; their use is further discussed in Chapter 6.
We will use ten criteria to distinguish the research tools presented in this
book. These criteria can guide your decisions about when particular tools
should be used:
1. Some tools are particularly useful in circumstances, such as at the beginning
of a project, when little is known about the topics being researched;
others are particularly useful in other circumstances, such as at the end of
a project to investigate a few remaining outstanding issues. This is summarized
in the text as little known and lot known.
2. Tools may collect information relatively quickly or it may require more
time to use them to gather useful information. This is noted using the
terms quick and not quick.
3. The information obtained with a tool might be straightforward or complex
to interpret and discuss. This is summarized in the text as info straightforward
and info complex.
4. Data collected can also be distinguished based on whether they are generally
textual or verbal or more numeric—noted as verbal and numeric.
5. Some tools are particularly useful for answering “why” questions and others
for “how” questions—hence why and how.
6. Sometimes researchers need to be at the client site when information is
collected and with other tools they can be off-site. The abbreviations onsite
and off-site OK are used to describe this tool parameter.
7. Some methodologies are more well established so they have more
accepted protocols and there are more related online resources and others
are more innovative—summarized as well-established and innovative.
8. Tools may be better for collecting information about current conditions
(from which future places will be developed) or better for learning about
expected future conditions—thus, the terms current and future are used to
describe a tool’s rating on this factor.
9. Special software or hardware might or might not be required to collect
information (special, no special).
10. Some techniques are so generally useful they should be considered for
any project, and probably utilized, while others are much more specialized
(generally considered, potentially considered).
A chart at the beginning of the discussion of each research tool will indicate
that tool’s rating on these ten criteria.