Measurement

Qualitative and Quantitative

Why Measure?

Quantitative and Qualitative Measurement Parts of the Measurement Process

Quantitative Conceptualization and Operationalization Qualitative Conceptualization and Operationalization Reliability and Validity

Reliability and Validity in Quantitative Research Reliability and Validity in Qualitative Research Relationship between Reliability and Validity Other Uses of the Terms Reliable and Valid A Guide to Quantitative Measurement

Levels of Measurement

Specialized Measures: Scales and Indexes Index Construction

The Purpose Weighting Missing Data

Rates and Standardization Scales

The Purpose Logic of Scaling Commonly Used Scales

WHY MEASURE?

Perhaps you have heard of the Stanford–Binet IQ test to measure intelligence, the Index of Dissimilarity to measure racial segregation, the poverty line to measure whether a person is poor, or Uniform Crime Reports to measure the amount of crime. Social researchers want to measure concepts and variables so they can test a hypothesis, evaluate an explanation, provide empirical support for a theory, or study an applied issue. This chapter explores how we measure the aspects of the social world—such as intelligence, segregation, poverty, crime, self- esteem, political power, alienation, or racial prejudice—for the purpose of research.

In quantitative research, we are far more concerned about measurement issues than for qualitative research. In a quantitative study, measurement is a special step in the research process. It occurs prior to data collection and has a distinct terminology. There are many specialized quantitative measurement techniques.

Quantitative measurement follows a deductive approach. It starts with an abstract concept. We next create empirical measures that precisely and accurately capture the concept in a form that we can express in numbers.

In qualitative research, we approach measurement very differently. We have ways to capture and express concepts using alterna- tives to numbers. Often we follow an inductive approach. In qualitative studies, we may measure features of social life as one part of a broader process in which we create new concepts or theories at the same time. Instead of a separate step in the research process, we inte- grate measurement with data collecting and theorizing.

How we conceptualize and operationalize variables (conceptualizing and operation- alizing are discussed later in this chapter) can significantly affect social issues beyond con- cerns of doing a study to advance knowledge.

For example, psychologists debate the meaning and measurement of intelligence. Most of the

intelligence tests used in schools, on job appli- cations, and in making statements about racial or other inherited superiority measure only analytic reasoning (i.e., one’s capacity to think abstractly and to infer logically). Yet, many argue that there are other types of intelligence in addition to analytic. Some say there is practical and creative intelligence. Others suggest more types, such as social–interpersonal, emotional, body–kinesthetic, musical, or spatial. If there are many forms of intelligence but people narrowly limit measurement to one type, it seriously restricts how schools identify and nurture learn- ing; how larger society evaluates, promotes, and recognizes the contributions of people; and how a society values diverse human abilities.

Likewise, policymakers and researchers debate how to conceptualize and operationalize poverty. How we measure poverty can determine whether some people will get assis- tance from numerous social programs (e.g., subsidized housing, food aid, health care, childcare, etc.). For example, some say that people are poor only if they cannot afford the food required to prevent malnutrition. Others say that people are poor if they have an annual income that is less than one-half of the average (median) income. Still others say that people are poor if they earn below a “living wage” based on a judgment about the income needed to meet minimal community standards of health, safety, and decency in hygiene, housing, clothing, diet, transportation, and so forth. Decisions about how to measure a variable—poverty—can greatly influence the daily living conditions of millions of people.

We use many measures in our daily lives.

For example, this morning I woke up and hopped onto a bathroom scale to see how well my diet is working. I glanced at a thermometer to find out whether to wear a coat. Next, I got into my car and checked the gas gauge to be sure I could make it to campus. As I drove, I watched the speedometer so I would not get a speeding ticket. By 8:00 A.M., I had measured weight, temperature, gasoline volume, and

magnetism with your natural senses. Magnetism comes from a theory about the physical world.

We observe its effects indirectly; for instance, metal flecks move near a magnet. The magnet allows you to “see” or measure the magnetic fields. Natural scientists have invented thou- sands of measures to “see” very tiny things (molecules or insect organs) or very large things (huge geological landmasses or planets) that are not observable through ordinary senses. In addition, researchers are constantly creating new measures.

Some of the things we wish to measure are easy to see (e.g., age, sex, skin color, etc.), but many things cannot be directly observed (e.g., attitudes, ideologies, divorce rates, devi- ance, sex roles, etc.). Like the natural scien- tist who must create indirect measures of the

“invisible” objects and forces of the physical world, social researchers devise measures for difficult-to-observe aspects of the social world.

QUANTITATIVE AND

QUALITATIVE MEASUREMENT All social researchers use careful, systematic methods to gather high-quality data. Yet, qualitative and quantitative research each approaches the measurement process differently based on type of data and study design. Their approaches to measurement differ in four ways (see Table 1 ).

speed—all measures about the physical world.

Such precise, well-developed measures, which we use in daily life, are fundamental in the natural sciences.

We also measure the nonphysical world in everyday life, but usually in less exact terms. We are measuring when we say that a restaurant is excellent, that Pablo is smart, that Karen has a negative attitude toward life, that Johnson is really prejudiced, or that the movie last night had a lot of violence in it. However, such everyday judgments as “really prejudiced” or “a lot of violence” are imprecise, vague measures.

Measurement also extends our senses. The astronomer or biologist uses the telescope or the microscope to extend natural vision. Sci- entific measurement is more sensitive, varies less with the specific observer, and yields more exact information than using our senses alone.

You recognize that a thermometer gives more specific, precise information about temperature than touch can. Likewise, a good bathroom scale gives you more specific, consistent, and precise information about the weight of a 5-year-old girl than you get by lifting her and calling her “heavy” or “light.” Social measures provide precise information about social reality.

In addition to extending human senses, measurement helps us observe what is other- wise invisible. It lets us observe things that were once unseen and unknown but our theories predicted. For example, you cannot see or feel

T A B L E 1 Measurement in Quantitative and Qualitative Social Research

Measurement Process Area

Measurement in Quantitative Research

Measurement in Qualitative Research Measurement Design Timing Before data collection During data collection

Final Data Form Numbers Many diverse formats

Links of Construct to Data Sequential Interactive Process Direction Largely deductive Largely inductive

Often we start with empirical data, generate abstract ideas based on the data, align ideas with the data, and end with an interconnected mix of ideas and data.

PARTS OF THE MEASUREMENT PROCESS

As you measure a concept, you are linking an idea or construct ¹ to a measure (i.e., a technique, process, or procedure). The measurement procedure enables you to capture or observe the abstract idea (e.g., prejudice) in empirical data (e.g., survey results, a person’s statements).

Although a quantitative study has sequence and structure, the measurement process is not rigid and inflexible. As you develop measurement procedures, you may reflect on and refine the constructs because the process of developing a way to measure ideas can clarify them. Like- wise, as you apply a measurement procedure to gather data, you might adjust the measurement technique to better align with or capture details of the specific data. Although you have flexibil- ity and develop concepts inductively in a qualitative study, you will still rely on concepts from before you started data collection.

In both qualitative and quantitative studies you use two processes to measure your ideas or constructs about how the social world operates:

conceptualization and operationalization. To conceptualize you take a construct and refine it by creating a conceptual or theoretical definition for it. A conceptual definition is a definition in abstract, theoretical terms. It refers to other ideas or constructs. There is no magical way for you to create a precise conceptual definition for a concept. You must think carefully, observe closely, consult with others, read what others have said, and try possible alternative definitions.

Here is an example. How might you develop a conceptual definition of the construct preju- dice? When beginning to develop a conceptual definition, you can rely on multiple sources—

personal experience and deep thinking, 1. Timing. In a quantitative study, we think

about variables and convert them into specific actions during a separate planning stage that happens before gathering or analyzing data.

Measurement for qualitative research occurs during the data collection process.

2. Data Form. In quantitative research, we develop techniques that can produce quantitative data (i.e., data in the form of numbers). We move from abstract ideas to specific data collection techniques to precise numerical information. The numerical information is an empirical representation of the abstract ideas. In contrast, data for qualitative research comes in the form of numbers, written or spoken words, actions, sounds, symbols, physical objects, or visual images (e.g., maps, photographs, videos, etc.).

Rather than convert all data into a single, com- mon medium, numbers, in qualitative research we develop many ongoing measurement processes that produce data in diverse shapes, sizes, and forms.

3. Linkages. In all research, data are empirical representations of ideas or concepts and the measurement process links data to concepts. In quantitative studies, we follow a clear sequence:

contemplate and reflect on concepts, and then develop preplanned measurement techniques that can bridge between concepts and data before collecting data. In qualitative research, we also reflect on ideas before data collection, but we continue to reflect on, refine, and develop new concepts while gathering and examining data.

In an interactive process, we simultaneously examine and evaluate the data and reconsider and adjust the concepts. Thus, a measurement process is created “on the fly” based on what we encounter in the data and with concepts that are continuously being readjusted.

4. Direction. In quantitative research, we primarily follow a deductive path. We start with abstract ideas, create ways to measure the ideas, and end with empirical data. For a qualitative study, we primarily follow an inductive route.

discussions with other people, and the existing scholarly literature. You might reflect on what you know about prejudice, ask others what they think about it, and go to the library and look up its many definitions. As you gather definitions, the core idea should get clearer. Nonetheless, you will have many definitions and need to sort them out. Most definitions state that prejudice is an attitude about another group and involves a prejudgment, or judging prior to getting specific information.

As you think about the construct, you may notice that all the definitions refer to prejudice as an attitude. Usually it is an attitude about the members of another group. There are many forms of prejudice, but most are negative views about persons of a different racial-ethnic group.

Prejudice could be about other kinds of groups (e.g., people of a religion, of a physical stature, or from a certain region). It is always about a collectivity or group to which someone does not belong. Many constructs have multiple dimen- sions or types. You may consider whether there could be several different types of prejudice—

racial prejudice, religious prejudice, age prejudice, gender prejudice, nation prejudice, and so forth.

You read about units of analysis in the last chapter. You need to consider the units of analysis that best fit your definition of the construct. So far, you know that prejudice is an attitude. Individuals hold and express attitudes, but so might groups (e.g., families, clubs, churches, companies, media outlets).

You need to decide, do you want your definition of prejudice to include only the attitudes of individuals, or should it include attitudes held by groups, organizations, and institutions as well? Can you say, the school or newspaper was prejudiced? You also need to distinguish the construct from closely related ones. For example, how is prejudice similar to or different from ideas such as discrimination, stereo- type, or racism?

Conceptualization is the process of carefully thinking through a construct’s meaning.

At this stage, you decided that prejudice means an inflexible negative attitude held by an indi- vidual and directed toward a racial or ethnic out-group. Prejudice can, but does not always, lead to behavior, such as treating people unequally (i.e., discrimination). It generally relies on a person’s stereotypes of out-group members. Thus, your initial idea, “Prejudice is a negative feeling,” has become a precisely defined construct.

Even with all the conceptualization, you will need to be even more specific. For example, if prejudice is a negative attitude about a race or an ethnic group to which one does not belong, you must specify what you mean by race or ethnic group. You cannot assume that everyone sees racial-ethnic categories the same.

Likewise, it is possible for someone to have a positive prejudgment. If so, is that a kind of positive prejudice? The main point is that conceptualization requires you to be very clear and state what you mean in very explicit terms for other people to see.

Operationalization links a conceptual definition to a specific set of things you do (i.e., measurement techniques or procedures). It is the construct’s operational definition (i.e., a definition in terms of the specific operations of actions). An operational definition could be a survey questionnaire, a method of observing events in a field setting, a way to measure sym- bolic content in the mass media, or any process that reflects, documents, or represents the abstract construct in a conceptual definition.

Usually there are multiple ways to measure a construct. Some are better or worse, and some are more or less practical, than other ways. The key is to fit a measure to your specific conceptual definition. You must do this within practical constraints (e.g., time, money, available research participants, etc.). Your measure is also limited to the research techniques you know or can learn. You can develop a brand new measure from scratch, or you can borrow a measure other researchers already use (see Expansion Box 1).

definition or using it to measure as you collect the data. There are several ways to link abstract ideas to measurement procedures in rigorous ways that will yield precise quantitative data.

Figure 1 illustrates the measurement process for two variables. The variables are linked together both at the level of abstract theory and at the level of a testable empirical hypothesis. There are three levels to consider: conceptual, operational, and empirical. At the most abstract level, you want to examine the causal relationship between two abstract constructs.

This is your conceptual hypothesis . At the level of operational definitions, you want to test an empirical hypothesis to learn whether specific measures, or indicators, are associ- ated. This is the level at which you use corre- lations, statistics, questionnaires, and the like.

The third level is the concrete empirical world in which real people live and breathe, laugh and cry, fight and love.

If you logically link the operational definition or measures of a variable (e.g., questionnaires) to an abstract construct (e.g., racial prejudice), you can connect what happens in the concrete social world to the level of abstract theory.

The measurement process links three levels:

abstract theory, specific measures or indicators, and the concrete social reality of human activ- ity. If you adopt a deductive path, you move from the abstract to the concrete. First, you conceptualize a variable and assign it a clear conceptual definition. Next, you operationalize the variable; that is, you develop an operational definition for it. Last, you apply the operational definition in empirical reality or the concrete social world, and test empirical hypotheses. By using good measures of variables, you carefully build links between abstract ideas and empirical reality. This enables you to connect the results of empirical hypotheses back to conceptual hypotheses, and these are part of abstract theories that explain how the world works. In this way, measurement links abstract theory to empirical data.

Operationalization connects the language of abstract ideas with that of concrete empirical measures. The world of theory is filled with abstract concepts, assumptions, relationships, definitions, and causality. Empirical measures describe how particular people talk or act in a concrete setting or what specific events have occurred. A measurement procedure or technique is a set of specific operations that indicate the presence of an abstract idea in observable reality.

Quantitative Conceptualization and Operationalization

As stated earlier, quantitative research measurement flows in a straightforward sequence:

first conceptualization, next comes operationalization, followed by applying the operational

EXPANSION BOX

Five Suggestions for Coming Up with a Measure

1

1. Remember the conceptual definition. The under- lying principle for any measure is to match it to the specific conceptual definition of the construct that will be used in the study.

2. Keep an open mind. Do not get locked into a single measure or type of measure. Be creative and constantly look for better measures.

3. Borrow from others. Do not be afraid to borrow from other researchers, as long as credit is given. Good ideas for measures can be found in other studies or modified from other measures.

4. Anticipate difficulties. Logical and practical prob- lems often arise when trying to measure variables of interest. Sometimes a problem can be anticipated and avoided with careful fore- thought and planning.

5. Do not forget your units of analysis. Your measure should fit with the units of analysis of the study and permit you to generalize to the universe of interest.

another person’s theory of the creative class and a theory of how certain subcultures support a

“new” nontraditional political life in a city. The creative class includes certain occupations—

scientists and professionals in architecture and design or in the entertainment industry. Sharp and Joslyn mention other features that sustain the subculture, including a high score on

a creativity index consisting of the extent to which the workforce is in one of the “creative class” occupations noted above, the area’s innovativeness (measured by patented inno- vations per capita), the share of the area’s eco- nomic output that is from high-tech industry, and the prevalence of gays in the population.

(2008:574)

We look at indexes later in this chapter.

Other features sustaining a subculture of unconventional politics included many people who were not highly religious, who self-identified as political liberals, who had a college or graduate educa- tions, and who were young to middle-age adults.

A hypothesis has at least two variables. You must apply the processes of conceptualization and operationalization to each variable. In the preceding example, prejudice is not a hypothesis. It is one variable. It could be a dependent variable caused by something else, or it could be an independent variable causing something else. It depends on your theoretical explanation.

To illustrate the measurement process, let us look at the explanatory, quantitative study you read about in Chapter 1 by Sharp and Joslyn on tolerance in U.S. cities. The researchers had two main variables in a causal hypothesis. Their independent variable was a subculture that sup- ported “new” or “unconventional” politics. The presence of people with certain attitudes and a large number of the “creative class” in a city sustained such a subculture. Their dependent variable was racial tolerance. The conceptual hypothesis was that cities vary by subculture, with some subcultures supporting “new” or

“unconventional” politics. In such cities, they hypothesized the subculture also encourages racial tolerance. The researchers relied on

F I G U R E 1 Conceptualization and Operationalization

Conceptualization Conceptualization

Operationalization Operationalization

Hypothetical Causal Relationship

Tested Empirical Hypothesis

Theoretical Level

Operational Level

Empirical Level

Independent Variable Dependent Variable

Abstract Construct to Concrete Measure

Abstract Construct

Conceptual Definition

Indicator or Measure

Abstract Construct

Conceptual Definition

Indicator or Measure

Sharp, E. B. and Joslyn, M. R. (2008), ‘Culture, Segregation, and Tolerance in Urban America.’ Social Science Quarterly, 89: 573–591. Used with permission

Dalam dokumen Neuman, Lawrence W. - Basics of Social Research qualitative & quantitative approaches-Pearson Education Limited (2014) (Halaman 136-169)