Deck 5: Measurement

Question 1

A researcher interested in measuring religiosity first defines the meaning and identifies possible dimensions of this concept. Which step in the measurement process does this represent?

A) conceptualization
B) validity assessment
C) operationalization
D) index construction

Answer 1

A

Question 2

Sociologist Robert Blauner identified four dimensions of alienation: powerlessness, meaninglessness, isolation, and self-estrangement. Blauner's analysis represents one aspect of

A) conceptualization.
B) operationalization.
C) reliability assessment.
D) validation.

Answer 2

A

Question 3

A researcher interested in measuring religiosity decides to ask respondents how often they pray and how often they listen to religious programs on the radio. These questions represent the researcher's attempt to

A) provide a conceptual definition of the concept.
B) refine the meaning of the concept.
C) specify empirical indicators.
D) assess the quality of the researcher's operational definition.

Answer 3

C

Question 4

Which of the following is an empirical indicator of social capital?

A) good will, fellowship, and sympathy among the individuals that make up a social unit
B) the bonding and bridging of people with one another
C) the number of people individuals report that they can rely on in a time of need
D) the ability to accrue benefits through membership in social networks

Question 5

Some sociologists define "formalization" as the extent of regulation in an organization. In one study this was indicated by the number of the following types of documents that organizations had created: written employment contracts, information booklets, an organization chart, written job descriptions, and a manual of procedures. The total count of documents is an example of

A) conceptualization.
B) operationalization.
C) reliability assessment.
D) validity assessment.

Question 6

Which of the following is an empirical indicator of alienation?

A) a feeling of estrangement from a situation, group, or culture
B) the inability to control one's immediate work processes
C) lack of a sense of purpose that connects one's job to the overall organization of production
D) whether an individual agrees with the statement "I am often left out of things going on around me"

Question 7

Which of the following is an example of an operational definition?

A) fear of crime: a series of complex changes in bodily functioning that alerts an individual to potential danger
B) life satisfaction: relatively stable assessment of how well people's lives match their expectations
C) mastery: people's sense of control over their environment
D) social isolation: the breakdown of social relations that provide meaning and social support
E) criminal activity: the number of crimes reported to the police per 1,000 population in the city or town where a respondent resides

Question 8

In social measurement, indicator is to index/scale as __________ is to __________.

A) measured; manipulated
B) one questionnaire item; two or more items
C) observation; verbal reports
D) reliable; unreliable

Question 9

Manipulation operations are used most commonly in

A) experiments.
B) surveys.
C) qualitative field research.
D) the analysis of existing data.

Question 10

In Tilcsik's experimental study of job discrimination against gay men, what type of operational definition did he use to measure "employment discrimination"?

A) manipulation operations
B) verbal report
C) observation
D) archival records

Question 11

In selecting or creating an operational definition of a concept, the textbook suggests all of the following guidelines except

A) carefully consider the theory in which the concept is embedded.
B) use more than one indicator to measure complex concepts.
C) select an operational definition that captures the meaning of the concept.
D) begin by identifying empirical indicators of the concept.
E) use quantitative measures whenever possible.

Question 12

A variable's level of measurement indicates the

A) number of questions used to measure the variable.
B) number of variable categories.
C) kinds of comparisons that can be made between cases in different variable categories.
D) degree of correspondence between conceptual and operational definitions.

Question 13

Which of the following is an example of nominal-level measurement?

A) family size (number of children in a family)
B) political participation (number of times voted in the last 10 civic elections)
C) educational attainment (highest year or grade in school completed)
D) political party affiliation (Republican/Democrat/Independent/Other/None)
E) religiosity (whether one attends religious services frequently, occasionally, or never)

Question 14

Suppose a researcher uses the following set of categories to measure employment status: working full-time; working part-time; in school. This set of categories is

A) exhaustive but not mutually exclusive.
B) mutually exclusive but not exhaustive.
C) both exhaustive and mutually exclusive.
D) neither exhaustive nor mutually exclusive.

Question 15

A survey question measuring frequency of exercise asks respondents, "On how many of the past 7 days did you exercise or participate in sports activities?" The answer categories are 0, 1 - 3, 3 - 5, 5 or more. This set of categories is

A) exhaustive but not mutually exclusive.
B) mutually exclusive but not exhaustive.
C) both exhaustive and mutually exclusive.
D) neither exhaustive nor mutually exclusive.

Question 16

As a measure of religiosity, a researcher asks, "How important would you say religion is in your own life? Is it very important, somewhat important, not very important, or not at all important?" This is an example of __________-level measurement.

A) nominal
B) ordinal
C) interval
D) ratio

Question 17

A measure of attendance at religious services asks respondents to place themselves in one of the following categories: every week, two to three times a month, once a month, less than once a month. Which level of measurement is this?

A) nominal
B) ordinal
C) interval
D) ratio

Question 18

Suppose we measured length of residence by asking people, "How many years have you lived at your present address?" This constitutes __________-level measurement.

A) nominal
B) ordinal
C) interval
D) ratio

Question 19

The GSS measure of television viewing asks, "On the average day, about how many hours do you personally watch television?" Which level of measurement is this?

A) nominal
B) ordinal
C) interval
D) ratio

Question 20

If you measured age by calculating the difference between the year a respondent was interviewed and the year the respondent was born, then your level of measurement would be __________; if you measured age by placing people in one of four categories (18 and under, 19-39, 40-59, 60 and over), then your level of measurement would be __________.

A) ratio; ratio
B) ratio; interval
C) ratio; ordinal
D) ordinal; ordinal

Question 21

Which level of measurement has an absolute or "true" zero point?

A) nominal
B) ordinal
C) interval
D) ratio

Question 22

Which of the following factors would not be a source of measurement error for scores on this exam?

A) differences among students in the ability to take multiple choice exams
B) differences in knowledge and understanding of research methods
C) ambiguously worded questions
D) students' state of health

Question 23

Systematic error would be least likely to occur with which of the following measures?

A) a miscalibrated yardstick that is an inch short
B) an IQ test developed for English-speaking U.S. citizens that is administered to French Canadians
C) a question measuring child abuse that asks, "How often do you hit your child when she doesn't obey you?"
D) scores on the multiple choice portion of this exam
E) reports of gambling winnings on IRS tax returns

Question 24

The social desirability effect refers to

A) class differences in happiness and mental health.
B) respondents' tendency to give answers that project a positive self-image.
C) the tendency to agree with a statement regardless of its content.
D) higher response rates obtained by pleasant than unpleasant interviewers.

Question 25

Research indicates that in recent years respondents' answers to survey questions measuring racial prejudice underestimate actual racial prejudice. This underestimate is an example of __________ measurement error, which affects measurement __________.

A) random; reliability
B) random; reliability and validity
C) systematic; validity
D) systematic; reliability and validity

Question 26

Suppose that on the SAT (Scholastic Aptitude Test), some test-takers do better than others because they make more lucky than unlucky guesses on questions that they cannot solve. This constitutes __________ measurement error, which lowers the SAT's __________.

A) random; measurement level
B) random; reliability and validity
C) systematic; validity
D) systematic; reliability and validity

Question 27

All of the following combinations of reliability and validity are possible except

A) low reliability; low validity
B) low reliability; high validity
C) high reliability; low validity
D) high reliability; high validity

Question 28

Which of the following combinations is possible?

A) low random error; low reliability
B) high random error; high reliability
C) low random error; low validity
D) high random error; high validity

Question 29

Measures that are subject to very little random error but a major source of systematic error

A) are reliable but not valid.
B) are valid but not reliable.
C) are neither reliable nor valid.
D) may be both reliable and valid.

Question 30

Which of the following questions most clearly concerns validity and not reliability?

A) Do self-reports of alcohol consumption underestimate the incidence of binge drinking?
B) Do respondents' answers to the question change with repeated questioning?
C) What is the average inter-item correlation among the scale items?
D) Do different coders agree in their ratings of violence on Saturday morning cartoons?

Question 31

Which of the following questions concerns reliability?

A) What does the Scholastic Aptitude Test (SAT) really measure?
B) Do attitude scales tap personal feelings or just what people think investigators want to hear?
C) Is poverty best represented by measures of low absolute income (a minimum subsistence) or low relative income (e.g., 50 percent of national median income)?
D) How closely does self-reported church attendance correspond to actual church attendance?
E) Assuming that students did not learn the answers in the interim, would they score the same on this exam if they took it again tomorrow?

Question 32

Survey estimates of church attendance based on self-reports are very stable, as respondents tend to give consistent answers. Observations of church attendance indicate, however, that it is about one-half the level reported in surveys. This suggests that, as a measure of church attendance, self-reported attendance is

A) reliable but not valid.
B) valid but not reliable.
C) neither reliable nor valid.
D) both reliable and valid.

Question 33

Test-retest, split-half, and internal consistency are forms of

A) measurement control.
B) construct validation.
C) reliability assessment.
D) validity assessment.

Question 34

Suppose you used a single questionnaire item to measure a particular concept. Which of the following methods could not be used to assess its reliability or validity?

A) test-retest reliability
B) internal consistency reliability
C) face validity
D) criterion-related validity
E) construct validity

Question 35

Suppose a researcher asks the exact same question at two different points in a lengthy interview, near the beginning and near the end. If respondents give the same answer each time the question is asked, this supports the question's

A) test-retest reliability.
B) internal consistency.
C) convergent validity.
D) construct validity.

Question 36

For various reasons, which form of reliability assessment is least likely to be used?

A) intercoder reliability
B) internal consistency
C) test-retest reliability

Question 37

Two observers rate individual leadership by recording the number of times that each member of a group initiates discussion. The level of agreement between the observers' independent records provides evidence of

A) reliability.
B) validity.
C) neither reliability nor validity.
D) both reliability and validity.

Question 38

Which of the following statements correctly applies to the reliability of this exam?

A) If nearly everyone misses a particular question, this will lower the exam's reliability.
B) The best way to assess the exam's reliability is to carefully judge how representative the questions are of the course content.
C) Ambiguously worded questions that evoke guessing have little or no effect on reliability.
D) Increasing the number of questions on the exam is likely to increase its reliability.

Question 39

Reliability assessment is a matter of checking for __________, whereas validity assessment is a matter of determining the degree of __________.

A) accuracy; consistency
B) consistency; accuracy
C) reactivity; similarity
D) accuracy; similarity

Question 40

Suppose a researcher constructs a six-item index for measuring risk-taking propensity. Evidence that eighth graders who score high on the index are more likely to engage in risky behaviors (e.g., initiate sexual activity, use drugs) supports the index's

A) test-retest reliability.
B) internal consistency.
C) intercoder reliability.
D) convergent validity.

Question 41

By reporting that scores on his Love Scale were correlated with the feeling of being "in love," Zick Rubin provided evidence of the scale's

A) internal consistency.
B) test-retest reliability.
C) intercoder reliability.
D) convergent validity.

Question 42

Which of the following statements is true regarding the assessment of measurement validity?

A) Subjective evaluation of operational definitions has no place in social science.
B) Face validity is an adequate means of assessing the validity of most operational definitions.
C) Criterion-related validity is the most common method of validity assessment in sociological research.
D) Construct validity depends on an accumulation of research evidence rather than a single prediction.

Question 43

A comprehensive exam purported to cover an entire semester's work covers only material from the first eight weeks of the course. As a measure of how much students learned overall, this exam lacks

A) reliability.
B) construct validity.
C) content validity.
D) criterion-related validity.

Question 44

Suppose that a large company administers a new employment test to fifty applicants for the position of clerk-typist. All fifty are hired, and then, after nine months on the job, are rated on their job performance by their supervisors. Correlating test scores with supervisor ratings is an attempt to assess the test's

A) test-retest reliability.
B) internal consistency reliability.
C) criterion-related validity.
D) construct validity.

Question 45

Evidence that scores on the Rosenberg Self-Esteem Scale are associated, as expected, with measures of depression and peer popularity supports the scale's

A) test-retest reliability.
B) internal consistency.
C) discriminant validity.
D) construct validity.

Question 46

A near-perfect correlation between a measure of romantic love and a measure of liking suggests that the measure of romantic love

A) is reliable.
B) is internally consistent.
C) lacks discriminant validity.
D) lacks convergent validity.

Question 47

In contrast to criterion-related validity, construct validity is important to assess when

A) the measure in question is intended for practical application rather than theoretical purposes.
B) there is a single, agreed-upon criterion for assessing validity.
C) the primary concern is the accuracy of prediction.
D) the primary concern is with capturing the theoretical meaning of the concept being measured.

Question 48

In assessing your newly developed scale for measuring anti-Semitism, you get highly similar results when you administer your scale twice to the same group. You also get a high correlation between your new anti-Semitism scale and another scale designed to measure the tendency of people to give socially desirable responses. According to this evidence, your scale is

A) reliable but not valid.
B) valid but reliable.
C) both reliable and valid.
D) neither reliable nor valid.

Question 49

The logic of construct validation is based on all of the following except

A) theoretical relations between the concept being measured and other concepts.
B) the internal consistency of variables with which the concept is theoretically related.
C) theoretical predictions of variables with which the concept should be associated.
D) theoretical predictions of variables with which the concept should not be associated.
E) an accumulation of evidence rather than a single association between the concept and a related variable.

Question 50

The measurement process moves from the concrete to the abstract.

Question 51

Operational definitions are intended to capture the meaning of conceptual definitions.

Question 52

Operational definitions may be manipulated or measured.

Question 53

The most common form of social measurement is direct observation.

Question 54

One empirical indicator is as good as another.

Question 55

Whenever possible, researchers should avoid the use of multiple indicators of a concept.

Question 56

Often, many different indicators may be chosen to measure a given concept.

Question 57

There is almost always an imperfect correspondence between conceptual and operational definitions.

Question 58

Measurement level is determined by the number of empirical indicators used to operationalize a concept.

Question 59

In nominal measurement, the numbers assigned to categories may be added together but not multiplied.

Question 60

The lowest level of measurement is nominal.

Question 61

Interval-level measures have an arbitrary zero point.

Question 62

Class rank in a graduation class is an example of ratio measurement.

Question 63

It is possible for a measure to be reliable but invalid.

Question 64

If all the variation in a measurement is due to true differences in the concept, then the measure is both reliable and valid.

Question 65

Systematic measurement error tends to lower reliability.

Question 66

The cultural bias of IQ tests is an example of systematic measurement error.

Question 67

Reliability assessment is a matter of checking for consistency.

Question 68

Internal consistency is used to assess the reliability of single-item measures.

Question 69

Using additional indicators of a concept (e.g., additional questionnaire items) can increase reliability.

Question 70

All forms of measurement validation are subjective to a certain extent.

Question 71

Criterion-related validity generally is reserved for tests and measures that are intended for practical application.

Question 72

The social desirability effect lowers measurement validity.

Question 73

Construct validation depends on an accumulation of research evidence rather than a single prediction.

Question 74

Correlations among different measures of the same concept are evidence of the measures' convergent validity.

Question 75

Correlations among different measures of the same concept are evidence of the measures' discriminant validity.

Question 76

A near-perfect correlation between level of education and scores on a mental health scale suggests that the scale lacks discriminant validity.

Question 77

Briefly explain how the process of measurement moves from the abstract to the concrete.

Question 78

Explain how the text's example of a recipe for carrot cake illustrates an operational definition in social research. What are the recipe ingredients analogous to in social measurement? What do the steps for combining ingredients and baking the cake illustrate about an operational definition?

Question 79

Levels of measurement indicate the inferences that can be drawn when comparing one category or value of a variable with another: Do they differ? Is one ranked higher than the other? What is the numerical difference? What is the ratio of the values? Identify the four levels of measurement. Then, for each level describe the inferences that can be made when comparing units (e.g., people) who are in different variable categories.

Question 80

Describe the relationship between reliability and validity. (NOTE: Your description should answer the following two questions: Is it possible to have a reliable but invalid measure? Is it possible to have an unreliable but valid measure?)