Deck 4: Conceptualization and Measurement

Question 1

Whether the SAT can accurately predict college GPA is a question of its concurrent validity.

Answer 1

False

Question 2

A scale is an index in which different items are given different weights.

Answer 2

True

Question 3

Use the following to answer questions 16-18:
In each set, match a concept from Group A to a definition or example from Group B.
Measurement Procedures
Group A
1. Available Data
2. Constructing Questions
3. Direct Observations
4. Content Analysis
5. Triangulation

Group B
a. Systematically analyzing and making inferences from text
b. Using more than one procedure to measure the same variable
c. Using the General Social Survey to test hypotheses
d. Writing a survey or conducting interviews
e. Riding in police cars to determine police-citizen interactions
f. Applying an experimental stimulus to one group and no stimulus to another

Answer 3

1-c
2-d
3-e
4-a
5-b

Question 4

Use the following to answer questions 16-18:
In each set, match a concept from Group A to a definition or example from Group B.
Levels of Measurement
Note that levels of measurement may be used more than once.
Group A
1. An index of political participation with a range from 5 to 45
2. Age, defined as child, young adult, adult, and senior
3. Gender: male or female
4. Number of siblings
5. Temperature in degrees Fahrenheit
6. Score on exam
7. Military rank
8. Citizenship: U.S. Citizen or Not U.S. citizen

Group B
a. Nominal
b. Ordinal
c. Interval
d. Ratio

Question 5

Age in years is a good example of a nominal level of measurement.

Question 6

Using archived (preexisting) data is a type of unobtrusive measure.

Question 7

Operationalization is the process of defining what exactly a researcher means by a key concept.

Question 8

The ordinal level of measurement has the highest mathematical precision.

Question 9

Concepts have meanings that are understood well by almost all members of a population.

Question 10

Nominal and ordinal level closed-ended questions must be mutually exclusive and exhaustive.

Question 11

An open-ended question is designed with explicit response choices.

Question 12

Use the following to answer questions 16-18:
In each set, match a concept from Group A to a definition or example from Group B.
Measurement Validity
Group A
1. Face validity
2. Content validity
3. Criterion validity
4. Concurrent validity
5. Predictive validity
6. Construct validity

Group B
a. A measure obviously pertains to the meaning of the concept being measured more than to other concepts
b. A measure is validated by being closely related to a criterion conducted at the same time
c. A measure is validated by relating to other measures specified by a theory
d. A measure covers the full range of a concept's meaning
e. A measure is validated by its ability to yield consistent scores
f. A measure is validated by predicting scores on a criterion measured in the future
g. The results of one measure match those obtained with a more direct or an already validated measure of the same phenomena

Question 13

Scott defines "minority owned" in his study of businesses in Northeastern City as being those businesses that are currently owned by women or African Americans. What has he done?

A) Made an ecological fallacy
B) Made a reductionism fallacy
C) Conceptualization
D) Operationalization
E) Triangulation

Question 14

Concepts that do not vary are known as constants.

Question 15

Inter-item reliability should occur when using multiple indicators of a single concept.

Question 16

Samples of individuals who use social media such as Facebook are representative of the whole population.

Question 17

Codes can be created by using key words from the text that is being analyzed.

Question 18

Age, gender, authoritarianism, depression, and safety are all examples of:

A) Indicators
B) Concepts
C) Constants
D) Indexes
E) Scales

Question 19

Reliability is a prerequisite for measurement validity.

Question 20

Textual analysis is a research method for systematically analyzing and making inferences from text.

Question 21

Construct validity measures how well something is measured in comparison with:

A) A population
B) A target population
C) A measure taken later in time
D) Dimensions of a theoretical concept
E) Dimensions of reliability

Question 22

The General Social Survey only asked currently married people about their marital happiness. Thus, in any study of marital happiness using GSS data, what must be a constant?

A) The independent variable
B) The dependent variable
C) The direction of association
D) Marital status
E) All control variables

Question 23

What is wrong with the following question's response categories: What is your class status: freshman, sophomore, junior, senior.

A) Not mutually exclusive
B) Not exhaustive
C) Not a numerical level of measurement
D) Not reliable
E) Both a and b

Question 24

When researchers measure an unchanging phenomenon at two different times, the degree to which the two measurements are related is called:

A) Test-Retest reliability
B) Criterion validity
C) Predictive validity
D) Inter-item reliability
E) Alternate-forms reliability

Question 25

Which of the following is an example of a variable measured at the ratio level of measurement?

A) Income in dollars
B) An index of depression with a potential range of 10 to 100
C) A person's ethnicity
D) Military rank
E) Socioeconomic status measured as high, medium, and low

Question 26

Which of the following is a synonym for the social scientific meaning of the work "operation"?

A) Measurement
B) Mechanism
C) Context
D) Process analysis
E) Variable

Question 27

Which of the following is an example of a variable measured at the ordinal level of measurement?

A) Military rank
B) Birth order of siblings
C) Age measured as child, teen, adult, and elderly
D) Order in which subjects complete a given task
E) All of the above are measured at the ordinal level

Question 28

Which of the following is true about indexes but false about scales?

A) Multiple items are combined to determine scores
B) Items are given equal weight when summing or averaging
C) Items can be written in a matrix format
D) They are designs to measure concepts
E) Previous developed items promote comparability between studies

Question 29

Inter-observer reliability occurs when different observers:

A) Measure the same phenomena in the same way
B) Agree on a definition of measurement
C) Administer an instrument
D) Contribute to the conceptualization process
E) Measure some phenomena concurrently

Question 30

Which of the following is an example of a variable measured at the interval level of measurement?

A) Age in years
B) Income in dollars
C) Temperature in degrees Fahrenheit
D) Acres of land planted in food crops
E) None of the above are measured at the interval level

Question 31

To establish concurrent validity, a researcher must include which of the following in a survey that includes a new index of depression:

A) A system for weighting responses
B) Multiple forms of the same index
C) A previously validated measure of depression
D) Single items and matrix questions
E) Open-ended questions that ask for details about responses

Question 32

Using more than one measure of the same variable, such as using a survey and direct observation, is known as:

A) Validation
B) Reliability
C) Conceptualization
D) Triangulation
E) Operationalization

Question 33

When researchers use multiple items to measure a single concept, they hope for high levels of consistent responses on these items. If they achieve this sort of internal consistence, they have achieved:

A) Test-Retest reliability
B) Construct validity
C) Content validity
D) Inter-item reliability
E) Inter-observer reliability

Question 34

When conducting a survey, researchers may often create two versions of the questionnaire. They may vary the order of the questions or slight wording of the questions. They then divide the sample into two equal sub-samples and administer each version. When surveys are collected, answers on the two different questionnaires are compared to determine:

A) Test-Retest reliability
B) Criterion validity
C) Alternate-forms reliability
D) Construct validity
E) Content validity

Question 35

Which level of measurement is the most mathematically precise?

A) Interval
B) Nominal
C) Ordinal
D) Ratio
E) All levels of measurement are equally precise

Question 36

What is wrong with the following question's response categories: How often to you attend live music concerts: never, 1-2 times per week, 2-5 times per week, 5 or more times per week.

A) Not mutually exclusive
B) Not exhaustive
C) Not a numerical level of measurement
D) Not reliable
E) Both a and b

Question 37

Reliability refers to which of the following?

A) That a measurement captures what it intends to measure
B) That a measurement includes all dimensions of a concept
C) That a measurement is free of error
D) That a measurement is based on empirical reality
E) That a measure yields consistent scores

Question 38

Predictive validity occurs when:

A) A measure can predict scores on a criterion measured in the past.
B) A measure relates to other measures specified in a social theory.
C) A measure is associated with a criterion collected at the same time.
D) A measure can predict scores on a criterion measured in the future.
E) A measure is operationalized as a variable.

Question 39

Open-ended questions are used most commonly by which type of researcher?

A) Researchers who study diverse people
B) Researchers who study small numbers of people
C) Researchers who use quantitative measures
D) Researchers who do descriptive analysis
E) Researchers who want a representative sample

Question 40

Which of the following is an example of a variable measured at the nominal level of measurement?

A) Location in which respondent was born
B) Religiosity measured as not religious, somewhat religious, and very religious
C) Time in seconds in which a subject completes a given task
D) Number of respondents' first cousins
E) Level of education in years completed

Question 41

Propose measures of job satisfaction at the nominal, ordinal, and interval or ratio levels. Discuss the relative merits of each. Suggest procedures for establishing validity and reliability of all three of your job satisfaction measures. Explain your rationale for each procedure.

Question 42

What is the difference between conceptualization and operationalization? After defining these terms, demonstrate this difference by conceptualizing and operationalizing one of the following concepts: happiness, prejudice, deviance, environmentalism, justice, poverty. Discuss potential measurement procedures for the concept.

Question 43

When are open-ended questions more appropriate than closed-ended questions? Describe scenarios where each would be appropriate for studying one of the following concepts: deviance, romance, educational quality, economic development.
Answer Key

Question 44

Which of the following is considered a form of new social media?

A) Facebook
B) MySpace
C) Twitter
D) Reddit
E) All of the above

Question 45

John wants to analyze how female characters are portrayed in various cartoon shows. What type of analysis would John most likely employ?

A) Content analysis
B) Deductive analysis
C) Inductive analysis
D) Contextual analysis
E) Cartoon analysis

Question 46

Propose a study using direct observations and unobtrusive measures for measuring conspicuous consumption among college students. Identify several measures that you might use in this study. Discuss how using direct observation and unobtrusive measures has advantages and disadvantages relative to surveys for measuring conspicuous consumption.

Question 47

You must develop a measure of student satisfaction with your school. Propose measurement procedures using available data, questions, and observations. Discuss the advantages and disadvantages of the three measurement approaches.

Question 48

Develop a conceptualized definition of terrorism, making certain to consider the potential dimensions of the concept (and therefore construct validity). Write an index that measures this concept with closed ended questions.

Question 49

What are the advantages and disadvantages of using existing data sources for research? What is the GSS and why is it so useful for social research?

Question 50

What are the advantages and disadvantages of using single questions rather than multidimensional index scores to measure concepts?

Question 51

Which of the following is not a type of unobtrusive measure?

A) Archives
B) Contrived observation
C) Indexes
D) Physical trace evidence
E) Simple observations

Question 52

Questions that allow a respondent to select from a predetermined set of responses are called:

A) Closed-ended questions
B) Fixed-choice questions
C) Open-ended questions
D) Both a and b
E) Both a and c

Question 53

Counting the number of articles devoted to women's anxiety in women's magazines in 1930, 1950, 1970, and 1990 is an example of:

A) A survey
B) An obtrusive measure
C) An unobtrusive measure
D) A content analysis
E) Using available data

Question 54

Construct a short survey of five questions designed to measure political attitudes. At least one question must be open ended, one must be closed-ended, and one must be an index or a scale. Discuss an alternative procedure to measure political attitudes, and how the survey would compare with this alternative procedure in terms of validity.

Question 55

Which of the following would not be analyzed using content analysis?

A) Romance novels
B) Interview transcripts
C) Science fiction films
D) Women's magazines
E) Newspaper articles

Question 56

What is triangulation? Why is triangulated measurement superior to single forms of measurement?

Question 57

Roth (1989: 148-149) defined homelessness as follows in her Ohio study: "In this study a person was defined as homeless if he/she sleeps/lives in: (1) limited or no shelter for any length of time (for example, those seeking shelter under bridges, inside door stoops, in cars, in abandoned buildings, in a bus station or all-night café, or in any public facility); (2) shelters or missions, run by religious organizations or public agencies for any length of time (facilities that are specifically for homeless people. Are run on a drop-in basis and charge no fee or a minimal fee, including the Salvation Army, Volunteers of America, and Open Shelter of Columbus); (3) cheap hotels or motels when actual length of stay, or the intent to stay, is 45 days or less; and (4) other unique situations that do not fall into the above categories and when the actual length of stay, or the intent of stay, is 45 days or less (for example, staying with family and friends, living in tent cities, or spending a night in jail)." Rewrite Roth's definition so that it is more succinct and conceptual and less operational. Next, rewrite your conceptual definition with as much level of operational detail as in Roth's definition, but without repeating the specifics of the original definition. What ambiguities remain in your definition that might confuse a research assistant trying to apply the definition in the field?