Deck 6: Developing a Measurement Strategy

Question 1

List the steps of the measurement process.

Answer 1

No Answer

Question 2

Distinguish between manifest variables and hypothetical constructs.

Answer 2

No Answer

Question 3

Explain the concept of measurement error.

Answer 3

No Answer

Question 4

Distinguish between random measurement error and systematic measurement error.

Question 5

In measurement theory, what are the components of an observed score?

Question 6

Describe the relationship between reliability and validity of measurement using the concept of measurement error.

Question 7

Define the concept of reliability of measurement.

Question 8

What is alternate forms reliability and how is it computed?

Question 9

What is interrater reliability and how is it assessed?

Question 10

Define internal consistency and describe two ways to assess it.

Question 11

Describe three limitations of Cronbach's alpha.

Question 12

Describe the factors that influence the decision about the best approach to use in evaluating the reliability of a particular measure

Question 13

What level of reliability can be considered adequate? Describe three factors that can affect estimates of a measure's reliability.

Question 14

Explain the difference between the classical test theory and item response theory approaches to reliability of measurement.

Question 15

Describe the two types of information obtained in item response theory analyses. Explain how that information can be used to improve measurement of a construct.

Question 16

Describe two examples of the application of item response theory.

Question 17

According to your text, validity cannot be directly measured. Describe the types of evidence researchers use to evaluate the validity of a measure.

Question 18

Define content validity and explain its two aspects. How is content validity assessed?

Question 19

Briefly describe a measure that you might use in a research study. How would you establish the content validity of that measure?

Question 20

Explain the concept of structural validity.

Question 21

Explain the concept of relational (external) validity of measurement. List the steps you would take to establish that a measure exhibits relational validity.

Question 22

List three criteria that can be used to assess a measure's level of relational validity.

Question 23

Explain the concept of substantive validity. How is it established?

Question 24

Explain generalizability as an aspect of measurement validity.

Question 25

What is differential validity? How is it assessed?

Question 26

Describe two of the potential sources of differential validity in cross-cultural research.

Question 27

Distinguish between convergent and discriminant validity. Explain why discriminant validity is important.

Question 28

Explain the difference between discriminant validity and differential validity.

Question 29

Describe the steps a researcher should follow to construct a valid measure of a construct.

Question 30

What are self-report measures? List two advantages and two disadvantages of using self-report measures.

Question 31

What are behavioral measures? List two advantages and two disadvantages of using self-report measures.

Question 32

What are behavioroid measures? How are they different from other self-report measures?

Question 33

What are physiological measures? List two advantages and two disadvantages of using physiological measures

Question 34

Explain why it is difficult to establish the construct validity of physiological measures.

Question 35

Distinguish between explicit and implicit measures. Give an example of each.

Question 36

Explain the concept of response competition in the Implicit Association Test.

Question 37

List two reasons why a researcher would chose to use the Implicit Association Test.

Question 38

Describe two advantages and two limitations of implicit measures.

Question 39

Define the concept of multiple operationalism. What would be the advantage of using this strategy?

Question 40

According to your text, psychometric tests can be used as research measures, but research measures cannot be used a psychometric tests. Explain why.

Question 41

Describe the process by which a psychological test is normed.

Question 42

Explain the difference between developed and ad-hoc research measures. Why are ad hoc research measures generally poor measures of a construct?

Question 43

Describe two sources for locating measures for research.

Question 44

Imagine you are choosing a measure for your research project. You have reviewed the literature and have identified two possible measures. Describe how you would evaluate those measures to ensure you have chosen the best one for your research study.

Question 45

Explain the concept of response bias. Name and define two types of response bias of special concern to researchers.

Question 46

Measurement is central to scientific psychology because

A) it is how we make abstract concepts concrete so that we can do research on them.
B) assigning numbers to variables allows us to make comparisons and find relationships.
C) it allows us to classify individuals and things into meaningful categories.
D) all of the above.

Question 47

Measurement is central to scientific psychology because

A) statistical programs such as SPSS can only be used to evaluate measures that use an interval or ratio scale.
B) assigning numbers to variables allows us to make comparisons and find relationships.
C) only quantitative research is truly scientific.
D) all of the above.

Question 48

Which of the following statements about manifest variables is FALSE?

A) Manifest variables are abstract terms for variables that cannot be directly observed.
B) Manifest variables can be directly observed.
C) Manifest variables can be used as operational definitions of hypothetical constructs.
D) Although manifest variables can be used as operational definitions, sometimes they are studied for their own sakes.

Question 49

Which of the following statements is/are TRUE?

A) Manifest variables are variables that we can directly observe.
B) Hypothetical constructs are abstract terms invented to refer to variables that cannot be directly observed.
C) Manifest variables can be used as operational definitions of hypothetical constructs.
D) all of the above.

Question 50

In research, measures function as

A) operational definitions of manifest variables.
B) abstract representations of concrete concepts.
C) operational definitions of hypothetical constructs.
D) abstract representations of abstract concepts

Question 51

Which of the following statements about the role of inference in measurement is/are TRUE?

A) Because we cannot observe hypothetical constructs, we must infer their strength based on operational definitions of the constructs.
B) Inferences present no problem for measurement because we can be confident that our inferences are accurate within a small margin of error.
C) Inference is problematic in measurement because we cannot directly measure manifest variables.
D) Inference plays no role in measurement.

Question 52

Which of the following pairs of concepts is correctly matched?

A) reliability; accuracy
B) reliability; consistency
C) validity; consistency
D) validity; concreteness

Question 53

________ measurement error is present every time a measurement is made.

A) Systematic
B) True
C) Random
D) Observed

Question 54

________ measurement error fluctuates each time a measurement is made.

A) Systematic
B) True
C) Random
D) Observed

Question 55

A high level of ________ measurement error leads to low reliability.

A) systematic
B) true
C) random
D) observed

Question 56

A high level of ________ measurement error leads to low validity.

A) systematic
B) true
C) random
D) either a or c

Question 57

Which of the following statements about measurement error is FALSE?

A) The observed score on a measure contains both random and systematic error.
B) If a measure has a high degree of reliability it will also have a high degree of validity.
C) A measure with a high degree of random measurement error will have a low level of reliability.
D) A measure with a high degree of systematic measurement error will have a low level of validity

Question 58

________ reliability indicates consistency in scores on a measure across time.

A) Test-retest
B) Alternate forms
C) Split-half
D) Inter-rater

Question 59

Cohen's kappa is a statistic used to assess the degree of ________ reliability of a measure.

A) internal consistency
B) test-retest
C) alternate forms
D) inter-rater

Question 60

________ reliability assesses the degree of consistency among scores on items on a measure.

A) Test-retest
B) Alternate forms
C) Split-half
D) Inter-rater

Question 61

Cronbach's alpha is a statistic that represents a measure's degree of ________ reliability.

A) test-retest
B) internal consistency
C) alternate forms
D) inter-rater

Question 62

Roberto administers the Rosenberg Self-Esteem scale to a group of introductory psychology students. Six weeks later, he administers this measure to the same group of students and computes the correlation between their scores at Time 1 and Time 2. Roberto has assessed the _________ reliability of this measure.

A) test-retest
B) internal consistency
C) alternate forms
D) inter-rater

Question 63

Nour develops a measure of anxiety that has 40 items. A group of 200 adults in an inpatient facility complete the measure. Nour computes the correlation between their responses to 20 of the items and their responses to the other 20 items. Nour has assessed the _________ reliability of the measure.

A) test-retest
B) construct
C) alternate forms
D) split-half

Question 64

Tabitha develops a measure of self-esteem that has 30 items. A group of 200 college students complete the measure. Tabitha computes Cronbach's alpha for the measure. Tabitha has assessed the _________ reliability of the measure.

A) test-retest
B) internal consistency
C) alternate forms
D) split-half

Question 65

Which of the following statements about Cronbach's alpha is TRUE?

A) Cronbach's alpha assesses the degree to which people make similar responses to the items on a measure.
B) A high Cronbach's alpha indicates that the measure has a high degree of validity.
C) A high Cronbach's alpha indicates that a measure is unidimensional (that is, all the items on the measure assess the same construct.
D) As the number of items on a scale increases, Cronbach's alpha decreases.

Question 66

When drawing conclusions about the reliability of a measure, one should take into account

A) the length of time between test sessions when test-retest reliability is used.
B) the natural degree of stability or instability of the construct being measured.
C) the circumstances under which the reliability check took place.
D) all of the above.

Question 67

You read a journal article that reports that the Cronbach's alpha for one of the measures used in the study was .57. Based on this information, you can correctly conclude that

A) the measure is both reliable and valid.
B) the measure is unidimensional (that is, all the items on the measure assess the same construct.
C) the measure does not meet accepted standards of reliability.
D) both b and c, but not a.

Question 68

Item response theory analyses are used to assess the ________ of a measure.

A) validity
B) reliability
C) structure
D) stability

Question 69

In item response theory analyses, the item characteristic called the difficulty threshold represents

A) the level of ability at which a test taker has a 50% likelihood of endorsing the item.
B) the internal consistency of the measure.
C) how well an item distinguishes between people who are high and low on the trait being assessed.
D) the structural validity of the measure.

Question 70

In item response theory analyses, the item characteristic called discrimination represents

A) the level of ability at which a test taker has a 50% likelihood of endorsing the item.
B) the extent to which scores on the measure are biased against minority group respondents.
C) how well an item distinguishes between people who are high and low on the trait being assessed.
D) the structural validity of the measure.

Question 71

Information from item response theory analyses can be used for

A) creating short forms of measures.
B) developing measures that screen for a trait.
C) computerized adaptive testing.
D) all of the above.

Question 72

Which of the following statements about the concept of validity in measurement is TRUE?

A) A measure can be valid for one purpose but not for others.
B) There is no such thing as degrees of validity: A measure is either valid or it is not.
C) There are several different types of validity that are unrelated to each other.
D) Validity is always assessed through the collection of direct evidence; no inferences are involved.

Question 73

In the context of measurement, the term convergent validity refers to

A) evidence that a measure is not assessing something that it is not supposed to assess.
B) different types of evidence indicating that a measure is assessing the construct it was designed to assess.
C) evidence that the items on a measure have an appropriate difficulty threshold.
D) evidence that the measure is assessing one and only one construct.

Question 74

A psychologist measured dependency by noting how many times children in a school asked an adult for help. A critic argued that asking for help is not a good measure of dependency; in fact, the critic said, asking for help shows resourcefulness and initiative. The critic in this situation was disputing the _______ of the dependency measure.

A) difficulty
B) stability
C) validity
D) reliability

Question 75

In the context of measurement, ________ refers to the extent to which a measure provides similar results across time, research settings, and populations.

A) relational validity
B) content validity
C) substantive validity
D) generalizability

Question 76

In the context of measurement, ________ refers to the extent to which a measure adequately assesses all aspects of the construct being measured.

A) relational validity
B) content validity
C) substantive validity
D) generalizability

Question 77

In the context of measurement, ________ refers to how well scores on a measure correlate with relevant factors or criteria that are conceptually relevant to the construct being measured.

A) relational validity
B) content validity
C) substantive validity
D) generalizability

Question 78

In the context of measurement, ________ refers to how well people who get high and low scores on a measure behave in ways predicted by the theory that underlies the measure.

A) relational validity
B) content validity
C) substantive validity
D) generalizability

Question 79

In the context of content validity, representativeness refers to the degree to which the content of a measure

A) provides similar results across time, research settings, and populations.
B) includes only items that pertain to the construct being measured.
C) includes items that assess all aspects of the construct being measured.
D) correlates with relevant factors or criteria that are conceptually relevant to the construct being measured.

Question 80

In the context of content validity, relevance refers to the degree to which the content of a measure

A) provides similar results across time, research settings, and populations.
B) includes only items that pertain to the construct being measured.
C) includes items that assess all aspects of the construct being measured.
D) correlates with relevant factors or criteria outside the measure.