Exam 3: Measurement Reliability

Briefly define the following components of Campbell and Fiske's (1959) Multi-trait Multi-Method Matrix:

Generating a large pool of items, consulting a panel of experts, and conducting a review of the literature, are all ways to improve:

In a study that investigated the relationship between television watching habits and work productivity, researchers asked respondents three items related to their TV watching: a. Item 1: On an average week, do you watch television? b. Item 2: How often do you watch television with others (rather than by yourself)? c. Item 3: How many hours per day do you watch television? However, when the researchers ran their item response analyses, they realized they never learned how to interpret them - so they called you. Your task is to describe what is happening in the figure below. Specifically: (a) briefly describe what we are looking at - what do "discrimination" and "ability (or theta)" represent in the context of the study? (b) interpret the three curves for someone at theta = -2, and describe to the researchers what that means; (c) describe how we would determine whether a curve reflects "good" discrimination; and (d) based on that information, suggest to the researchers which item(s) they should keep and which item(s) they should remove, and be sure to explain why you are offering those suggestions.

Edward and Elvira developed a 50-item scale that measured the extent to which people were gullible to practical jokes. After collecting data from 1,000 participants, their scale demonstrated poor internal consistency (Cronbach's alpha = 0.40). What (if any) suggestions would have the potential to dramatically increase the overall reliability of their scale? (circle all that apply)

In terms of reliability, describe how analyses with latent factors are different from classical test theory techniques like Cronbach's alpha or test-retest. Are they better or worse, and why?

Imagine that Susan is a high school senior completing a survey for your research study. Presume that earlier that morning, Susan received a rejection letter from one of her top colleges. In addition, your survey involves the topic of racism in everyday life. Susan is also an international student, and English is not her first language. You look at Susan's completed survey and notice that all of her item responses lie close to the midpoint of your 9-point scale. Given this information, which of the following threats to validity may be at play?

William Smythe wanted to create a scale that measured the likelihood that people will burst out in laughter in a movie theater. So far, he has four items written. These three items yielded a Cronbach's alpha of 0.50 and the following output: Given what you know about Cronbach's alpha and improving internal consistency, describe the ways in which Will Smythe could potentially increase the internal consistency of his scale (hint - there may be more than one way)? Item-Total Correlations Item 1 0.767 Item 2 0.629 Item 3 0.105 Item 4 0.520 Given what you know about Cronbach's alpha and improving internal consistency, describe the ways in which Will Smythe could potentially increase the internal consistency of his scale (hint - there may be more than one way)?

Face validity differs from other forms of validity in that:

In the MTMMM, monotrait-heteromethod triangles:

In an experiment on teen aggression and violent video games, some teens were driven to the study by their parents, while others drove themselves. It turned out that participants who were driven by their parents were generally more angry when they arrived (and thus exhibited a stronger relationship between aggression and video games), compared to participants who drove themselves. The issue of transportation to the study represents what potential problem?

_____(a)_____ is the degree of relationship between an instrument and the construct it is intended to measure. _____(b)_____ is the consistency with which an instrument assesses a given construct.

In confirmatory factor analysis, the measurement error terms represent what type(s) of error excluded from the latent factor? (circle all that apply)

Describe what is meant by the equation: O = T + ΣE_r+s In your response, be sure to identify and describe each element of the equation.

If a hypothetical scale meant to assess knowledge of zoology contained questions about animals, plants, and inanimate objects, the scale would be said to have low _______ validity.

A person's observed response pattern on any measurement scale is a combination of true score and error. In the context of this equation, "true score" refers to:

Explain the relationship between true score, observed score, and random error.

How can convergent and discriminant validation techniques be combined to establish construct validity? (triangulation, different sources of error, MTMMM)

What are some advantages and unique possibilities of latent factor analysis, compared to more classical test theory approaches?

Which of the following are methods of assessing the internal consistency of a scale? (circle all that apply)

Compare and contrast the test-retest and parallel forms techniques for assessing reliability.