Deck 9: Nonexperimental Research

Question 1

Sometimes, variables we are interested in researching are such that participants arrive at a study with preexisting levels of these characteristics. Accordingly, these variables cannot be experimentally manipulated. Which of the following are example(s) of variables that cannot be manipulated?

A) political affiliation
B) affective state
C) openness to experience
D) A and B
E) A and C
F) all of the above

Answer 1

E

Question 2

Which of the following statistical analyses can be used with nonexperimental (i.e., correlational) research designs? (circle all that apply; if you choose e or f, just circle e or

A) multiple regression
B) multilevel modeling
C) ANOVA
D) path analysis
E) all of the above
F) none of the above

Answer 2

E

Question 3

Which of the following statistical analyses can be used to determine experimental causality? (circle all that apply; if you choose e or f, just circle e or

A) multiple regression
B) multilevel modeling
C) ANOVA
D) path analysis
E) all of the above
F) none of the above

Answer 3

C

Question 4

Which statement best describes the term: coefficient of determination?

A) the degree of relationship between two measured variables
B) the proportion of shared variance between two measured variables
C) the standardized correlation between two variables
D) the one-directional arrow that points from predictor to criterion

Question 5

The correlation between two variables in a study was extremely low, and much lower than expected (r = 0.01). Which of the following statements could be feasible explanations for the small observed relationship between X and Y?

A) in reality, there is in fact negligible systematic variation of X and Y
B) the relationship between X and Y is curvilinear
C) the range of response options on X was too narrow
D) A and B
E) B and C
F) all of the above

Question 6

Why does generalizing a multiple regression equation to another sample tend to produce a lower R² in the second sample?

A) the internal validity of nonexperimental methods is lower than for experimental methods
B) the external validity of nonexperimental methods is lower than for experimental methods
C) the regression equation assumes that data are free from measurement error, so any error present in sample 1 will likely be different from error in sample 2, and in turn, decreasing the amount of correspondence between sample 1 regression weights and sample 2 data.
D) the regression equation assumes that data are free from sampling error, so any error present in sample 1 will make the regression weights artificially high, and in turn, sample 2 reveals variable weights that are more accurate reflections of reality

Question 7

Two researchers wanted to investigate the relationships between the Big Five personality traits (i.e., openness to experience, conscientiousness, extroversion, agreeableness, neuroticism) and right wing authoritarianism, as well as how these factors combined to predict people's attitudes toward public schools. They conducted a survey to assess respondent's scores on these seven variables. What would be the most appropriate design to analyze these relationships?

A) hierarchical linear modeling
B) multiple regression
C) Pearson correlations between all possible bivariate combinations
D) structural equation modeling

Question 8

A correlational relationship between any two variables (A and

A) C causes common variation in both A and B, hence the observed correlation between A and
B) C may mediate the relationship between A and B, such that A leads to changes in C, and C then leads to changes in
C) C and A are measures of the same underlying construct, with different sources of error, such that when C is included in a multiple regression, the relationship between A and B disappears.
D) A and B are, in fact, unrelated were we able to objectively measure this relationship in reality, so the observed correlation between A and B is due to a third variable: measurement error.

Question 9

Jesse and Hank investigated the relationships between the geographic region in which respondents lived, respondents' attitudes toward public school, and their willingness to break rules to help others, what Jesse and Hank deem the "robin hood effect." They found that in a multiple regression, with geographic region and public school attitudes as the predictors and willingness to break rules to help others as the criterion, the partial correlation between geographic location and willingness to break rules was 0.00 after controlling for public school attitudes. Conversely, after controlling for location, the partial correlation between public school attitudes and willingness to break rules was 0.50! Considering what we know about nonexperimental methods and the potential effects of error, which of the following statements is NOT a probable explanation for the observed patterns of partial correlations?

A) public school attitudes and willingness to break rules do, in fact, share common variation that is not shared by geographic location
B) geographic location and willingness to break rules do not, in fact, share any common variation after controlling for public school attitudes
C) after controlling for geographic location, the significant partial correlation between public school attitudes and willingness to break rules can be attributed to other factors, including measurement error, third variables, etc.
D) A and B
E) B and C
F) all of the above

Question 10

Researchers wanted to investigate the relationships between number of years of school completed, whether students have had primarily male, female, or equal proportions of male and female teachers, and academic aptitude. To encompass as representative a sample as possible, researchers recruited all 6th, 9th, and 12th grade cohorts from each of 150 schools around the country. To test the effects of number of years in school and teachers' gender on scholastic aptitude, which nonexperimental research design is most appropriate?

A) multiple regression
B) structural equation modeling
C) multilevel modeling
D) path analysis

Question 11

In a multilevel modeling design investigating the effects of geographic location, political orientation, and whether respondents attended public or private high schools, on attitudes toward proposed environmental regulations, the observed ICC (i.e., intraclass correlation) was .95. This value suggests that:

A) the researchers can probably assume that people from a given geographic location differed substantially and systematically from those in other locations; therefore, the nesting element of the design should not be ignored
B) the researchers can probably assume that people from a given geographic location did not differ substantially and systematically from those in other location; therefore, they could ignore the nesting element of the design
C) there is considerable error present between at least one predictor and the criterion
D) none of the above

Question 12

A multilevel modeling design recruited people who had lived in the same city their entire lives, and investigated the effects of geographic location, political orientation, number of children, and whether respondents attended the local public or private high school (there were only two in each town), on attitudes toward proposed environmental regulations. In this example, number of children is likely a _____(a)_____ predictor of attitudes, and geographic location is likely a _____(b)_____ predictor of attitudes.

A) Level 1; Level 2
B) Level 2; Level 3
C) Level 3; Level 1
D) Level 1; Level 3

Question 13

In nonexperimental methods, which of the following designs allows the simultaneous testing of multiple predictor variables on multiple criterion variables? (circle all that apply)

A) multiple regression
B) latent structural equation models
C) path analysis
D) multilevel modeling

Question 14

In structural equation modeling, exogenous variables __________, and endogenous variables __________. (choose the most accurate response)

A) are predicted by variables; predict variables but are not predicted by variables
B) predict variables but are not predicted by variables; are predicted by variables and can predict variables
C) are predicted by variables and can predict variables; predict variables
D) are variables that are not included in the theorized model; are variables that are included in the theoretical model

Question 15

In structural equation modeling, recursive models refer to _____(a)_____, and nonrecursive models refer to _____(b)_____.

A) models that allow causal paths to "back track" (i.e., bidirectional causation); models that do not allow for bidirectional causation
B) models where the specified paths "fit" the data well; models where the specified paths do not fit the data well
C) models that do not all causal paths to back track; models that allow causal paths to back track
D) models where the specified paths do not fit the data well; models where the specified paths fit the data well

Question 16

In structural equation modeling, the path between two latent factors possesses which of the following characteristics? (circle all that apply)

A) the extent to which latent factors are correlated
B) the strength of the relationship between underlying constructs after measurement error in their respective items has been removed
C) represents an element of the "measurement model"
D) represents an element of the "structural model"

Question 17

A study interested in gender differences in how men and women experience a treatment vs. a control would be an example of a __________. If the treatment instead was a scale assessing emotional intelligence, the study would be an example of a ____________.

A) Mixed Factorial design, nonexperimental design
B) Correlational design, factorial design
C) Blocked design, mixed factorial design
D) Nonexperimental design, correlational design
E) None of the above

Question 18

The Pearson product-moment correlation:

A) can be positive, negative, or curvilinear
B) represents the degree to which variation in X accompanies variation in Y
C) provides an estimate of the proportion of variance shared by X and Y
D) measures the degree of prediction error in a regression formula

Question 19

If a Pearson's product moment correlation is obtained between two measures with Cohen's coefficient alpha values of 0.30 and 0.40, then it is likely that:

A) there is no systematic relationship between the variables
B) a hidden third factor is responsible for the relationship between the two measures
C) the measures being used are invalid
D) the Pearson's product-moment correlation is small
E) measurement error is attenuating the association

Question 20

In multiple regression equation, regression weights:

A) are typically stable across samples of participants
B) refer to the degree to which a predictor and an outcome variable are correlated
C) are the proportion of variance in an outcome explained by a predictor
D) are used individually to explain variability in an outcome
E) maximize prediction of a given criterion variable

Question 21

Partial correlation - correct vs. incorrect interpretation?
A study predicting the quality of children's classroom experiences conducted across a number of public and private schools in a given jurisdiction would be most suited for which type of analysis?

A) correlation
B) multiple regression
C) multilevel modeling
D) structural equation modeling
E) none of the above

Question 22

Take the hypothesized relationship between similarity, reciprocity, and liking, where similarity predicts liking of another person, but because individuals assume that the person will like them back (reciprocity). The portion of the similarity-liking connection that is explained by reciprocity is an example of:

A) an indirect effect
B) an exogenous variable
C) a direct effect
D) a latent variable
E) a partial mediator

Question 23

Unlike path models, latent structural equation models:

A) are always recursive
B) use only measured variables to specify a set of relationships
C) estimate measurement error
D) cannot be used to predict exogenous variables
E) none of the above

Question 24

Which of the following is not desired when fitting a structural equation model:

A) a smaller difference between predicted and actual correlation values
B) over-identification
C) under-identification
D) a larger number of estimates explaining the correlations among a smaller set of variables
E) fewer estimates explaining correlations among a larger set of variables

Question 25

Often, researchers are interested in variables that cannot be experimentally manipulated (e.g., gender differences, personality differences), and how these variables influence a criterion (e.g., voting behavior, resistance to persuasion). Although we cannot determine causality between such variables, there are some benefits to nonexperimental designs. What is the major advantage to using nonexperimental designs that cannot be obtained from experimental studies where all levels of the independent variables are manipulated? In other words, what information about the relationship between predictor and criterion variables can we obtain in nonexperimental designs but not necessarily in experimental designs? Be sure to explain why this is so.

Question 26

In terms of nonexperimental methods, what is meant by the equation: Error = 1 - r² ? In your response, be sure to describe each element of the equation, why error would be computed by subtracting r² from 1, and what the "error" in this equation refers to.

Question 27

Two researchers hypothesized that people's short-term memory capacity would be related to their IQ score; however, when they computed the correlation between these two variables, the correlation was 0.00. What are three possible explanations for why they might have obtained a zero correlation?

Question 28

Researchers wanted to investigate the relationships between the number of years of school completed, whether students have had primarily male, female, or equal proportions of male and female teachers, and academic aptitude. To encompass as representative a sample as possible, researchers recruited all 6th, 9th, and 12th grade cohorts from each of 150 schools around the country. Using multilevel modeling, the researchers wanted to test the effects of number of years in school and teachers' gender on scholastic aptitude of individual students. How many levels would be present in this design? In your response, be sure to identify each nested level, and provide a brief explanation for why each level would be appropriate. Feel free to provide a diagram if it helps illustrate your response; however, do not substitute a diagram for your explanations.
*note to instructor: do not include this item and item 10 from the multiple choice section in the same exam

Question 29

For the following structural equation model, list the possible direct and indirect effects.

Question 30

Researchers wanted to investigate how the type of town or city in which people lived (e.g., rural area, large town, suburb, metropolis), the type of residence (i.e., single-family homes or apartment complexes), number of children, attitudes about civic engagement, and religiosity (i.e., how involved people are with their religion), predicted people's voting behavior. You recommended they use multilevel modeling to test the relationships among these variables - (a) explain why and how this was the right or wrong choice. What considerations should go into deciding what nonexperimental design to use? (b) Regardless of whether it was the correct or incorrect choice, outline how these variables would be organized in a hierarchical linear model, and explain why data would be organized in this way. After data were collected, the researchers changed their mind and analyzed this model using multiple regression. (c) Explain what sort of information would be lost, or why it would be inappropriate, to use multiple regression in this example.

Question 31

Suppose a friend of yours wanted to use a median split before conducting his or her analyses. What would you say to this friend? What issues would you urge him or her to consider?

Question 32

What are the key differences between correlational analysis and regression analysis?

Question 33

What are some research questions that might be best answered through the use of multi-level modeling?

Question 34

Define the terms "latent factor," "measurement model," and "disturbance." What role does each play in the structural equation modeling technique?

Question 35

Even though we cannot achieve random assignment to various demographic categories (e.g., we cannot randomly assign some participants to be male and some to be female), is there ever a situation where we would be able to make causal inferences about a demographic characteristic (e.g., sex, race/ethnicity, age, marital status, political orientation)? Why or why not? What would that situation be? What if you collected qualitative data, wherein many respondents spontaneously reported that they held a certain attitude because they were male [female] - could we conclude there is a causal relationship in that example?

Question 36

Given that path analyses and structural equation models are fundamentally correlational designs, do they mislead readers by suggesting a particular series of events occurs, usually from left to right, to explain the relationships between exogenous predictors, endogenous predictors, and criterion? Should these models instead be visualized in such a way that readers are less likely to interpret the model as "A leads to B, which leads to C, which leads to D?"