Deck 19: Literature Reviewing

Question 1

Distinguish between conceptual and integrative literature reviews.

Answer 1

No Answer

Question 2

Describe the three approaches to theory researchers can take when conducting a conceptual literature review.

Answer 2

No Answer

Question 3

Describe the major purposes of integrative literature reviews.

Answer 3

No Answer

Question 4

Distinguish between a main effects analysis and a moderator variables analysis in an integrative literature review. Why is each important?

Question 5

What strategies can researchers use to identify potential moderator variables in an integrative literature review?

Question 6

What factors should researchers consider in defining the scope of the question they will address in an integrative literature review?

Question 7

Distinguish between the hypothesis-testing approach and the exploratory approach to conducting an integrative literature review.

Question 8

What steps should a researcher take to locate the relevant research for an integrative literature review?

Question 9

What is the fugitive or grey research literature? Explain why it is important to include studies from that literature in an integrative literature review.

Question 10

Explain the limitations of relying only on studies identified by a computerized database search. What additional strategies can researchers use to identify studies for an integrative literature review?

Question 11

Describe the three major options to consider when deciding which studies to include in an integrative literature review. Describe the advantages and disadvantages of each option.

Question 12

Name the four types of validity that should be assessed when deciding which studies to include in an integrative literature review.

Question 13

Explain how the theoretical validity of the studies included in an integrative literature review can affect the outcome of the review.

Question 14

Define design flaw analysis. What is its purpose? How does one conduct a design flaw analysis?

Question 15

Explain how and why stratified sampling could be used to select studies for an integrative literature review.

Question 16

Distinguish among narrative literature reviews, meta-analysis, and meta-synthesis. What are the advantages and disadvantages of each approach?

Question 17

Explain why it is important to ensure that your integrative literature review is focused at the appropriate level of analysis. What are the risks of not doing so?

Question 18

Explain the "apples and oranges" problem in integrative literature reviewing.

Question 19

What is an effect size and what does it indicate? What type of effect size is computed when the independent variable is categorical? What type is computed when the independent variable is continuous?

Question 20

Explain the vote counting approach to operationally defining study outcome for an integrative literature review. Describes its advantages and limitations as a technique for integrative literature reviewing.

Question 21

What is a narrative literature review? Describe the advantages and limitations of narrative literature reviews.

Question 22

What is a meta-analysis? Describe the advantages and limitations of meta-analysis.

Question 23

Explain why some researchers consider the results of a meta-analysis to be more precise than the results of a narrative literature review. Do you agree or disagree with this perspective? Explain your reasoning.

Question 24

Explain why some researchers consider the results of a meta-analysis to be more objective than the results of a narrative literature review. Do you agree or disagree with this perspective? Explain your reasoning.

Question 25

What is a meta-synthesis? Describe the advantages and limitations of meta-synthesis.

Question 26

Compare and contrast meta-analysis and meta-synthesis: In what ways are they similar and in what ways do they differ?

Question 27

What is best evidence literature reviewing? What steps should a researcher follow in conducting this type of review?

Question 28

List the types of judgment calls a researcher must make when conducting an integrative literature review. Explain how those decisions can affect the conclusions drawn from that review.

Question 29

If a literature reviewer identifies a variable that moderates study outcomes, should that variable be considered a cause of the outcome differences? Why or why not?

Question 30

Name three factors that should be considered when evaluating the quality of the research question that was formulated for a literature review.

Question 31

What factors affect the quality of the data collected for a literature review?

Question 32

What factors should a reader consider when evaluating the effectiveness of the data analysis used in a literature review?

Question 33

What factors should a reader consider when evaluating whether a literature reviewer appropriately interpreted her or his results?

Question 34

Which of the following is NOT a goal of literature reviews?

A) determining how much empirical support a hypothesis has
B) identifying possible causes of inconsistencies in the results of studies that test the same hypothesis
C) identifying the implications that the research reviewed has for theory, research, and application
D) presenting new research data bearing on a hypothesis

Question 35

Conceptual literature reviews focus on

A) the effect that an independent variable has on a dependent variable.
B) factors that moderate the effect of an independent variable.
C) broad theoretical and methodological issues.
D) the conditions under which the effects of an independent variable are found.

Question 36

Which of the following is NOT a primary focus of an integrative literature review?

A) the effect that an independent variable has on a dependent variable.
B) factors that moderate the effect of an independent variable.
C) broad theoretical and methodological issues.
D) the conditions under which the effects of an independent variable are found.

Question 37

Which of the following would NOT be an appropriate topic for a conceptual literature review?

A) examining the results of relevant studies to determine the extent to which attitudes can predict behavior
B) collating information from existing theories and empirical research to develop a new theory about anxiety
C) identifying problems or weaknesses in existing research testing Sternberg's Triangular Theory of Love
D) examining the literature on Baddeley's model of working memory to evaluate the validity of the theory

Question 38

In primary research, the researcher

A) summarizes the results of previous research.
B) collects data from research participants.
C) focuses on the most important issues raised by a theory.
D) evaluates the validity of theories though the use of existing data.

Question 39

In integrative literatures, the results of existing studies are analogous to ________ in primary research.

A) independent variables
B) dependent variables
C) participants
D) operational definitions

Question 40

Policy makers are most likely to make use of the results of integrative literature reviews that

A) identify the moderator variables that affect the relationship between independent and dependent variables.
B) identify the estimator variables that mediate the effects of independent variables on dependent variables.
C) focus on the main effects that independent variables have, with less focus on moderating and mediating variables.
D) focus on the theoretical implications of the research being reviewed.

Question 41

Integrative literature reviews can be used to assess

A) the average effect that an independent variable has on a dependent variable.
B) the ecological validity of a body of research.
C) the generalizability of the effect an independent variable has on a dependent variable.
D) all of the above.

Question 42

To identify potential moderator variables for an integrative literature review one can

A) see if the theory that guided the research being reviewed includes any moderating variables.
B) look at previous research on the topic to see if any moderator variables were found in those studies.
C) use logic and one's knowledge of existing theory and research to identify potential moderator variables that have been overlooked.
D) do any of the above.

Question 43

The scope of a literature review refers to

A) the number and types of issues the review includes.
B) the intended length of the manuscript reporting the results of the review.
C) whether the research question guiding the review is based on theory or on results of prior research.
D) the audience to which the results of the review are directed (such as other researchers, policy makers, practitioners).

Question 44

In the ________ approach to conducting an integrative literature review, the researcher searches the relevant literature to determine which independent variables consistently produce a result.

A) hypothesis testing
B) exploratory
C) qualitative
D) policy capturing

Question 45

Dr. Phillips conducts a review that tests a specific set of propositions proposed by a theory they are interested in. She is taking the ________ approach to conducting a literature review.

A) exploratory
B) quantitative
C) policy capturing
D) hypothesis testing

Question 46

The ________ approach to literature reviewing can be used to identify important independent variables that can be used to develop new theories.

A) hypothesis testing
B) policy capturing
C) exploratory
D) qualitative

Question 47

The term fugitive literature or grey literature refers to research reports

A) that exist in forms, such as technical reports, that are often not covered by the more commonly used indexes.
B) published in mainstream journals but are not often cited by other researchers.
C) published prior to the time that indexes to the research literature were established and so are difficult to locate.
D) that describe studies that appear to be low in internal validity.

Question 48

When searching for studies to include in a literature review, you should

A) conduct computerized searchers of appropriate indexes using relevant search terms.
B) check the reference lists of articles already identified for additional relevant articles.
C) examine issues of journals that publish research on your topic.
D) do all of the above

Question 49

Dr. Sowell plans to find all the research that exists on the topic of his literature review. Why might he decide otherwise?

A) He might find it difficult or impossible to locate all relevant studies.
B) He might find that including all relevant studies gives equal importance to both highly valid and poor research.
C) There might be so many studies, it would be impossible for him to include all of them.
D) Any of the above constraints might apply.

Question 50

Which of the following is an advantage of including only published studies in a literature review?

A) Studies that find null results are often not published, preventing such low-information studies from becoming a distraction.
B) The peer review process provides a check on the internal validity of published studies.
C) The information in published studies is generally more useful than information in unpublished research.
D) Published studies are more likely to be based on results that contradict accepted theories.

Question 51

Which of the following statements about using only published studies in literature reviews is FALSE?

A) Because of the bias against publishing null results, published research might overestimate the effects of an independent variable.
B) Journals tend to publish studies that support accepted theories and not publish studies that contradict those theories.
C) Few published studies provide enough methodological information to provide an accurate picture of how the research was conducted.
D) The peer review process provides a check on the internal validity of published studies.

Question 52

A study exhibits theoretical validity to the extent that the

A) study appears to be valid when it is actually of low validity.
B) conditions under which the study was conducted match the conditions that the theory being tested say are necessary for the theory to work properly.
C) researchers can confidently say that the independent variable was the only possible cause of the observed effects.
D) conditions under which the study was conducted match the natural situation in which the results of the study are to be applied.

Question 53

One should evaluate a study's ________ validity when deciding whether to include it in a literature review.

A) internal
B) ecological
C) theoretical
D) all of the above

Question 54

When choosing which studies to include in a literature review, one should always EXCLUDE studies that

A) have such low internal validity that they cannot provide any useful information.
B) produced results that contradict the hypothesis on which you based your research review.
C) were not peer reviewed.
D) have any of the above characteristics.

Question 55

The purpose of design flaw analysis is to

A) seek the advice of an expert who can check a literature review for design errors.
B) ensure that only studies that provided appropriate effect size information were included in the analysis.
C) provide evidence to the Institutional Review Board that the meta-analysis will be conducted in an ethical manner.
D) look for threats to internal validity that exist in all or most of the studies being reviewed.

Question 56

A design flaw analysis looks for

A) threats to internal validity.
B) threats to external validity.
C) whether a literature review is comprehensive.
D) all of the above

Question 57

Dr. Swensen finds that the number of studies on her topic is so large that including all of them in a review would be difficult. What could she do to reduce the number of studies?

A) Classify the studies in terms of relevant characteristics and randomly choose studies from each category.
B) Have experts on the topic choose the best and most relevant studies for the review.
C) do either a or b.
D) do neither a or b: she should include all relevant studies.

Question 58

The ________ approach to literature reviewing takes a subjective approach to integrating the results of studies.

A) hypothesis-testing
B) narrative
C) meta-synthesis
D) meta-analytic

Question 59

The ________ approach to literature reviewing takes a quantitative approach to integrating the results of studies.

A) hypothesis-testing
B) narrative
C) meta-synthesis
D) meta-analytic

Question 60

The ________ approach to literature reviewing is generally best suited to addressing conceptual questions such as theory development.

A) narrative
B) meta-synthesis
C) internal
D) meta-analytic

Question 61

The ________ approach to literature reviewing uses a set of structured techniques to integrate the results of qualitative studies.

A) hypothesis-testing
B) narrative
C) meta-synthesis
D) meta-analytic

Question 62

In literature reviewing, the issue of "mixing apples and oranges" refers to

A) making inappropriate comparisons across different types of moderator variables.
B) not including a mediating variable when doing so would provide clearer results.
C) inappropriately combining results of studies that use different independent variables, dependent variables, or other characteristics of the studies being reviewed.
D) including studies that did and did not have significant results.

Question 63

Dr. Cameron decided to combine different types of variables into a single category for a literature review. This decision is LEAST justifiable if the studies

A) used different types of settings, such as laboratory settings and natural settings.
B) used different types of dependent variables such as self-report and behavioral observation.
C) provided effect sizes based on different types of statistics such as the F-test and the t-test.
D) provided effect sizes based on different types of research designs such as experimental and correlational.

Question 64

What is the best way to operationally define study outcome for an integrative literature review?

A) Make subjective evaluations of study outcome.
B) Classify studies as either supporting or not supporting the hypothesis being investigated.
C) Use multiple outcome categories to classify studies as to the degree to which the results support the hypothesis being investigated.
D) Estimate the size of the effect the independent variable had on the dependent variable.

Question 65

The d statistic represents the

A) proportion of studies in a literature review that found statistically significant results supporting the hypothesis being reviewed.
B) size of the difference between conditions of an independent variable expressed in standard deviation units.
C) percentage of variance in a dependent variable accounted for by an independent variable.
D) percentage of participants in the experimental group whose scores on the dependent variable exceeded the mean score of the control group.

Question 66

Which of the following statistics is also an effect size indicator?

A) F
B) t
C) r
D) p

Question 67

A major drawback to using the vote-counting approach to integrate the results of research is that

A) the statistical significance of the results of a study is dependent on sample size.
B) the information needed to classify study results is not always available in the research report.
C) studies with very high statistical power might be incorrectly classified as not supporting the hypothesis being investigated.
D) it can only be used with studies that used the qualitative research paradigm.

Question 68

Which of the following statements about the relative strengths and limitations of narrative literature reviews and meta-analyses is TRUE?

A) Narrative literature reviews are better for addressing broad theoretical questions.
B) Narrative literature reviews are less vulnerable to Type II errors in the studies reviewed.
C) Meta-analyses can be conducted when there are only a few studies available for review whereas narrative literature reviews require a larger number of studies.
D) Meta-analyses are less efficient when a large number of studies must be included in the literature review.

Question 69

Which of the following statements about the relative strengths and limitations of narrative literature reviews and meta-analyses is TRUE?

A) Meta-analyses can include the results of quantitative studies whereas narrative literature reviews cannot.
B) Narrative literature reviews are less vulnerable to Type I errors caused by publication biases.
C) Meta-analyses are more efficient when a large number of studies must be included in the literature review.
D) Narrative literature reviews provide more precise conclusions than meta-analyses.

Question 70

Which of the following is NOT a factor that limits the precision with which meta-analysis can estimate a mean effect size?

A) The bias against publishing null results might inflate the Type I error rate in the set of studies being reviewed.
B) The mean effect size can be sensitive to the effect of outliers.
C) There are no generally accepted statistical techniques for conducting meta-analyses.
D) High levels of precision require effect sizes from a large number of studies which may not always be available.

Question 71

Researchers who use meta-analysis as a technique for integrating the results of studies must make a number of subjective judgments. These include which of the following?

A) defining the scope of the literature review.
B) establishing criteria for deciding which studies to include in the review.
C) interpreting the meaning of the results of the review.
D) all of the above.

Question 72

As a literature review technique, meta-analysis has a number of limitations. These include which of the following?

A) the inability to include the results of qualitative studies.
B) the fact that not all studies provide the information required to calculate effect sizes, which are the basis of meta-analysis.
C) difficulty in finding the data necessary to test interactions that are predicted by theories.
D) all of the above.

Question 73

Meta-synthesis and meta-analysis are similar in that users of both techniques

A) have specific sets of rules for integrating the results of the studies reviewed.
B) strive to be transparent about their methods and the decisions they make.
C) need to have a minimum of 10 to 15 studies in order to draw strong conclusions from their analyses.
D) all of the above.

Question 74

A major difference between meta-analysis and meta-synthesis is that

A) the goal of meta-analysis is to make precise estimates of the effect of an independent variable, whereas the goal of meta-synthesis is to identify broad themes that characterize the process being studied.
B) meta-analysis takes a systematic approach to integrating the results of studies, whereas meta-synthesis takes a subjective approach.
C) researchers who use meta-analysis are transparent about their methods and the decisions they make, researchers who use meta-synthesis see no need for such transparency.
D) all of the above.

Question 75

________ literature reviews combine elements of narrative literature reviews, meta-analysis, and meta-synthesis.

A) Integrative
B) Best evidence
C) Theoretically valid
D) Policy oriented

Question 76

Which of the following statements about interpreting the results of a literature review is/are TRUE?

A) A main effect for a moderator variable in a literature review is equivalent to an interaction in primary research.
B) Different reviews of studies testing the same hypothesis can reach different conclusions because of decisions made by the researchers who conducted the reviews.
C) Any effect found for a moderator variable is a correlational relationship, not a causal relationship.
D) All of the above statements are true.

Question 77

Which of the following statements about interpreting the results of a meta-analysis is/are FALSE?

A) The results of a meta-analysis represent the average effect found across a number of studies, so it is not necessary to think about the effects of publication biases.
B) A main effect for a moderator variable in a meta-analysis is equivalent to an interaction in primary research.
C) Any effect found for a moderator variable is a correlational relationship, not a causal relationship.
D) Because meta-analysis is a quantitative technique, the judgment calls made during the analysis are not relevant to interpreting the results.