Deck 18: Interpreting Research Results

Question 1

The interpretation of research results can be divided into three broad steps. Describe those steps

Answer 1

No Answer

Question 2

What three questions should researchers answer when describing the results of a study?

Answer 2

No Answer

Question 3

Briefly explain the process of comparing the relationship a researcher observes in the data to the relationship predicted by the study's hypothesis.

Answer 3

No Answer

Question 4

What information does a researcher obtain from inferential statistics?

Question 5

What is the conventional criterion level for determining whether a relationship is statistically significant? What does this criterion level indicate?

Question 6

Distinguish between Type I and Type II errors. How does one determine the probability of making each type of error?

Question 7

Describe three of the criticisms that have been made of null hypothesis statistical testing (NHST). Why does NHST continued to be used despite these criticisms?

Question 8

According to your text, the validity of the entire chain of inference rests on conducting the correct statistical analysis. Explain why this is true.

Question 9

What is an omnibus statistical test? If such a test is significant, how does a researcher determine the meaning of the obtained results? How is this process similar or different when specific patterns of results are hypothesized?

Question 10

What is an effect size? What information does a researcher gain by computing an effect size? Can that information be used to compare results across research studies? Why or why not?

Question 11

Distinguish between statistical significance and practical significance. Why is each important to the inference process?

Question 12

Describe two situations where research results with a small effect size can have important practical significance.

Question 13

Distinguish between the logical positivist and the social constructionist view of knowledge.

Question 14

Name and describe two types of bias that can affect how research results are interpreted. Give an example of each.

Question 15

Define the naturalistic fallacy and give an example of it.

Question 16

Give two reasons why, in research comparing social groups (such as women and men), treating one group as the "control" group can lead to inferential errors.

Question 17

Assume you have found a significant relationship between your independent variable and your dependent variable. Explain why the conclusions drawn would differ depending on whether your dependent variable was measured an interval level or a ratio level.

Question 18

What is the fallacy of the mean? Why might it result in incorrect inferences about your statistical results?

Question 19

What conclusions can be drawn from a non-significant statistical test? What role does statistical power play?

Question 20

Explain how representing data visually in the form of a graph can aid in drawing valid inferences.

Question 21

Explain why researchers must be careful about drawing conclusions based on hypothetical constructs. Describe the roles played by the operational definitions of the independent variable and the dependent variable in this inference process.

Question 22

Distinguish between interpreting results descriptively and interpreting them evaluatively. Which is more problematic? Why?

Question 23

You have measured your participants' level of introversion and extroversion and your statistical results show that extroverts talk more in class than do introverts. What conclusions can you draw about the causal nature of or this relationship? Explain your answer.

Question 24

Explain why it is important to search for alternative explanations for the effects found in your research.

Question 25

Describe two ways in which statistical results that show a null relationship between the independent variable and the dependent variable are useful.

Question 26

How can null results be used to demonstrate that a set of research results are generalizable?

Question 27

Explain why there is prejudice against the null hypothesis in psychological research. What undesirable effects does this prejudice have?

Question 28

Your statistical analysis showed no significant difference between the experimental and control group. Name and describe four factors that should be considered when evaluating whether this result was due to Type II error.

Question 29

Name three criteria that are used to evaluate whether a null result should be accepted. What additional criteria are used to evaluate whether unexpected null results should be considered true.

Question 30

Explain why it is important to consider a study's results in the broader context of scientific knowledge. How does one go about doing so?

Question 31

Your study's results are inconsistent with those found in previous research. Explain the process you should use for comparing your results with those of previous research on your topic.

Question 32

What questions should you address when identifying the implications of your research findings for future research?

Question 33

Name three factors that you should consider when evaluating the implications of your research results. Why is each important?

Question 34

Once you have analyzed your data and have determined the nature of the relationship revealed by the data, the next step is to

A) decide what implications the results have for various theories of the phenomenon you are studying.
B) determine how well your results match the predictions made by your hypotheses.
C) determine the degree of practical significance of the results.
D) decide what practical applications the results might have.

Question 35

Preparing and using an expected outcome diagram for a study can help the researcher determine the

A) extent to which the relationships found in the study match the predictions made in the study's hypotheses.
B) degree of statistical significance of the research results.
C) degree of practical significance of the research results.
D) extent to which bias might be influencing interpretations of the research results.

Question 36

Inferential statistics are used to help the researcher

A) describe the results of a study in qualitative terms.
B) describe the size of the effect that the independent variable had on the dependent variable.
C) determine the degree of practical significance of the results.
D) decide if what appears to be a statistical relationship was likely due to chance.

Question 37

If the result of a statistical test is statistically significant (e.g., p < .01), then one can conclude that

A) the results of the research have practical importance.
B) the independent variable had a real (i.e., non-chance) effect on the dependent variable.
C) the results of the research were due to chance factors.
D) what appears to be a statistical relationship was likely due to chance.

Question 38

A Type I error consists of

A) proposing an incorrect hypothesis.
B) concluding from your sample data that the independent variable had no effect on the dependent variable when, in the population, the independent does have an effect.
C) concluding from your sample data that the independent variable had an effect on the dependent variable when, in the population, the independent does not have an effect.
D) using the wrong statistical test to analyze your data.

Question 39

Dr. Mischel concluded that her independent variable had an effect on the dependent variable; however, the independent does not have an effect in the population. Dr. Mischel

A) made a Type I error.
B) made a Type II error.
C) had low statistical power.
D) both a and c.

Question 40

A Type II error consists of

A) proposing an incorrect hypothesis.
B) concluding from your sample data that the independent variable had no effect on the dependent variable when, in the population, the independent does have an effect.
C) concluding from your sample data that the independent variable had an effect on the dependent variable when, in the population, the independent does not have an effect.
D) using the wrong statistical test to analyze your data.

Question 41

Dr. Mischel concluded that her independent variable did not have effect on the dependent variable when, in the population, the independent did have an effect. Dr. Mischel

A) made a Type I error.
B) made a Type II error.
C) used an inappropriately large sample size.
D) both b and c

Question 42

Low statistical power in a study is a problem because

A) it increases the probability of making a Type I error.
B) it increases the probability of making a Type II error.
C) if the results of the statistical test are not significant, one cannot draw any firm conclusions about whether or not the hypothesis was supported.
D) both b and c.

Question 43

Which of the following concerns have been raised over researchers' use of null hypothesis statistical testing?

A) Interpreting p-values using terms such as "highly significant" is misleading because p-values provide no information about effect sizes.
B) It is incorrect to assume that a study with a lower p-value is likely to be replicated.
C) Non-significant p-values can mean that the null hypothesis is correct, but they can also result from having with low statistical power.
D) all of the above

Question 44

You conduct an experiment and get the following means, with F(2, 48) = 11.56, p < .001: Condition A = 5.8, Condition B = 2.6, and Condition C = 2.8. Based on this information, you can correctly conclude that

A) all three means are significantly different from one another.
B) Condition A differs significantly from Conditions B and C.
C) none of the conditions differ from any of the others.
D) none of the above: you need additional information before drawing any conclusions about differences between means.

Question 45

If you conduct an experiment with three or more conditions and have specific hypotheses about the differences that should be found between conditions, you should conduct

A) only an omnibus ANOVA (F-test).
B) planned comparisons of condition means.
C) an omnibus ANOVA followed by post-hoc comparisons of condition means.
D) comparisons of condition variances rather than condition means.

Question 46

The value of the overall F-test in the analysis of variance is called a(n)

A) effect size.
B) post-hoc test.
C) omnibus test.
D) zero order test.

Question 47

The term effect size refers to the

A) strength of the relationship between an independent variable and a dependent variable.
B) theoretical importance of the results of a study.
C) practical importance of the results of a study.
D) degree of statistical power of a study.

Question 48

Looking at research results in terms of effect size is advantageous because effect sizes

A) indicate the strength of the relationship between the independent and the dependent variable.
B) are standardized, which allows for direct comparison between studies that use different operational definitions.
C) can be used to calculate the optimal sample size for a subsequent study.
D) fulfill all of the above purposes.

Question 49

Which of the following statements about effect sizes is TRUE?

A) A large effect size indicates a high level of practical significance.
B) The concept of effect size is a statistical abstraction that has no practical application.
C) Small effect sizes can be important under the right circumstances.
D) Effect sizes are useful in estimating the probability of making a Type I error but not in estimating the probability of making a Type II error.

Question 50

The practical significance of an effect

A) is directly proportional to the size of the effect: larger effects have more practical significance than smaller effects.
B) is inversely proportional to the p-value of the statistical test of the effect: smaller p-values indicate greater practical significance.
C) depends on how effect is going to be used: the same effect can have more or less practical significance depending on its intended use.
D) is usually not an important issue in psychological research.

Question 51

Which of the following statements about practical significance is TRUE?

A) A small effect size can have practical significance when it is applied to a large population.
B) An effect can be clinically significant without being practically significant.
C) Smaller p-values for statistical tests indicate higher degrees of practical significance.
D) The term practical significance is misnamed; it really refers to how important a theory is.

Question 52

According to the social constructionist viewpoint, scientific knowledge

A) consists of a set of absolute facts that, once discovered, remain unchanged and unchangeable.
B) consists of a set of interpretations of data, so that different people can make different interpretations of the same data.
C) can be either absolute or a matter of interpretation depending on the research methods used to collect the data.
D) is an unattainable ideal; we can never really "know" anything.

Question 53

According to the logical positivist viewpoint, scientific knowledge

A) is an unattainable ideal; we can never really "know" anything.
B) consists of a set of interpretations of data, so that different people can make different interpretations of the same data.
C) consists of a set of absolute facts that, once discovered, remain unchanged and unchangeable.
D) can be either absolute or a matter of interpretation depending on the research methods used to collect the data.

Question 54

________ biases in the interpretation of results derive from the scientific perspectives researchers take on the problems they investigate.

A) Personal
B) Statistical
C) Empirical
D) Theoretical

Question 55

Being overly focused on single theory when interpreting the results of a study can lead the researcher to

A) make a Type II error.
B) overlook alternative explanations for the results.
C) mistake statistical significance for practical significance.
D) fall victim to the theoretical interpretation fallacy.

Question 56

The naturalistic fallacy is

A) the belief that because something has a biological basis, it is natural and unchangeable.
B) another term for the logical positivist view on scientific knowledge.
C) the belief that the mean group score on a variable is characteristic of all members of the group.
D) the belief that the results of research conducted in laboratory settings automatically generalize to natural settings.

Question 57

________ is belief that because something has a biological basis, it is natural and unchangeable.

A) The naturalistic fallacy
B) The fallacy of the mean
C) Foundationalism
D) Structuralism

Question 58

Errors in drawing conclusions from research results that compare social groups (such as women and men) include

A) assuming that any group differences found stem from biological factors without data to support that assumption.
B) treating one of the groups as the "control" group against which findings from the other group are evaluated.
C) applying the results of research conducted with one group to members of another group without evidence to support that generalization.
D) all of the above.

Question 59

Using valid and reliable measures, a researcher finds that men score higher than women on self-esteem. The researcher concludes that women are deficient in self-esteem and suggests interventions to correct the problem. The researcher's conclusions

A) are reasonable given the data.
B) inappropriately use men's scores as the standard for what constitutes a desirable level of self-esteem.
C) represent a Type II error.
D) are based on the fallacy of the mean.

Question 60

In describing research results, statements such as "The mean experimental group score improved twice as much as the mean control group score" should be made only when the

A) dependent variable is measured at the ratio level.
B) study that produced the results was a true experiment.
C) research participants constitute a representative sample of the population.
D) study meets all of the above criteria.

Question 61

The term fallacy of the mean refers to

A) drawing ratio-level conclusions from mean scores that were obtained at a lower level of measurement.
B) implicitly assuming that the mean group score on a variable is characteristic of all members of the group.
C) concluding that a statistically significant omnibus F-test for a 3-condition experiment means that each group mean is different from each of the other two means.
D) using means to describe the results of a study when it would have been more appropriate to use medians.

Question 62

Implicitly assuming that the mean group score on a variable is characteristic of all members of the group is

A) a fallacy of the mean.
B) the tragedy of the commons.
C) a statistical artifact.
D) a naturalistic fallacy.

Question 63

Visual representations of data can be useful because they

A) can suggest alternative interpretations of the data.
B) show the exact form of that a statistically significant interaction takes.
C) can indicate when outliers might be influencing the results.
D) can do all of the above.

Question 64

You find a journal article with the title "The Effect of Self-Esteem on Academic Achievement." This title is misleading because

A) it is an example of the naturalistic fallacy.
B) it implies the existence of a causal relationship from correlational data.
C) shows a bias against people who have low self-esteem.
D) of all of the above.

Question 65

Errors can occur in interpreting research results when

A) the operational definitions used in the research are of uncertain validity.
B) results are stated in evaluative rather than descriptive terms.
C) the mean group score on a variable is assumed to be characteristic of all members of the group.
D) any of the above situations occur.

Question 66

Errors can occur in interpreting research results when

A) the researchers do not consider possible alternative explanations for the results.
B) researchers generalize their results without supporting evidence.
C) the mean group score on a variable is assumed to be characteristic of all members of the group.
D) any of the above situations occur.

Question 67

The term null results refers to a research outcome

A) that has little practical significance.
B) in which the result of the statistical test is significant but the effect size is small.
C) in which the result of the statistical test is not significant.
D) that fits any of these situations.

Question 68

Researchers might test the null hypothesis as a way to

A) rule out an alternative explanation for the effect of an independent variable.
B) assess the discriminant validity of a measure.
C) test the theoretical proposition that there is no relationship between two variables.
D) do any of the above.

Question 69

Prejudice against the null hypothesis exists because

A) researchers tend to think of theory testing in terms of finding hypothesized relationships rather than not finding relationships.
B) the meaning of null results can be ambiguous because they could represent Type II errors.
C) researchers see null results as sources of disappoint?ment rather than as sources of information.
D) of all of the above

Question 70

Possible sources of Type II errors include

A) the use of operational definitions that are of low validity.
B) weak manipulations of independent variables.
C) poorly designed manipulations of the independent variable.
D) all of the above.

Question 71

Possible sources of Type II errors include

A) not manipulating the independent variable properly and consistently.
B) the use of insensitive measures of the dependent variable.
C) a restriction in range of measures used in correlational research.
D) all of the above.

Question 72

Possible sources of Type II errors include

A) using a manipulation of the independent that is too strong.
B) the use of ratio level measurement of the dependent variable.
C) a restriction in range of measures used in correlational research.
D) all of the above.

Question 73

Which of the following is NOT a possible source of Type II errors?

A) high statistical power
B) overlooking the possibility of a curvilinear relationship between the independent and dependent variables
C) overlooking the possibility of a moderating variable having an influence on the results
D) the use of insensitive measures of the dependent variable

Question 74

It can be appropriate to accept null results as valid if

A) the statistical power of the study was very high.
B) the study produced a small effect size.
C) the results are confirmed through replication.
D) all of the above apply.

Question 75

When identifying the implications of the results of a study, one should take into consideration

A) how the results compare to the results of previous research on the topic.
B) the ways in which research procedures may have affected the results.
C) the ways in which the theory tested in the research is related to other theories.
D) all of the above.