Deck 15: The Role of Research Methods in Making Sense of Numbers

Question 1

What does the first of the "W's," "What," refer to?

A) topic of study
B) location
C) time period
D) type of case

Answer 1

A

Question 2

What does the honorary W, "How", refer to?

A) time period of study
B) interpreting results
C) study design and methods
D) location of study

Answer 2

C

Question 3

A researcher is conducting a study on the inventory needed by rental car agencies to meet daily demand for car types. In her study design, she decides on a sample of three car rental agencies for inclusion in the study. Which W or honorary W does this sampling task represent?

A) how
B) how many
C) what
D) where

Answer 3

B

Question 4

What does "substantive significance" mean in the context of research?

A) significance level of a test
B) validity of a sample
C) p value of a significance test
D) practical importance of a result

Question 5

What does a p value measure?

A) probability of wrongly concluding the true parameter is 0
B) the significance level of a research test
C) probability of the alternative hypothesis being true
D) the correlation between two levels

Question 6

Which of the following cannot be confirmed by a statistical significance test?

A) accept the null hypothesis
B) cause and effect relationship
C) population parameter is nonzero
D) reject the alternative hypothesis

Question 7

In which of the following cases, should statistical significance of an association not be the reason for an intervention?

A) the dependent variable is modifiable
B) the size of the association is large
C) direction of association is known
D) the independent variable cannot be modified

Question 8

Which of the following lists the dimensions of "importance" of a research result?

A) statistical significance and substantive significance
B) practical importance, statistical significance, internal and external validity
C) measurement level, causality, and practical importance
D) benchmarks, measurement level, internal validity, and central tendency

Question 9

What criteria is used to assess the practical importance of a research result?

A) confidence limits
B) mechanism
C) effect size
D) p value

Question 10

Which of the following criteria is used to assess the internal validity of a research result?

A) empirical association
B) effect size
C) p value
D) measurement value

Question 11

A regression analysis was conducted with dependent variable A and independent variables B and C. The regression coefficients fell within confidence intervals that did not equal 0. Moreover, the coefficients had p values less than 0.0001. Which dimension of "importance" do these results satisfy?

A) cause-and-effect
B) statistical significance
C) benchmarking
D) effect size

Question 12

What factor(s) does the criteria "empirical association" depend on in a research study?

A) cross-sectional study design
B) experimental study design
C) consistency of direction and size
D) sampling method

Question 13

What does the criteria "Nonspuriousness" refer to, in checking for internal validity of a research result?

A) a random sample using probability methods
B) a p value less than .05 for a coefficient
C) separate multiple topics into different studies
D) true association after accounting for confounders

Question 14

In a research study, a random sample was taken of a binary dependent variable and three independent variables hypothesized to be associated with the dependent variable. A statistical model was developed on the sample data, and the model was then applied to a validation sample. The results of the model were checked to see if the variable distributions were similar to the population from which the samples were taken. This check is related to which dimension of determining importance of results?

A) measurement level
B) external validity
C) internal validity
D) statistical significance

Question 15

In determining practical importance of a result, which measure is referred to, along with effect size?

A) variation
B) mean
C) standard error
D) confidence level

Question 16

How is the sample size related to the measure of spread in a research study?

A) weakly related
B) positively related
C) inversely related
D) not related

Question 17

What is the relationship between sample size and width of confidence intervals?

A) inversely related
B) not related
C) positively related
D) weakly related

Question 18

An empirical association is shown to be stronger due to which of the following factor(s)?

A) wider confidence limits
B) larger effect sizes and smaller p values
C) larger measures of dispersion
D) smaller margin of error and confidence level

Question 19

In addition to empirical association, what are the two other criteria needed to assess a cause-and-effect relationship between a dependent and an independent variable?

A) measurement level and p value
B) effect size and value
C) confidence limits and standard error
D) mechanism and nonspuriousness

Question 20

A study on median house prices in Middlesex county used a random sample of 200 houses from three zip codes of the county. Which dimension of 'importance' of study results would this sampling affect? Choose the best answer.

A) internal validity
B) statistical significance
C) external validity
D) practical importance

Question 21

Under which type of studies is the chance of confounding reduced or less likely?

A) observational
B) experimental
C) cross-sectional
D) longitudinal

Question 22

In deciding on units or categories of a dependent variable for a study, which of the following would serve as a guiding principle?

A) assess the "W's" of the study
B) identify appropriate benchmarks
C) review prior literature on the variable
D) assess the context of the study

Question 23

If a table template already exists for presentation of results, how should the researcher design the study? Choose the best answer.

A) design data collection as per template
B) identify the measures to be presented
C) complement the table with a chart
D) assess the "W's" as per the table

Question 24

Depending on whether the study is cross-sectional, prospective, or retrospective, what factor should be kept in mind for the task of collecting data?

A) number of independent variables
B) appropriate temporal design
C) statistical methods to be used
D) units of the variables in the study

Question 25

In the case of observational studies with multiple independent variables, what factor should be kept in mind during the data collection task to ensure stronger empirical association?

A) units of the variables in the study
B) type of variable to be used
C) temporal design of the study
D) information on potential confounders

Question 26

A 'substantive significance of an estimate is assessed by its p value and confidence limits.

Question 27

Statistical significance should be complemented by practical importance in determining the true importance of an estimate.

Question 28

In inferential statistics, the researcher would like the p value to be as large as possible.

Question 29

The results of a research study showed the regression coefficient of the variable GENDER to have a p value of .76 at the 95% confidence level. Hence, GENDER could be concluded to be statistically significant.

Question 30

The R² value of association between dependent variable A and independent variable B was found to be 0.61, with a p value of .0001 at the 95% confidence level. It can be concluded that variable B has a causal effect on variable

Question 31

In many instances, an association between an outcome and an independent variable could be confounded by another independent variable.

Question 32

In some cases, it may be possible to test an association between two variables by conducting a "true experiment".

Question 33

A statistical significance test of association can indicate the direction of the association.

Question 34

In some cases, the magnitude of a statistical significance test cannot tell the researcher whether the effect is meaningful in the real world.

Question 35

'External validity' is one of the four dimensions used for assessing importance of research results.

Question 36

What are the criteria for assessing practical meaning of a research result? Explain what factors these criteria depend on.

Question 37

Provide an example of how statistical significance of an estimate is determined.

Question 38

Discuss the quantitative criteria needed to determine internal validity of a study. Illustrate with the help of an example.

Question 39

What type of study design should be implemented in order to assess external validity of a study? Explain your answer.

Question 40

Explain in your own words, how sample size affects p values for hypothesis testing.

Question 41

Provide an example of a dose-response relationship, in the context of assessing internal validity of research results.

Question 42

With the help of an example, explain what is nonspuriousness of an association between two variables.

Question 43

What type of study design is conducive to assessing the mechanism linking two variables in a study? Give an example.

Question 44

Discuss the sampling methods required in ensuring external validity of a study. Give an example of one such method.

Question 45

Explain in your own words, why statistical significance does not necessarily lead to practical importance of a research result.

Question 46

Refer to the research article you chose in answering the essay questions for Chapter 13. Comment on how the authors incorporated practical importance, in addition to statistical significance, in interpreting their research results.

Question 47

Refer to the research article you used in Question 1. Discuss the usage of the four dimensions to determine overall importance of results in the article. Were all the four dimensions satisfied in the study? Explain your answer.

Question 48

Refer to the research article you used in Question 1. Comment on the study design, sample size, and the W's related to criteria used by the authors in assessing overall importance of their research results.