Deck 6: Evaluating the Merits of Quantitative Research Studies in Kinesiology

Question 1

What type of validity refers to the extent to which a researcher's claim that any change in an outcome is a result of her or his intervention and not the result of other factors?

A) Construct validity
B) Logical validity
C) External validity
D) Internal validity

Answer 1

D

Question 2

In yesterday's class lecture, Dr. Walker highlighted his previous experimental studies showing that an individual's work satisfaction is more important than her or his leisure satisfaction in explaining general life satisfaction. He also stated that the findings come from studies he conducted in the early 1980s in the United States. What type of validity is specifically at risk if Dr. Walker interpreted the findings from his studies as applying to today's context?

A) Logical validity
B) Internal validity
C) Construct validity
D) External validity

Answer 2

D

Question 3

Which of the following statements is true?

A) If a researcher was looking to test construct validity by showing how a measure can discriminate itself from other non-related constructs, support for this evidence would be demonstrated by strong negative correlations.
B) Construct validity is all-or-none; a measure is either construct valid or it is not.
C) If a measure changes from its original form (e.g., changing a balance test protocol), it is important to test the construct validity of the modified measure.
D) Logical validity is best tested via the use of statistics.

Answer 3

C

Question 4

Which kind of validity refers to the quality of researchers' arguments, their application of theory to support the needs for the study, and the appropriate interpretation of results based on the data?

A) Internal validity
B) Construct validity
C) Logical validity
D) External validity

Question 5

Does a correlation in the magnitude of r = .36, between a measure (Measure 1) and another measure (Measure 2) of a similar construct, support the construct validity of the first measure (Measure 1)?

A) Yes
B) No
C) It depends, most importantly, on the specific content of both measures
D) It depends, most importantly, on whether or not the correlation is for two physical activity measures

Question 6

Which of the following statements is true?

A) A measure of a variable must be both consistent and accurate to support construct validity.
B) A nomological network refers to a researcher's ontological and epistemological beliefs.
C) Construct validity is all-or-none; a measure is either construct valid or it is not.
D) The inclusion of WEIRD samples in research is one of the best ways to reduce threats to external validity.

Question 7

In contrast to experimental study designs, which types of validity evidence are not relevant to correlational study designs?

A) Logical validity and internal validity
B) External validity and construct validity
C) Internal validity and external validity
D) Construct validity and internal validity

Question 8

What type of validity used to evaluate the merits of experimental research studies most closely aligns with the concept of generalizability, the latter of which can be used to evaluate the merits of non-experimental research studies?

A) Internal validity
B) External validity
C) Logical validity
D) Construct validity

Question 9

The CONSORT 2010 statement was developed to assist researchers and readers with which of the following?

A) Randomized control trial research designs
B) Longitudinal research designs
C) Non-experimental research designs
D) Cross-sectional research designs

Question 10

Which of the following needs to be read in order to adequately evaluate the logical validity of a research study published in a journal?

A) The entire study from beginning to end
B) The measures section of the study
C) The literature review section of the study
D) The methods section of the study

Question 11

The development of a nomological network can be useful in evaluating what type of validity?

A) Logical validity
B) Internal validity
C) External validity
D) Construct validity

Question 12

What type of validity is best represented in the following example?
Dr) Piper is conducting a research study that examines the relationship between height and bone fracture occurrences over 35 years. Dr. Piper measures adolescents' sitting height at age 15, and has them report number of annual bone fractures for the next 35 years. In his previous longitudinal research studies, Dr. Piper found that sitting height in adolescence is predictive of actual height in adulthood, so he is confident that by using sitting height at age 15 he has an adequate measure of the participants' actual height when they are 50 years old.

A) Internal validity
B) Construct validity
C) External validity
D) Logical validity

Question 13

What is required in order to evaluate the internal validity of a study?

A) A control group
B) Manipulation of the independent variable
C) A sample that is representative of the population
D) Random assignment of participants to groups

Question 14

What are two key features of true experimental designs?

A) Including a control group; random assignment of participants to groups
B) Having a large sample size; manipulation of the independent variable
C) Seeing a change in the dependent variable; standardization of experimental procedures
D) Understanding that what constitutes truth is temporary and context specific; having a sample that is representative of the population

Question 15

What is the one of the most effective strategies to increase the external validity of a research study in kinesiology?

A) Having clear inclusion criteria to identify potential participants
B) Developing an intervention that clearly manipulates the independent variable
C) Ensuring WEIRD samples are included in most studies
D) Selecting volunteers who have an interest in physical activity

Question 16

What type of validity is most threatened by a reliance on WEIRD samples to conduct research?

A) Construct validity
B) External validity
C) Internal validity
D) Logical validity

Question 17

Henrich, Heine, and Norenzayan (2010) discuss how researchers often rely on WEIRD samples to conduct their research. What does WEIRD stand for?

A) White, Educated, Insightful, Reflective, and Diplomatic
B) Worldly, Eastern, Insightful, Responsive, and Dynamic
C) Western, Educated, Industrialized, Rich, and Democratic
D) White, Eastern, Industrialized, Resentful, and Disruptive

Question 18

Which of the following refers to the validity of the measure(s) using a research study?

A) Logical validity
B) Internal validity
C) External validity
D) Construct validity

Question 19

You conduct a correlational study on the relationship between physical activity levels and carbohydrate intake. You collect all of your data on a single measurement occasion using a questionnaire measuring physical activity levels and carbohydrate intake. What types of validity can you use to evaluate whether your study is any good?

A) Construct validity and logical validity
B) Internal validity and external validity
C) All of the above
D) None of the above

Question 20

Dr. Ferguson is conducting a research study on the effects of a vegetarian diet on women exercisers' body image. After completing her study, Dr. Ferguson is concerned that the improvements in women exercisers' body image may actually be due to the social support they received from the other study participants rather than the vegetarian diet. Dr. Ferguson's concern represents a threat to what type of validity?

A) External validity
B) Internal validity
C) Logical validity
D) Construct validity

Question 21

Experimental studies are always better than other types of studies.

Question 22

Randomized control trial research designs are often considered the "gold standard" of experimental research.

Question 23

Logical validity is all-or-nothing; a research study is either logically valid or it is not.

Question 24

The specific sample identified in a research study is an irrelevant factor in the evaluation of logical validity.

Question 25

Undergraduate students cannot have their names included on published research studies.

Question 26

One of the best ways to establish the construct validity of a new measure is to test its relation with other existing measures of the same construct.

Question 27

There can be contradictory evidence found across different research studies when looking at group differences as evidence of construct validity.

Question 28

There is always a clear answer to the question "Is construct validity supported?"

Question 29

If a particular exercise training intervention only works with a very specific sample under very specific laboratory conditions, the external validity would necessarily be very low.

Question 30

If a particular exercise training intervention only works with a very specific sample under very specific laboratory conditions, the internal validity would necessarily be very low.

Question 31

Knudson, Elliott, and Ackland (2012) provide an overview of principles that researchers can use to accurately cite research evidence when writing their own research manuscripts. List five of their principles.

Question 32

Provide a research example of "blinding", and using your example discuss why blinding is so important to high quality experimental research.

Question 33

Describe why non-random selection of participants (e.g., volunteers) poses such a high threat to the external validity of research studies in kinesiology.

Question 34

What is a "WEIRD" sample, and why is a reliance on WEIRD samples a potential concern to researchers in kinesiology?

Question 35

How do we know if a measure of a construct is reliable and valid?

Question 36

Within your area of research interest, identify a theoretical construct that you would need to measure in a quantitative study. Next, identify a measure or instrument that you might use to assess that theoretical construct. Lastly, provide specific examples of three types of construct validity evidence that you could evaluate to support your use of that measure or instrument.

Question 37

A researcher was interested in studying the effects of 60 minutes of flexibility training per day on enjoyment of exercise. To recruit participants she put up a poster in the university fitness centre; thirty women subsequently volunteered for her study. Her research design included a pre-test of enjoyment of exercise followed by her intervention in which she asked all participants to participate in a three month long flexibility training program (i.e., each participant completed the 60 minutes of flexibility training on her own in the fitness centre after her regular workout each day). At the end of the training program, twenty-six participants completed a post-test of enjoyment of exercise (four participants dropped out of the study half-way through due to the time commitment required). Based on the changes between the pre-test and post-test on participants' enjoyment scores, the researcher concluded that 60 minutes per day of flexibility training was effective in increasing enjoyment of exercise. Based on the research design described, is the researcher's conclusion justified? Clearly justify your answer with particular reference to internal validity and external validity.

Question 38

A researcher is interested in knowing whether or not a revolutionary dynamic strength-training program aids in developing general flexibility beyond standard daily flexibility exercises alone for individuals above 65 years of age living in Edmonton. She has hypothesized that this effect might be different for males and females because of the specific exercises that are used in the dynamic strength-training program. Describe a study design that would best enable her to answer her research question. This description should include participant selection and data analysis, as well as the specific design strategy.