Deck 5: Validity

Question 1

__________ refers to the degree to which a test 'appears' to measure what it is designed to measure.

A)Content validity
B)Construct validity
C)Criterion-related validity
D)Face validity

Answer 1

Face validity

Question 2

Contemporary psychometric standards emphasize:

A)three different types of validity.
B)that different validity types apply to different assessments.
C)that validity is a unitary construct.
D)that validity is conceptualized as being all or none.

Answer 2

that validity is a unitary construct.

Question 3

Research has shown that validity coefficients:

A)can rarely be generalized across samples.
B)can be generalized more than previously thought.
C)must be > .90 to have any utility.
D)are of little utility in evaluating the validity of score interpretations.

Answer 3

can be generalized more than previously thought.

Question 4

The GRE is given to students prior to entering their first year of graduate school.What type of validity evidence is most important for this test?

A)Validity evidence based on consequences of testing
B)Validity evidence based on test content
C)Validity evidence based on relations to other variables
D)Validity evidence based on response processes
E)Validity evidence based on internal structure

Question 5

Reynolds and Kamphaus (2003)note that ___________ is a statistical approach that allows one to evaluate the presence and structure of any latent constructs existing among a set of variables.

A)factor analysis
B)linear regression
C)multimatrix transformation
D)multi-trait regression

Question 6

_________ refers to the appropriateness or accuracy of the interpretations of test scores.

A)Correlation
B)Evidence
C)Reliability
D)Validity

Question 7

A 3^rd grade mathematics exam designed to be a comprehensive measure of the mathematics skills covered in a 4^th grade curriculum only includes questions involving fractions.This is an example of which threat to validity?

A)Construct-irrelevant variance
B)Construct underrepresentation
C)Factor variance
D)Item homogeneity

Question 8

The __________ of a measure to a diagnostic condition is the ability of the test at a predetermined cut score to detect the presence of the disorder.

A)correlation
B)reliability
C)sensitivity
D)specificity

Question 9

A statistic called the _____________ is used to describe the amount of prediction error resulting from the imperfect correlation between a test score and a criterion.

A)error variance
B)observed score variance
C)standard error of estimate
D)standard error of measurement

Question 10

________ models help the test user determine how much information a predictor test can contribute when making classification decisions.

A)Choice-theory
B)Classification-theory
C)Decision-theory
D)Selection-theory

Question 11

Only true score variance is reliable,and only _________ can be related systematically to any construct the test is designed to measure.

A)true score variance
B)observed score variance
C)error variance

Question 12

According to the authors of the text,the minimum size for acceptable validity coefficients is:

A)0.60.
B)0.70.
C)0.80.
D)0.90.
E)indeterminate - it depends on many factors.

Question 13

A math test intended to cover the year's content was administered to a class.The teacher ran out of time the night before and only asked questions from the first semester's material.What type of threat to validity is present?

A)Construct underrepresentation
B)Construct-irrelevant variance
C)Face validity
D)Discriminant validity

Question 14

What is produced by an examination of the relationship between the content of the test and the construct or domain the test is designed to measure?

A)Validity evidence based on test content
B)Validity evidence based on internal structure
C)Validity evidence based on response processes
D)Validity evidence based on relations to other variables
E)Validity evidence based on consequences of testing

Question 15

Dr.Jones has developed a new test to measure intelligence in children and decides to see how well it correlates with the Wechsler Intelligence Scale for Children (WISC-IV).This would be an example of which type of validity evidence?

A)Convergent evidence of validity
B)Discriminant evidence of validity
C)Validity evidence based on response processes
D)Validity evidence based on test content

Question 16

When deciding how many factors to retain in a factor analysis,a common approach,referred to as the Kaiser-Guttman criteria,is to examine eigenvalues and retain those greater than ______.

A)0.3
B)0.5
C)0.8
D)1.0

Question 17

What type of validity study used to collect test-criterion evidence involves an administration of the test,a time interval,and then a measure of the criterion?

A)Concurrent study
B)Construct study
C)Delayed evidence study
D)Predictive study

Question 18

A test developer assembles a group of high school science teachers to evaluate and compare the themes and wording of the items on a science test with the intended objectives.Which category of validity evidence does this best represent?

A)Validity evidence based on the consequences of testing
B)Validity evidence based on test content
C)Validity evidence based on relations to other variables
D)Validity evidence based on internal structure
E)Validity evidence based on response processes

Question 19

Of the following,which is NOT a threat to validity?

A)Coaching
B)Examinee characteristics such as high test anxiety
C)Appropriateness of reference group
D)High reliability coefficients

Question 20

A 4^th grade final exam for mathematics has extensive written instructions.In addition to math skills,this exam is probably measuring reading comprehension skills as well.This is an example of which threat to validity?

A)Construct-irrelevant variance
B)Construct underrepresentation
C)Factor variance
D)Item homogeneity

Question 21

_______________ is the most common approach to establishing the validity of academic achievement tests.

A)Validity evidence based on consequences of testing
B)Validity evidence based on test content
C)Validity evidence based on relations to other variables
D)Validity evidence based on response processes
E)Validity evidence based on internal structure

Question 22

When tests are used to make decisions regarding personnel selection,the proportion of applicants needed to fill positions is referred to as the:

A)base rate.
B)positive retention rate.
C)negative retention rate.
D)selection ratio.

Question 23

Which type of validity evidence is most controversial and ignites considerable debate among the experts?

A)Validity evidence based on consequences of testing
B)Validity evidence based on test content
C)Validity evidence based on relations to other variables
D)Validity evidence based on response processes
E)Validity evidence based on internal structure

Question 24

When discriminant measures are used in validation studies,we expect _________.

A)positive correlations
B)negative correlations
C)Indeterminate - it depends on the nature of relationship.

Question 25

A new intelligence test is developed and then given to a group of students with no known disabilities and a group students diagnosed with mental retardation.This method of obtaining validity evidence is referred to as a:

A)contrasted groups study.
B)test-retest study.
C)alternate form study.
D)norm referenced study.

Question 26

Dr.Handle has developed a new test designed to measure anxiety.He subsequently correlates his new test with a test that measures sensation seeking.He discovers a negative correlation between the two measures.This is an example of which type of validity evidence?

A)Convergent evidence
B)Discriminant evidence
C)Validity evidence based on test content
D)Validity evidence based on response processes

Question 27

Using the scree plot below,how many factors should be retained?

A)2
B)4
C)5
D)7

Question 28

After examinees complete a math reasoning test,they are interviewed by researchers and asked what processes and strategies they engaged in when completing the test.Which type of validity evidence are these researchers collecting?

A)Validity evidence based on consequences of testing
B)Validity evidence based on test content
C)Validity evidence based on relations to other variables
D)Validity evidence based on response processes
E)Validity evidence based on internal structure

Question 29

The __________ of a measure is the ability of the test at a predetermined cut score to determine accurately the absence of the disorder.

A)correlation
B)reliability
C)sensitivity
D)specificity

Question 30

When tests are used to make decisions regarding personnel selection,the proportion of applicants who can be successful on the criterion is referred to as the:

A)base rate.
B)positive retention rate.
C)negative retention rate.
D)selection ratio.

Question 31

Which factor analytic method analyzes only shared variance while excluding unique and error variance?

A)Component Factor Analysis
B)Confirmatory Factor Analysis
C)Principal Component Analysis
D)Principal Factor Analysis

Question 32

Which type of validity evidence can best be strengthened by the use of a table of specifications?

A)Validity evidence based on consequences of testing
B)Validity evidence based on test content
C)Validity evidence based on relations to other variables
D)Validity evidence based on response processes
E)Validity evidence based on internal structure

Question 33

Test-criterion validity evidence is an example of:

A)validity evidence based on consequences of testing.
B)validity evidence based on test content.
C)validity evidence based on relations to other variables.
D)validity evidence based on response processes.
E)validity evidence based on internal structure.

Question 34

There exists a relatively sophisticated validation technique referred to as ___________ that combines convergent and divergent strategies.

A)analysis of variance
B)factor analysis
C)linear regression
D)multitrait-multimethod matrix

Question 35

____________ is typically used to asses classroom academic achievement tests.

A)Validity evidence based on consequences of testing
B)Validity evidence based on test content
C)Validity evidence based on relations to other variables
D)Validity evidence based on response processes
E)Validity evidence based on internal structure

Question 36

Typically,qualitative approaches such as having experts judge the relevance of items on a test have been used to collect which type of validity evidence?

A)Validity evidence based on consequences of testing
B)Validity evidence based on test content
C)Validity evidence based on relations to other variables
D)Validity evidence based on response processes
E)Validity evidence based on internal structure

Question 37

Factor analysis is commonly used to collect validity evidence based on:

A)consequences of testing.
B)test content .
C)relations to other variables.
D)response processes.
E)internal structure.

Question 38

Which statistical procedure allows one to predict performance on one test from performance on another?

A)factor analysis
B)linear regression
C)sensitivity analysis
D)analysis of variance

Question 39

In order to avoid __________ when collecting test-criterion evidence,it is important that predictor and criterion scores be obtained independently.

A)construct underrepresentation
B)criterion contamination
C)construct-irrelevant variance
D)criterion misrepresentation