Deck 5: Test Worthiness: Validity, Reliability, Crosscultural Fairness, and Practicality

Question 1

In developing a multiple aptitude test, the test developers wanted to assure that the various subtests do not correlate strongly with one another, thus assuring the integrity of each subtest. They conduct a number of intercorrelations among subtests. This type of validity is called:

A) experimental design validity.
B) factor analysis validity.
C) convergent validity.
D) criterion-related validity.
E) predictive validity.

Answer 1

factor analysis validity.

Question 2

If you are interested in examining whether or not the GREs can accurately tell how students perform in graduate school, you would most likely want to investigate the validity of the test.

A) content
B) concurrent
C) predictive
D) construct
E) convergent

Answer 2

predictive

Question 3

Which of the following is a type of criterion-related validity?

A) Content validity
B) Predictive validity
C) Convergent validity
D) Discriminant validity
E) None of these are criterion-related.

Answer 3

Predictive validity

Question 4

In assessing the validity of a test, you randomly assign 100 individuals to two groups: one group assesses intelligence through a newly developed test, and a second group assesses intelligence through a traditional intelligence test that has been shown to be valid. You then conduct a statistical analysis to see if there are significant differences between the groups. Your hypothesis states that there will be no significant differences. This type of validity is called:

A) experimental design validity.
B) factor analysis validity.
C) convergent validity.
D) criterion-related validity.
E) discriminant validity.

Question 5

A correlation coefficient of .60 was found when correlating the scores of 1,000 randomly chosen individuals who took a test that measures verbal reasoning with a test that measures problem solving abilities. Which statement is true about this correlation coefficient?

A) The coefficient of determination is .60.
B) The shared variance is .49.
C) The shared variance is .36.
D) The shared variance is 1.2.
E) None of these are true.

Question 6

Which of the following is an example of negative correlation?

A) Caloric intake and a person's weight
B) Exercise and a person's weight
C) Height and a person's weight
D) All of these are examples.
E) None of these are examples.

Question 7

Concurrent and predictive validity are types of:

A) content validity.
B) criterion-related validity.
C) construct validity.
D) test-retest reliability.
E) internal consistency reliability.

Question 8

A test developer creates a new instrument to measure psychotic tendencies. She correlates this instrument with an existing test that measures psychotic tendencies that has been shown to be valid. This type of validity is called:

A) experimental design validity.
B) factor analysis validity.
C) convergent validity.
D) criterion-related validity.
E) discriminant validity.

Question 9

Which of the following scatterplots would best define the relationship between intelligence (IQ) and height?

A)
B)
C)

Question 10

Which of the following is NOT a step in determining the content validity of an instrument?

A) The test developer adequately surveyed the content domain.
B) The content of the test was shown to match what was found in the survey of the domain.
C) The test items accurately reflect the content.
D) The test is shown to be highly correlated with another, similar test.
E) The number of items for each content area matches their relative importance.

Question 11

Which of the following has a correlation of approximately .70?

A)

B)

C)

D)

Question 12

Which of the following scatterplots would best identify the most reliable test-retest correlation?

A)
B)
C)

Question 13

Which of the following is NOT related to test worthiness?

A) Reliability
B) Statistical formation
C) Cross-cultural fairness
D) Validity
E) Practicality

Question 14

Which of the following has the strongest correlation?

A) −.87
B) +.86
C) + 0
D) + .3

Question 15

A test developer creates a new instrument to measure depression. He correlates this instrument with an existing test that measures anxiety. The test developer hopes to not find a high correlation, thus assuring the integrity of the construct he is measuring. This type of validity is called:

A) experimental design validity.
B) factor analysis validity.
C) convergent validity.
D) criterion-related validity.
E) discriminant validity.

Question 16

When a test has been shown to produce consistent test results, it has which of the following?

A) Reliability
B) Validity
C) Practicality
D) Cross-cultural fairness
E) None of these

Question 17

Which of the following is NOT a kind of construct validity?

A) Experimental design validity
B) Factor analysis validity
C) Convergent validity
D) Criterion-related validity
E) Discriminant validity

Question 18

A newly developed instrument for screening depression suggests that an individual is not depressed; however, later information reveals that in fact he is depressed. This problem can be attributed to:

A) parallel forms.
B) internal consistency.
C) a false positive.
D) a false negative.
E) sampling error.

Question 19

A test developer asks five clinicians who are experts in depression to rank 100 depressed clients on a 1 to 5 scale from minor depression to severe depression. The rankings are then correlated with test scores on a new test of depression. What kind of validity is being measured?

A) Experimental
B) Concurrent
C) Predictive
D) Construct
E) Convergent

Question 20

Which of the following does NOT describe validity?

A) It is the degree to which all of the accumulated evidence supports the intended interpretation of test scores for the intended purpose.
B) It is a unitary concept.
C) It examines how well a test measures what it is supposed to measure.
D) It tells you whether or not test scores are consistent.

Question 21

The Buckley Amendment, otherwise known as FERPA, asserted:

A) that accommodations must be made for individuals who are taking tests for employment and that testing must be shown to be relevant to the job in question.
B) that tests used for hiring and advancement at work must show that they can predict job performance for all groups.
C) the right of all individuals to their school records, including test records.
D) that individuals with disabilities, or who are disadvantaged, have access to vocational assessment, counseling, and placement.
E) the right of students to be tested, at a school system's expense, if they are suspected of having a disability that interferes with learning.

Question 22

Which of the following is NOT a type of reliability?

A) Split-half
B) Internal consistency
C) Cronbach's Coefficient Alpha
D) Kuder-Richardson
E) All of these are types of reliability.

Question 23

The Americans with Disabilities Act asserted:

A) that accommodations must be made for individuals who are taking tests for employment and that testing must be shown to be relevant to the job in question.
B) that any test used for employment or promotion must be shown to be suitable and valid for the job in question.
C) that tests used for hiring and advancement at work must show that they can predict job performance for all groups.
D) the right of individuals to access their federal records, including test records.
E) the right of students to be tested, at a school system's expense, if they are suspected of having a disability that interferes with learning.

Question 24

"A determination is made as to how scores on individual items relate to each other or to the test as a whole." This statement refers to which kind of reliability?

A) Test-retest
B) Alternate forms
C) Parallel forms
D) Internal consistency
E) None of these

Question 25

The Freedom of Information Act asserted:

A) the right of students to be tested, at a school system's expense, if they are suspected of having a disability that interferes with learning.
B) that individuals with disabilities, or who are disadvantaged, have access to vocational assessment, counseling, and placement.
C) that any instrument used to measure appropriateness for a program or service must be measuring the individual's ability, not be a reflection of his or her disability.
D) the right of individuals to access their federal records, including test scores.
E) the right of students to be tested, at a school system's expense, if they are suspected of having a disability that interferes with learning

Question 26

When one "attempts to estimate the reliability of all the possible split half combinations" of a test, one is assessing what kind of reliability?

A) Test-retest
B) Alternate forms
C) Parallel forms
D) Coefficient Alpha
E) None of these

Question 27

The Spearman-Brown formula is for which purpose?

A) To make adjustments due to concerns that examinees might look up answers between taking the same test twice
B) To make adjustments to reliability estimate due to concerns of shortness of the test (limited test items)
C) To adjust correlations during factor analysis to assure accurate representation of all items
D) To calculate the coefficient of determination (shared variance) when making adjustments to reliability estimates

Question 28

One problem with giving two forms of the same test is that, despite the publisher's attempt to make them similar, there could be major differences. This problem is most closely associated with which type of reliability?

A) Test-retest
B) Alternate forms
C) Internal consistency
D) Odd-even
E) Kuder-Richardson

Question 29

Which of the following is NOT a type of reliability?

A) Test-retest
B) Alternate forms
C) Parallel forms
D) Internal consistency
E) All of these are types of reliability.

Question 30

A type of reliability in which half of the test items are correlated with the other half is called:

A) test-retest.
B) alternate forms.
C) parallel forms.
D) odd-even.
E) none of these.

Question 31

The Carl Perkins Act (PL 98-524) asserted:

A) that any test used for employment or promotion must be shown to be suitable and valid for the job in question.
B) that tests used for hiring and advancement at work must show that they can predict job performance for all groups.
C) the right of individuals to access their federal records, including test records.
D) the right of students to be tested, at a school system's expense, if they are suspected of having a disability that interferes with learning.
E) that individuals with disabilities, or who are disadvantaged, have access to vocational assessment, counseling, and placement.

Question 32

Which of the following does NOT describe reliability?

A) The degree to which test scores are free from errors of measurement
B) The degree to which test scores are consistent and accurate
C) The degree to which a test measures what it's supposed to measure
D) All of these describe reliability.

Question 33

Which of the following is true regarding Classical Test Theory and/or Item Response Theory (IRT)?

A) Classical Test Theory provides more sophisticated information regarding test items.
B) IRT examines items individually to assess their ability to measure the trait.
C) In IRT, test items are viewed as a whole in an attempt to reduce measurement error.
D) As reliability increases in Classical Test Theory, the measurement error increases.

Question 34

In assessing the reliability of a new test, a sample of 1,000 examinees take the test, and one week later are asked to take the same test again. This is an example of which type of reliability?

A) Test-retest
B) Alternate forms
C) Internal consistency
D) Odd-even
E) None of these

Question 35

Which of the following is NOT a type of internal consistency reliability?

A) Odd-even
B) Parallel forms
C) Coefficient Alpha
D) Kuder-Richardson
E) All of these are types of internal consistency reliability.

Question 36

Theoretically, if several individuals who score high on a test have a substantial number of low responses, and individuals who score low on the test have a substantial number of high responses, then the test is likely to have which of the following?

A) High validity
B) Low validity
C) High reliability
D) Low reliability

Question 37

Which is true regarding the item characteristic curve, or "S" curve, in Item Response Theory (IRT)?

A) It assumes that as people's abilities increase, their probability of answering the item correctly decreases.
B) A tall "S" means that an item has difficulty discriminating across ability.
C) IRT and the S curve assist in writing items to target specific ranges of ability.
D) It is useful in predicting the Standard Error of the Estimate.

Question 38

Section 504 of the Rehabilitation Act asserted:

A) that any instrument used to measure appropriateness for a program or service must be measuring the individual's ability, not be a reflection of his or her disability.
B) the right of individuals to access their federal records, including test records.
C) the right of students to be tested, at a school system's expense, if they are suspected of having a disability that interferes with learning.
D) that tests used for hiring and advancement at work must show that they can predict job performance for all groups.
E) that accommodations must be made for individuals who are taking tests for employment and that testing must be shown to be relevant to the job in question.

Question 39

Griggs v. Duke Power Company asserted:

A) that accommodations must be made for individuals who are taking tests for employment and that testing must be shown to be relevant to the job in question.
B) the right of all individuals to their school records, including test records.
C) that individuals with disabilities, or who are disadvantaged, have access to vocational assessment, counseling, and placement.
D) that tests used for hiring and advancement at work must show that they can predict job performance for all groups.
E) that any instrument used to measure appropriateness for a program or service must be measuring the individual's ability, not be a reflection of his or her disability.

Question 40

PL 94-142 and the IDEA asserted:

A) that accommodations must be made for individuals who are taking tests for employment and that testing must be shown to be relevant to the job in question.
B) the right of all individuals to their school records, including test records.
C) that individuals with disabilities, or who are disadvantaged, have access to vocational assessment, counseling, and placement.
D) the right of students to be tested, at a school system's expense, if they are suspected of having a disability that interferes with learning.

Question 41

Rather than giving individualized tests to assess learning disabilities to thousands of students, a school system decides to first use a shortened version of the test that can be given in a group format. This decision is mostly concerned with which test worthiness issue?

A) Reliability
B) Statistical formation
C) Cross-cultural fairness
D) Validity
E) Practicality

Question 42

Cross-cultural fairness in testing has always been a critical factor in the development of tests.

Question 43

Concurrent validity is to criterion-related validity as discriminant validity is to construct validity.

Question 44

Match each description to the term or concept listed below.
A statistical expression of the relationship between sets of scores.

A)Validity
B)Reliability
C)Correlation
D)SEest

Question 45

Concurrent validity is sometimes called "here and now" validity.

Question 46

Match each description to the term or concept listed below.
The degree to which an instrument's scores are free from error of measurement.

A)Validity
B)Reliability
C)Correlation
D)SEest

Question 47

A correlation coefficient of .22 is stronger than a correlation coefficient of −.87.

Question 48

Practicality has to do with all of the following EXCEPT:

A) time.
B) cost.
C) format and readability.
D) ease of administration, scoring, and interpretation.
E) Practicality has to do with all of these.

Question 49

The most important form of content validity is face validity.

Question 50

Match each description to the term or concept listed below.
Uses the score of one variable to predict a range of scores for a second variable.

A)Validity
B)Reliability
C)Correlation
D)SEest

Question 51

According to the text, which of the following is NOT a step to selecting and administering a good test?

A) Determining the goals of your assessment
B) Accessing information about possible instruments
C) Examining the validity, reliability, cross-cultural fairness, and practicality of possible instruments
D) Making a wise choice when choosing an instrument
E) Running a "pilot study" to assess the instruments you have chosen

Question 52

Item Response Theory (IRT) has the advantage over Classical Test Theory in that it provides more detailed information regarding each item in a test.

Question 53

In selecting a test, one often goes to existing sources including all of the following EXCEPT:

A) experts in the field.
B) journals in the field.
C) The Assessment Catalog and Resource Index.
D) Mental Measurement Yearbook.
E) Buros.

Question 54

Match each description to the term or concept listed below.
The statistic used to express a reliability estimate.

A)Validity
B)Reliability
C)Correlation
D)SEest

Question 55

The square root of a correlation coefficient gives the coefficient of determination, otherwise known as the shared variance.

Question 56

Alternate forms reliability eliminates some of the problems found in test-retest reliability but raises other concerns.

Question 57

Match each description to the term or concept listed below.
What the instrument measures and how well it does it.

A)Validity
B)Reliability
C)Correlation
D)SEest

Question 58

One problem with parallel forms reliability is that individuals may look up questions between the time they take the two tests.

Question 59

Probably, the most basic form of validity is content validity.

Question 60

One problem with split-half reliability is that due to fact that you are correlating two shorter halves, you can create greater error in your correlation coefficient.

Question 61

In developing the national certification exam for counselors, the test developers wanted to assure adequate validity and reliability. For the following items, place the letter that represents the kind of validity or reliability described.
Test-retest reliability

A)The extent to which the instrument measures a theoretical or hypothetical trait or concept
B)Tests scores from one version of a test are correlated with another "alternate" version
C)The extent to which an instrument correlates with an outcome criterion in the present
D)Correlating one test administration with another administration of the same test at a later date
E)The degree the evidence shows items and questions represent the proper domain
F)The correlation of odd numbered items on a test with even numbered items
G)The extent to which the instrument correlates with future outcome criterion
H)A statistical analysis of each test item against all of the other test items

Question 62

In developing the national certification exam for counselors, the test developers wanted to assure adequate validity and reliability. For the following items, place the letter that represents the kind of validity or reliability described.
All the possible split-half correlations are correlated and then the mean, or average, of all of these scores is obtained.

A)Content validity
B)Concurrent validity
C)Construct validity
D)Discriminant validity
E)Parallel or equivalent form reliability
F)Internal consistency
G)Predictive validity
H)Odd-even reliability
I)Test-retest reliability

Question 63

In developing the national certification exam for counselors, the test developers wanted to assure adequate validity and reliability. For the following items, place the letter that represents the kind of validity or reliability described.
Students take two different versions of the same test and their scores are correlated.

A)Content validity
B)Concurrent validity
C)Construct validity
D)Discriminant validity
E)Parallel or equivalent form reliability
F)Internal consistency
G)Predictive validity
H)Odd-even reliability
I)Test-retest reliability

Question 64

In developing the national certification exam for counselors, the test developers wanted to assure adequate validity and reliability. For the following items, place the letter that represents the kind of validity or reliability described.
Student scores on the exam are correlated with how well supervisors rate them two years later.

A)Content validity
B)Concurrent validity
C)Construct validity
D)Discriminant validity
E)Parallel or equivalent form reliability
F)Internal consistency
G)Predictive validity
H)Odd-even reliability
I)Test-retest reliability

Question 65

In developing the national certification exam for counselors, the test developers wanted to assure adequate validity and reliability. For the following items, place the letter that represents the kind of validity or reliability described.
Kuder-Richardson reliability

A)The extent to which the instrument measures a theoretical or hypothetical trait or concept
B)Tests scores from one version of a test are correlated with another "alternate" version
C)The extent to which an instrument correlates with an outcome criterion in the present
D)Correlating one test administration with another administration of the same test at a later date
E)The degree the evidence shows items and questions represent the proper domain
F)The correlation of odd numbered items on a test with even numbered items
G)The extent to which the instrument correlates with future outcome criterion
H)A statistical analysis of each test item against all of the other test items

Question 66

In developing the national certification exam for counselors, the test developers wanted to assure adequate validity and reliability. For the following items, place the letter that represents the kind of validity or reliability described.
Concurrent validity

A)The extent to which the instrument measures a theoretical or hypothetical trait or concept
B)Tests scores from one version of a test are correlated with another "alternate" version
C)The extent to which an instrument correlates with an outcome criterion in the present
D)Correlating one test administration with another administration of the same test at a later date
E)The degree the evidence shows items and questions represent the proper domain
F)The correlation of odd numbered items on a test with even numbered items
G)The extent to which the instrument correlates with future outcome criterion
H)A statistical analysis of each test item against all of the other test items

Question 67

In developing the national certification exam for counselors, the test developers wanted to assure adequate validity and reliability. For the following items, place the letter that represents the kind of validity or reliability described.
Content validity

A)The extent to which the instrument measures a theoretical or hypothetical trait or concept
B)Tests scores from one version of a test are correlated with another "alternate" version
C)The extent to which an instrument correlates with an outcome criterion in the present
D)Correlating one test administration with another administration of the same test at a later date
E)The degree the evidence shows items and questions represent the proper domain
F)The correlation of odd numbered items on a test with even numbered items
G)The extent to which the instrument correlates with future outcome criterion
H)A statistical analysis of each test item against all of the other test items

Question 68

In developing the national certification exam for counselors, the test developers wanted to assure adequate validity and reliability. For the following items, place the letter that represents the kind of validity or reliability described.
Parallel form reliability

A)The extent to which the instrument measures a theoretical or hypothetical trait or concept
B)Tests scores from one version of a test are correlated with another "alternate" version
C)The extent to which an instrument correlates with an outcome criterion in the present
D)Correlating one test administration with another administration of the same test at a later date
E)The degree the evidence shows items and questions represent the proper domain
F)The correlation of odd numbered items on a test with even numbered items
G)The extent to which the instrument correlates with future outcome criterion
H)A statistical analysis of each test item against all of the other test items

Question 69

In developing the national certification exam for counselors, the test developers wanted to assure adequate validity and reliability. For the following items, place the letter that represents the kind of validity or reliability described.
In creating the test, they hypothesized that students who finished a master's degree in counseling would do significantly better than students who received a master's in psychology. Then, a study is conducted examining this hypothesis.

A)Content validity
B)Concurrent validity
C)Construct validity
D)Discriminant validity
E)Parallel or equivalent form reliability
F)Internal consistency
G)Predictive validity
H)Odd-even reliability
I)Test-retest reliability

Question 70

In developing the national certification exam for counselors, the test developers wanted to assure adequate validity and reliability. For the following items, place the letter that represents the kind of validity or reliability described.
Construct validity

A)The extent to which the instrument measures a theoretical or hypothetical trait or concept
B)Tests scores from one version of a test are correlated with another "alternate" version
C)The extent to which an instrument correlates with an outcome criterion in the present
D)Correlating one test administration with another administration of the same test at a later date
E)The degree the evidence shows items and questions represent the proper domain
F)The correlation of odd numbered items on a test with even numbered items
G)The extent to which the instrument correlates with future outcome criterion
H)A statistical analysis of each test item against all of the other test items

Question 71

In developing the national certification exam for counselors, the test developers wanted to assure adequate validity and reliability. For the following items, place the letter that represents the kind of validity or reliability described.
Half of the test is correlated with the other half of the test.

A)Content validity
B)Concurrent validity
C)Construct validity
D)Discriminant validity
E)Parallel or equivalent form reliability
F)Internal consistency
G)Predictive validity
H)Odd-even reliability
I)Test-retest reliability

Question 72

In developing the national certification exam for counselors, the test developers wanted to assure adequate validity and reliability. For the following items, place the letter that represents the kind of validity or reliability described.
Predictive validity

A)The extent to which the instrument measures a theoretical or hypothetical trait or concept
B)Tests scores from one version of a test are correlated with another "alternate" version
C)The extent to which an instrument correlates with an outcome criterion in the present
D)Correlating one test administration with another administration of the same test at a later date
E)The degree the evidence shows items and questions represent the proper domain
F)The correlation of odd numbered items on a test with even numbered items
G)The extent to which the instrument correlates with future outcome criterion
H)A statistical analysis of each test item against all of the other test items

Question 73

In developing the national certification exam for counselors, the test developers wanted to assure adequate validity and reliability. For the following items, place the letter that represents the kind of validity or reliability described.
The test is given to a group of students and then, one day later, they take the same test again.

A)Content validity
B)Concurrent validity
C)Construct validity
D)Discriminant validity
E)Parallel or equivalent form reliability
F)Internal consistency
G)Predictive validity
H)Odd-even reliability
I)Test-retest reliability

Question 74

Match each description to the term or concept listed below.
The degree to which test scores are consistent, dependable, and repeatable.

A)Validity
B)Reliability
C)Correlation
D)SEest

Question 75

In developing the national certification exam for counselors, the test developers wanted to assure adequate validity and reliability. For the following items, place the letter that represents the kind of validity or reliability described.
The test is correlated with the national comprehensive exams to assure that the two tests are not measuring the same thing.

A)Content validity
B)Concurrent validity
C)Construct validity
D)Discriminant validity
E)Parallel or equivalent form reliability
F)Internal consistency
G)Predictive validity
H)Odd-even reliability
I)Test-retest reliability

Question 76

In developing the national certification exam for counselors, the test developers wanted to assure adequate validity and reliability. For the following items, place the letter that represents the kind of validity or reliability described.
Split-half reliability

A)The extent to which the instrument measures a theoretical or hypothetical trait or concept
B)Tests scores from one version of a test are correlated with another "alternate" version
C)The extent to which an instrument correlates with an outcome criterion in the present
D)Correlating one test administration with another administration of the same test at a later date
E)The degree the evidence shows items and questions represent the proper domain
F)The correlation of odd numbered items on a test with even numbered items
G)The extent to which the instrument correlates with future outcome criterion
H)A statistical analysis of each test item against all of the other test items

Question 77

In developing the national certification exam for counselors, the test developers wanted to assure adequate validity and reliability. For the following items, place the letter that represents the kind of validity or reliability described.
Students who are graduating from their master's degrees take the test, and, at the same time, their test scores are correlated with professor rankings of how good a student they are.

A)Content validity
B)Concurrent validity
C)Construct validity
D)Discriminant validity
E)Parallel or equivalent form reliability
F)Internal consistency
G)Predictive validity
H)Odd-even reliability
I)Test-retest reliability

Question 78

In developing the national certification exam for counselors, the test developers wanted to assure adequate validity and reliability. For the following items, place the letter that represents the kind of validity or reliability described.
In developing the test, counseling books were surveyed and experts were contacted in an effort to create items.

A)Content validity
B)Concurrent validity
C)Construct validity
D)Discriminant validity
E)Parallel or equivalent form reliability
F)Internal consistency
G)Predictive validity
H)Odd-even reliability
I)Test-retest reliability