Deck 12: Evaluating the Methods Sectioninstruments

Question 1

______ scores are based on a predetermined level of performance to a comparison study.

A) Criterion-referenced
B) Norm-referenced
C) Analytically referenced
D) Reliant-referenced

Answer 1

A

Question 2

In ______ approach, scores are interpreted based on the comparison of one group's performance with other groups', all of whom represent a clearly defined population.

A) criterion-referenced
B) norm-referenced
C) analytically referenced
D) reliant-referenced

Answer 2

B

Question 3

Thorndike and Thorndike-Christ noted that many tests and measures are designed to "reflect the values and experiences of typical ______."

A) middle class
B) lower class
C) upper class
D) poor

Answer 3

A

Question 4

Reliability is reported as a correlation expressed as the degree of the relationship between two sets of scores. Possible scores in these correlations can range from ______.

A) -2.0 to + 2.0
B) 0 to +1.0
C) -3.0 to +3.0
D) -1.0 to +1.0

Question 5

Strong indicators of reliability in a particular test or measure have a score of at least ______.

A) .75
B) .60
C) .80
D) .50

Question 6

______ concerns the consistency of a measure based on content, for instance, when two forms of the measure are administered to a sample and reliability coefficients are obtained between them.

A) Test-retest reliability
B) Scorer reliability
C) Alternate-form reliability
D) Interrater reliability

Question 7

Which type of reliability involves the measurement of the consistency of scores over a period of time?

A) test-retest reliability
B) scorer reliability
C) alternate-form reliability
D) interrater reliability

Question 8

A basic type of validity that concerns the extent to which responses on test items represent a particular content is known as ______.

A) construct validity
B) criterion-related validity
C) content validity
D) convergence and divergence evidence

Question 9

Which type of validity involves obtaining evidence of prediction of a particular construct, or accuracy in identifying a construct in practice?

A) construct validity
B) criterion-related validity
C) content validity
D) internal validity

Question 10

All the potential effects that alter the true score, such as the limitations of the test, time of day in responding to the measure, and lack of consistent administration procedures, are referred to as ______.

A) truer score
B) error
C) formal structure
D) criterion structure

Question 11

Factor analysis is a source for ______.

A) criterion-related validity
B) convergence and divergence evidence
C) content-related validity
D) construct validity

Question 12

An approach to measuring dependent variables where scores are used to evaluate behavior is known as ______.

A) self-report
B) ratings
C) interviews
D) direct observation

Question 13

The strength of this approach to measuring dependent variables is ease and time of administration and does not require extensive training is ______.

A) ratings
B) interviews
C) self-report inventories
D) physiological method

Question 14

One of the cornerstone tools for the assessment of behavioral, social, and emotional problems is ______.

A) direct observation
B) indirect observation
C) interviews
D) self-report

Question 15

Assessments that measure heart rate, ECG, blood pressure, and galvanic skin response are known as ______.

A) direct observation
B) indirect observation
C) physiological methods
D) self-report

Question 16

Instrumentation is that area of the methods section where the researchers systematically describe the dependent measures used in the study.

Question 17

Standardization in administering the dependent measure theoretically provides more objective data.

Question 18

Researchers and practitioners have noted the importance of using measures of the dependent variable that are culturally, gender, and diversity-issue appropriate.

Question 19

Use of an inappropriate test or dependent measure can result in test bias and problems with interpreting outcomes.

Question 20

A higher positive correlation indicates a weak relationship between the two sets of scores.

Question 21

Scorer reliability is the most frequently reported type of reliability.

Question 22

The most general type of validity is construct validity because it supposedly concerns the degree to which a test measures an intended theoretical concept or construct.

Question 23

Without information on the reliability and validity of the instruments used in a quantitative study, it is difficult to conclude whether the study results obtained are accurate.

Question 24

Psychometricians have long acknowledged that validity is the least important characteristic of a test.

Question 25

Self-report and inventories are not the most frequently used methods of measuring the dependent variable in psychology, counseling, and education.

Question 26

Provide an example of a criterion-referenced score.

Question 27

Provide an example of a norm-referenced score.

Question 28

List a few of the ways to determine reliability.

Question 29

Give an example of the test-retest method of determining reliability.

Question 30

What are the three components of construct validity Urbina (2004) identified?

Question 31

Provide an example of content validity.

Question 32

Provide an example of a self-report inventory.

Question 33

What is the calculation of Kappa and when would you use this?

Question 34

An important consideration in assessing the objectivity and quality of a dependent measure is the extent to which the procedures and scoring methods are standardized. Discuss the standardization methods delineated in the text.

Question 35

Discuss the APA Standards for Educational and Psychological Tests and the American Counseling Association Code of Ethics regarding tests and diversity.

Question 36

Researchers have noted that one of the most important considerations in determining the quality of the dependent measures is the reliability of the instruments and dependent variables. Define and discuss reliability of the instruments and the various forms of determining reliability.

Question 37

Discuss the advantages and disadvantages of using self-report inventories.

Question 38

Discuss alternate-form reliability. What is a benefit of using this approach?