Exam 9: Validity and Reliability

Exhibit 9-4: Following is a list of testing practices.Assuming other things are equal, indicate whether the practice would lower reliability, raise reliability, or neither. -Refer to Exhibit 9-4.Changing from a multiple-choice test to an essay test covering the same materials.

It has been determined that 30 percent of a test's total variance is error.What is the test's reliability coefficient?

Which of the following would contribute the best evidence for the score validity of a new group intelligence test?

Exhibit 9-1: Choose the type of evidence for validity that is referred to in the following questions. -Refer to Exhibit 9-1.The sample of items is representative of the course subject matter.

Which of the following is not an internal consistency measure of reliability?

A researcher wishes to increase the score reliability of a reading test being used in a study.The score reliability could be most easily increased by increasing the

An individual's score on an achievement test is .82.The standard error of measurement for the test is reported to be 3 points.What are the chances that the individual's true score is between 76 and 88?

In the following scenarios, indicate whether the researcher is investigating validity or reliability: a.A researcher developed a measure of achievement motivation and then compared students judged to have high need to achieve with students judged with low need to achieve to see if they performed differently on the motivation measure. b.A researcher administered a test to a sample of students.He administered the same test six weeks later and correlated the results of the two administrations. c.A researcher administered a test of teaching ability to a group of graduating college seniors in a school of education and then at the end of their first year of teaching correlated their test scores with principals' ratings of their teaching effectiveness.He found a moderate correlation.

The standard error of measurement is based on the test's

Exhibit 9-8: The following data were obtained when two forms of a criterion-referenced test were given to a group of 10 students.There were 40 items on each form and a student had to receive75 percent correct on each form. Examinee Form 1 Form 2 1 30 30 2 28 29 3 34 37 4 38 39 5 32 40 6 26 28 7 28 35 8 35 36 9 38 29 10 36 36 -Refer to Exhibit 9-8.Express the score reliability of this test in terms of the simple agreement coefficient.

Exhibit 9-11: Choose the type of validity evidence that a researcher would need to determine the score validity of the following kinds of instruments. -Refer to Exhibit 9-11.A test designed to measure what fourth graders know about long division.

Exhibit 9-7: The following results were obtained when two forms of a criterion-referenced test were administered to a sample of 50 students: Form 1 Nonmaster Master -Refer to Exhibit 9-7.Calculate the index that shows the extent of agreement beyond that expected by chance (phi).

Reliability is to random error as validity is to

The reliability of a spelling test is .80.If the test is doubled in length, the reliability of the test will be approximately

A particular aptitude test shows a moderate correlation with college GPA.Some argue that many students who would not have succeeded in college anyway choose not to take the test or attempt college work, thus resulting in a form of self-preselection.What would be the effect on the validity index of the aptitude test if every high school senior took the test and attempted one year of college?

Exhibit 9-5: Choose the method of estimating score reliability that would be most useful for each of the types of tests listed below. -Refer to Exhibit 9-5.An achievement test to be given as a pretest and a posttest in a classroom research study

Exhibit 9-10: Indicate the type of reliability that would result from the procedures described below. -Refer to Exhibit 9-10.Scores at time A correlated with scores at time B.

Exhibit 9-6: Indicate which type of evidence is being gathered in the following questions. -Refer to Exhibit 9-6.A teacher compares the science test results with the results of a standardized test in science.

A test has a standard deviation of 4 and a reliability of .91.What is the standard error of measurement?

A test has high score reliability if