Deck 7: Utility

Question 1

The textbook authors caution that in addition to formulas,tables,and other formal methodologies,this must also be factored into any utility analysis:

A) common sense
B) physical property
C) intellectual property
D) All of these

Answer 1

A

Question 2

A utility analysis may be undertaken for many different purposes.Which of the following reasons is LEAST LIKELY to be one of them?

A) to determine if one test is preferable to another test
B) to determine if one method of intervention is preferable to another
C) to evaluate the validity of a test publisher's claim
D) to evaluate whether no training program is better than a training program

Answer 2

C

Question 3

If undertaken for the purpose of evaluating a training program or intervention,the utility analysis will help make decisions regarding whether

A) one training program is preferable to another.
B) one method of intervention is preferable to another.
C) no training program is preferable to any training program.
D) All of these

Answer 3

D

Question 4

Thinking of the Close-up in Chapter 7,what type of error would have been made if a driver who was hired was actually not qualified for the job?

A) a false positive
B) a false negative
C) a near outlier
D) none of these

Question 5

The EverydayPsychometrics feature in Chapter 7 began with a scenario about a man complaining of leg pain while playing basketball.The story ended with this "moral" to the story:

A) "Never play basketball, or other contact sport, if you are on the wrong side of 30."
B) "The utility of playing the game must be weighed against that of not playing."
C) "There are costs associated with testing, and costs associated with not testing."
D) None of these

Question 6

The selection policies of the admissions office of a university can be instrumental in

A) building a good reputation for the university.
B) developing a good learning environment for students.
C) building high morale for the university's faculty.
D) All of these

Question 7

As used in utility analyses,the term cost refers to

A) insurance payments.
B) mortgage payments.
C) payments for test protocols.
D) All of these

Question 8

A limitation of the Taylor-Russell tables is that

A) the relationship between the test and the rating of performance must be linear.
B) the relationship between the predictor and the criterion must be linear.
C) criterion score difficulties in differentiating "successful" from "not successful."
D) All of these

Question 9

In industrial settings,there are many non-economic benefits to be derived for the company that runs an effective testing program.Which of the following is NOT one such benefit?

A) increase in quality of workers' performance
B) decrease in time to train workers
C) reduction of work turnover
D) decrease in worker bonuses

Question 10

As noted by the Hansen et al.study cited in the text,the addition of two more x-ray views in addition to the conventional two is advisable because

A) it may make a more expensive procedure such as a CAT scan unnecessary.
B) the additional radiation exposure was shown to be insignificant.
C) it may be helpful in diagnosing whether physical abuse has occurred.
D) the procedure can do "double duty" as a drug screening tool.

Question 11

Your text listed many frequently raised questions related to test utility.Which is NOT one of those questions:

A) Is the time it takes to administer this test worth it?
B) What is the practical value of using this test over another?
C) Is the cost of using this test worth the cost savings that may result?
D) Does this test measure what it purports to measure?

Question 12

The textbook authors define a utility analysis as

A) a proprietary technique.
B) a family of techniques.
C) an exclusionary technique.
D) None of these

Question 13

The tools of assessment used to make clinical judgments regarding the need for involuntary hospitalization

A) benefit society at large.
B) may cause some to be unjustly denied their freedom.
C) include tests, case history data, and interviews.
D) All of these

Question 14

From Dr.Erik Viirre's Meet an Assessment Professional essay in Chapter 7,one could reasonably conclude that doing this while driving is not a good idea:

A) talking on a cell phone
B) drinking alcohol
C) getting too involved in music
D) All of these

Question 15

Top-down selection is a process whereby job applicants are awarded job positions

A) on the basis of independent adverse impact analyses.
B) with the lowest scorer in the first position to be hired.
C) with the highest scorer in the first position to be hired.
D) None of these

Question 16

The end point of a utility analysis is typically an educated decision about which of many possible courses of action is

A) optimal.
B) the "hands down" best.
C) the "hands down" worst
D) None of these

Question 17

A utility analysis of a test may best be thought of as

A) a model for understanding the costs of producing the test.
B) a way of envisioning new uses for the test.
C) an evaluation of the costs and benefits of the test.
D) an amalgam of reliability and validity data on the test.

Question 18

In comparing one test to another in terms of utility,test users will evaluate whether

A) no testing is better than testing.
B) one tool of assessment is preferable to another.
C) one particular test is preferable to another.
D) All of these

Question 19

Which of the following is NOT a factor that affects a test's utility?

A) the test's validity
B) the test's publisher
C) the cost of the test
D) the benefits of the test

Question 20

In a now-classic utility analysis,Cascio and Ramos found that the use of a particular approach to assessment for selecting managers by the telephone company

A) was costing the company over two million dollars per year.
B) could save the company more than $13 million over four years.
C) had to remain outdated because modernization was too costly.
D) None of these

Question 21

If undertaken for the purpose of evaluating a particular intervention,a utility analysis can be helpful in making decisions about whether

A) one training program is preferable to another training program.
B) any intervention is better than no intervention.
C) one tool of assessment is more practical than another.
D) All of these

Question 22

A group of testtakers all fail to follow the directions for taking a particular test.Which is TRUE?

A) The test results could still have great utility.
B) The test could still be psychometrically sound
C) The test results could still be valid.
D) The reason for this must have to do with the test and not the testtakers.

Question 23

The term multiple cut scores refers to

A) the use of two or more cut scores with reference to one predictor for the purpose of categorizing testtakers.
B) the use of two or more cut scores with reference to a multi-stage evaluation process that employs several predictors.
C) Both a and b
D) None of these

Question 24

The knowngroups method for setting cut scores is also known as

A) the method of contrasting groups.
B) an IRT-based method.
C) the method of discriminant analysis.
D) the Angoff method.

Question 25

The end-point of a utility analysis is typically an educated decision about

A) limits on monetary expenditures for a new test or testing program.
B) which of many possible courses of action is optimal.
C) where to fix a cut point on a new test.
D) how to structure an intervention so that it is most cost efficient.

Question 26

Multiplehurdles as used in a decision-making process regarding a selection decision refers to:

A) the use of two or more cut scores with reference to one predictor for the purpose of categorizing testtakers.
B) the multiple stages each applicant must successfully complete in order to get to the next stage in the evaluation process.
C) the obstacles to success placed before each of the contestants on Project Runway.
D) All of these

Question 27

Which is the BEST general statement regarding a self-report test of integrity?

A) It is a valid test.
B) It is a reliable test.
C) It is not a psychometrically sound test.
D) It is a test of questionable utility.

Question 28

The television program Dancing With the Stars

A) has a multiple hurdle selection model in place.
B) employs an absolute cut score for selection.
C) uses the known groups method to validate results.
D) once encouraged William Angoff to compete.

Question 29

High morale on a university campus can be ____________ resulting from a utility analysis of student selection procedures.

A) an economic cost
B) a non-economic cost
C) an economic benefit
D) a non-economic benefit

Question 30

Test validity

A) has nothing whatever to do with test utility.
B) sets a ceiling on test reliability.
C) sets a ceiling on test utility.
D) None of these

Question 31

When the purpose of a utility analysis is to answer some finance-related question with a dollars-and-cents answer,the utility analysis will most likely employ

A) the Cascio-Ramos formula.
B) the Brogden-Cronbach-Gleser formula.
C) Kuder-Richardson Formula 20.
D) Petty Formula 1.

Question 32

The specific objective of a utility analysis will dictate what sort of information will be required,as well as the specific

A) methods to be used.
B) expectancy tables to be used.
C) Naylor-Shine tables to be used.
D) Rise-and-Shine tables to be used.

Question 33

In the now classic utility analysis for the Bell system telephone company conducted in the 1980s,Cascio and Ramos concluded that

A) the use of a particular approach to assessment in selecting managers could save the company millions of dollars.
B) the use of a new approach to assessment in selecting managers was of little utility and would cost millions of dollars.
C) charging for 411 calls to "Information" could result in millions of dollars in new revenue for the company.
D) customers would make more calls if they could understand their phone bills.

Question 34

The "Achilles heel" of the Angoff method is

A) incremental validity.
B) test-retest reliability.
C) inter-rater reliability.
D) noneconomic costs.

Question 35

The Angoff method of setting cutting scores relies heavily on

A) data and empirical findings.
B) the judgment of experts.
C) researching the scholarly literature.
D) Both a and c

Question 36

Decision theory is used to determine a test's utility for hiring employees who need to have high attention to detail to be successful on the job.In this context,to what does the term hit rate refer?

A) the proportion of people the test accurately identified as having this characteristic
B) the proportion of people the test failed to identify as having this characteristic
C) the number of people the test incorrectly identified as having this characteristic
D) the number of people having this characteristic that the test did not identify

Question 37

An index of utility can be distinguished from an index of reliability and an index of validity in that an index of utility can tell us something about

A) how consistently a test measures what it measures in a particular context.
B) whether a test measures what it purports to measure in a particular context.
C) the practical value of the information derived from what a test measures.
D) None of these

Question 38

This is a tool that can be used to conduct a utility analysis.It is

A) expectancy tables.
B) Taylor-Russell tables.
C) Naylor-Shine tables.
D) All of these

Question 39

The Brogden-Cronbach-Gleser formula was developed by

A) the team of Brogden, Cronbach, and Gleser working together.
B) Brogden, Cronbach, and Gleser, each working independently.
C) the work of Brogden and later, the work of Cronbach and Gleser.
D) the work of Cronbach, and later, the work of Brogden and Gleser.

Question 40

A problem with using the known group method of setting cut score is that

A) there is no consistent method of obtaining contrasting groups.
B) strong deterrents to test user acceptance of the data are in place.
C) no standards are in place for choosing contrasting groups.
D) test users must be personally familiar with each member in the known group.

Question 41

When the selection ratio goes up,

A) the bookmark procedure has less validity.
B) hiring becomes less selective.
C) hiring becomes more selective.
D) hiring is unaffected by the selection ratio.

Question 42

The term bookmark method refers to an IRT-based method

A) of estimating item difficulty using Las Vegas style odds.
B) of setting cut scores based on scholarly book reviews.
C) derived by researchers on sabbatical in Monte Carlo.
D) None of these

Question 43

A potential non-economic benefit of a well-run evaluation program is

A) an increase in quality of workers' on-the-job performance.
B) a decrease in time it takes to train new workers.
C) a reduction in the number of workplace accidents.
D) All of these

Question 44

When the selection ratio for new personnel at a corporation goes down,

A) top-down selection policy can become discriminatory.
B) hiring becomes less selective.
C) competition for the position is likely to increase.
D) Both a and b

Question 45

When a cutscore is set based on norm-related considerations rather than on the relationship of test scores to a criterion,the cut score is referred to as

A) a relative cut score.
B) a fixed cut score.
C) an absolute cut score.
D) a referential cut score.

Question 46

A study that explored the utility of diagnostic X-rays in routine screening of children at risk for child abuse concluded that there was most utility in

A) a four-view series of X-rays.
B) a two-view series of X-rays.
C) one X-ray combined with visual examination.
D) visual examination only.

Question 47

Which is an example of a false negative in the context of employee selection?

A) hired applicants who scored at or above the cut-off score on the employment test and then went on to fail on the job
B) hired applicants who scored at or above the cut-off score on the employment test and then went on to succeed on the job
C) rejected applicants who scored below the cut-off score on the employment test who would have never gone on to succeed on the job
D) rejected applicants who scored below the cut-off score on the employment test who would have succeeded on the job had they been hired

Question 48

In Chapter 7,the pedagogical tool presented to help you remember the implications of a high selection ratio was:

A) "Selection ratio up, more coffee in your cup."
B) "Selection ratio up, the bigger the cut."
C) "Selection ratio high, more employees say 'Goodbye.'"
D) "Selection ratio high, more employees say 'Hi.'"

Question 49

In the context of utility analysis,which is NOT a direct economic benefit to a company?

A) increase in worker morale
B) less product being trashed as waste
C) increase in international sales
D) higher worker productivity

Question 50

The more complex the job,

A) the more people differ on how well they do it.
B) the less complex the decision process in hiring.
C) the less selective the hiring process.
D) the more the need for item-mapping procedures.

Question 51

A hospital uses a compensatory model of selection in hiring surgeons.In their hiring evaluations,ratings regarding past safety record is given more weight than ratings regarding the surgeon's "bedside manner." From this,one could reasonably conclude that the people who are in charge of hiring surgeons believe that:

A) bedside manner is not very important for surgical staff members.
B) surgical safety is an art and a skill that is amenable to training.
C) a safe, bedside manner is essential for all staff members.
D) bedside manner is more amenable to training than surgical safety.

Question 52

In Chapter 7,the pedagogical tool presented to help you remember the implications of a low selection ratio was:

A) "Selection ratio low, more employees to know."
B) "Selection ratio high, watch workers wave good-bye."
C) "Selection ratio down, less employees around."
D) "Selection ratio low, long road to hoe."

Question 53

Which is NOT an economic cost typically factored in a test utility analysis?

A) legal costs of doing business
B) fees paid by testtakers for testing services
C) computerized test processing services
D) cost of supplies of blank test protocols

Question 54

When setting a cut score on a predictor,

A) the Brogden-Cronbach-Gleser is usually used.
B) the utility of the test must first be quantitatively determined.
C) the goal of selection must be taken into account.
D) All of these

Question 55

Which of the following is a direct economic cost that could result as a consequence of NOT evaluating personnel for employment positions within a large corporation?

A) the cost of public confidence in the corporation
B) the cost of accounting services and other routine costs of doing business
C) the cost in terms of lowered morale for employees of the corporation
D) the cost of lawsuits against the corporation

Question 56

Which is an example of a false positive in the context of employee selection?

A) hired applicants who scored at or above the cut-off score on the employment test went on to fail on the job
B) hired applicants who scored at or above the cut-off score on the employment test went on to succeed on the job
C) rejected applicants who scored below the cut-off score on the employment test were rejected but would have gone on to succeed on the job had they been given a chance
D) rejected applicants who scored below the cut-off score on the employment test were rejected but went on to succeed at another, totally different job

Question 57

When setting a cut score on a predictor,an objective is to attain

A) the highest false positive rate.
B) the lowest false negative rate.
C) a moderate hit rate.
D) All of these

Question 58

The term bookmark method refers to an IRT-based method

A) of marking items with regard to difficulty in a book of items.
B) of setting cut scores based on expert judgment.
C) with possible drawbacks such as floor or ceiling effects.
D) All of these

Question 59

The term item-mapping refers to an IRT-based method of

A) setting cut scores that entails expert judgments based, in part, upon how culturally fair items are deemed to be.
B) setting cut scores that entails the use of experts rearranging items placed on maps by level of difficulty.
C) setting cut scores that entails a histographic representation of test items.
D) test construction that was first used for a high school geography achievement test.

Question 60

A large corporation scrupulously avoids any possibility of discrimination and adverse impact in its hiring practices.The selection procedure it probably has in place with regard to its entry-level test is one that entails

A) personnel selection based on a cut score.
B) a top-down selection policy based on test score.
C) personnel interviews using translators, if necessary.
D) None of these

Question 61

When minimum levels of several competencies are required,which of the following selection strategies is BEST?

A) compensatory selection
B) multiple hurdle with fixed cut scores
C) the method of predictive yield
D) norm-referenced cut scores

Question 62

If criterion-related validity is .40,and the base rate is .60,the false positive rate is likely to be highest if the selection ratio is:

A) .10.
B) .50.
C) .95.
D) This is impossible to answer based on the data provided.

Question 63

Over the last year,a personnel manager hired 20 new employees but only 12 of them performed successfully.Here,the base rate of successful performance is

A) 0.375.
B) 0.625.
C) 0.600.
D) 0.750.

Question 64

"It's a method of setting cut scores that relies heavily on judgments made by experts." Which of the following is BEST described by this statement?

A) the known groups method
B) the Angoff method
C) discriminant analysis
D) the Brogden-Cronbach-Gleser method

Question 65

"Multiple predictors may be used so that applicants must meet or exceed the cut score for each predictor before moving to the next round of the selection process." This BEST describes which of the following processes?

A) known-groups selection
B) top down selection
C) compensatory model of selection
D) multiple hurdle selection

Question 66

"An empirical standard used to divide a group of data into two or more distinct categories" is a description of a

A) cut score.
B) predictive yield.
C) norm-referenced test.
D) hit rate.

Question 67

Validity is to ____________ as utility is to ____________.

A) accuracy; consistency
B) usefulness; consistency
C) usefulness; accuracy
D) accuracy; usefulness

Question 68

Using a cut score of 50 on a predictor test a researcher finds a base rate of 1.00.This means that when a cut score of 50 is used,

A) 50% of applicants will perform successfully on a criterion measure.
B) 100% of applicants will perform successfully on a criterion measure.
C) 100% of applicants will fail on a criterion measure.
D) 50% of applicants will fail on a criterion measure.

Question 69

Mahalia applies for a job at a company called "The Medicinal Marijuana Factory" (MMF).MMF routinely administers a drug test during the hiring process.Despite the fact that Mahalia smokes marijuana almost daily,the company's test report indicates that she is drug-free.In this case,the BEST conclusion is that

A) a false positive was reported.
B) a false negative was reported.
C) a confirmed hit was reported.
D) the MMF personnel director needs to be drug tested.

Question 70

Consider an employment test that is highly accurate (about 98% correct in classifications),but very costly (about $5,000 per test).For which of the following positions would use of the test be most warranted?

A) Translator for the United States Ambassador to the U.N.
B) Short-order cook for the President of the United States
C) Assistant buyer for Sam's Club
D) Producer of the reality television show, Mob Wives.

Question 71

Having a personnel selection policy in place that scrupulously seeks to avoid the hiring of unqualified personnel,even at the expense of not hiring qualified personnel,

A) is patently discriminatory.
B) usually has costs attached to it that tend to outweigh the benefits.
C) can be justified if the position is one of great responsibility.
D) usually has benefits attached to it that tend to outweigh the costs.

Question 72

Which of the following is NOT an assumption of utility analysis?

A) the value of people and their performance can be estimated.
B) psychological tests are always preferred over other means of assessment
C) the performance of people in organizations can affect organizational viability.
D) large amounts of information can be integrated to make good decisions.

Question 73

An instructor assigns a grade of "A" to all students who earn 900 or more points out of a total of 1000 points during the semester.In this case,900 points is equal to A.

A) the cut score for an
B) the success rate.
C) the selection ratio.
D) the base rate of A-level students.

Question 74

Which of the following selection ratios reflects the most selective of universities?

A) 0.03
B) 0.50
C) 0.99
D) 1.00

Question 75

Which of the following is included in the calculation of a miss rate?

A) false positives
B) borderline negatives
C) test utility index
D) All of the above

Question 76

A student scores very high on a graduate school admission test and is admitted to graduate school,largely on the basis of his test score.The student subsequently flunks out.In retrospect,the student's score on the admission test is BEST be viewed as a

A) false positive.
B) false negative.
C) true negative.
D) positive hit.

Question 77

The term used to describe the proportion of people in a population who are distinctive due to their exhibition of a particular trait is

A) success rate.
B) base rate.
C) selection ratio.
D) sensitivity.

Question 78

The hit rate is equivalent to

A) the miss rate divided by the selection ratio.
B) the base rate divided by the selection ratio.
C) the success rate divided by the base rate of successful performance.
D) the number of correct classifications divided by the total number of classifications.

Question 79

Offering permanent positions to only top-performing applicants is a strategy that can backfire because

A) competing companies could offer these same applicants positions.
B) these applicants, once hired, might not stay.
C) it may be discriminatory.
D) All of these

Question 80

An educational psychologist conducts a utility analysis of a teaching program used to improve the handwriting of very young children.The measure of utility in this analysis will MOST likely be a variable related to

A) an increase in performance level.
B) a decrease in costs.
C) a reduction in accidents
D) an increase in revenue.