Deck 10: How Do We Develop a Test

Question 1

In organizational setting, on what do test developers often base their test plan?

A) the need to hire employees
B) a job theory
C) the number of test takers
D) a job analysis

Answer 1

D

Question 2

What are the incorrect responses in a multiple-choice test item called?

A) stems
B) stimuli
C) distracters
D) answers

Answer 2

C

Question 3

In educational settings, test developers base their test plan on ______.

A) a part of the course curriculum
B) a previously conducted job analysis
C) how well students can perform on tests
D) how poorly students are currently performing on tests

Answer 3

A

Question 4

When an individual must achieve a certain score to qualify for membership in a group, which one of the following approaches to test interpretation is being used?

A) normative
B) criterion
C) content
D) predictive

Question 5

Which one of the following is an objective test format?

A) interview
B) essay
C) rating scale
D) multiple-choice question

Question 6

What does a test developer conduct after writing the initial test questions?

A) a review of the testing literature
B) a validation study
C) a pilot test
D) the first administration of the test

Question 7

Tests in which the test taker is required to make up stories about a set of ambiguous pictures would be considered to be ______.

A) objective tests
B) subjective tests
C) projective tests
D) essay tests

Question 8

Which one of the following is a subjective test format?

A) essay
B) multiple choice
C) true/false
D) matching

Question 9

When a test taker displays a pattern of responses that indicates a particular clinical diagnosis of a psychological disorder or the attributes that make up a behavioral trait, what model of scoring is used?

A) cumulative
B) categorical
C) ipsative
D) normative

Question 10

Test developers refer to test stimuli or test questions as ______.

A) univariate items
B) the basis for scoring
C) test items
D) projective questions

Question 11

An academic achievement test where the test taker who achieves the highest score will receive a scholarship is an example of which approach to test interpretation?

A) normative
B) criterion-related
C) construct
D) predictive

Question 12

A test's purpose includes not only what the test will measure but also ______.

A) the test's reliability and validity
B) the type of setting in which the test will be used
C) the theory the test relies on
D) how the test users will use the test scores

Question 13

Which one of the following scoring models assumes that the more the test taker responds in a particular fashion (either with "correct" answers or ones that are consistent with a particular attribute), the more the test taker exhibits the attribute being measured?

A) categorical
B) ipsative
C) cumulative
D) normative

Question 14

According to your textbook, what test item format is used most often?

A) true/false
B) multiple choice
C) short essay
D) forced choice

Question 15

Which one of the following serves as the blueprint for developing a test?

A) test purpose
B) construct definition
C) nomological network
D) test plan

Question 16

The scientific method applies to developing surveys as well as to developing and validating ______.

A) test blueprints
B) psychological tests
C) test plans
D) strategies for test taking

Question 17

Which one of the following characteristics of a test is included in the test plan?

A) the planned date of publication
B) the content of each question on the test
C) an operational definition of the construct
D) a plan for validation

Question 18

The multiple-choice format consists of a question or partial sentence called a/an ______.

A) stem
B) item
C) distracter
D) forced response

Question 19

The first step in test development is to define the ______.

A) test developer, test user, and test publisher
B) subject of the test, the need for the test, and the market for the test
C) testing universe, the target audience, and purpose of the test
D) process for piloting the test

Question 20

What is the target audience?

A) the group who participated in developing a test
B) the group of individuals who will take the test
C) those test takers who participated in the first validation study for the test
D) those test users who ask for a new test to be developed

Question 21

What are response sets?

A) ways of responding that test takers find most comfortable
B) the four or five responses provided in a multiple-choice question
C) equipment, such as computers, used to respond to tests
D) patterns of responding that result in false or misleading information

Question 22

Which one of the following is likely to occur when test takers lack the necessary skills (such as reading) to take the test, do not wish to be evaluated, or lack the attention span necessary to complete the task?

A) random responding
B) social desirability
C) acquiescence
D) withdrawal

Question 23

Test developers are advised to write how many more questions than they will need when developing a new test?

A) 50% more than they will need
B) twice as many as they will need
C) 3 times as many as they will need
D) 4 times as many as they will need

Question 24

A high score on which of the following scale identifies test takers who are uncooperative, responding randomly, or faking?

A) Forced Choice Scale
B) Social Desirability Scale
C) Reliability Scale
D) Validity Scale

Question 25

Which one of the following test formats has little face validity, that is, no apparent connection with the stated purpose of the test?

A) true/false
B) multiple choice
C) forced choice
D) short essay

Question 26

Test and survey developers should balance items for which the correct response would be positive with an equal number of items for which the correct response would be negative to control for which bias?

A) social desirability
B) faking
C) random responding
D) acquiescence

Question 27

Which one of the following is likely to result in ineffective test items?

A) identification of item topics by consulting the test plan
B) ensuring that each item is based on an important learning objective or topic
C) using slang or colloquial language when developing items
D) ensuring that all items are independent

Question 28

What is the tendency of some test takers to provide or choose answers that are perceived to be most acceptable or that present the test taker in a favorable light called?

A) social desirability
B) social acquiescence
C) social faking
D) social responding

Question 29

What are the odds of guessing the correct answer on one true/false question?

A) "1 in 2" or 50%
B) "1 in 3" or 33.3%
C) "1 in 10" or 10%
D) "1 in 50" or 2%

Question 30

According to your textbook, which one of the following test formats do students prefer because it allows them to focus on demonstrating what they have learned rather than limiting them to answering specific questions?

A) true/false
B) multiple choice
C) essay
D) forced choice

Question 31

Which one of the following items is of particular importance when administering subjective tests where multiple people score each test taker?

A) content validity
B) interrater reliability
C) test-retest reliability
D) internal consistency

Question 32

What is the probability of guessing the correct answer on a multiple-choice item with four response choices?

A) .33
B) .25
C) .10
D) .5

Question 33

In organizations, what is a commonly used subjective test?

A) multiple-choice test
B) preemployment test
C) maximal performance test
D) interview

Question 34

"The tendency to agree with any ideas or behaviors presented" is the definition of ______.

A) social desirability
B) acquiescence
C) faking good
D) unrandom responding

Question 35

What does the forced choice format require the test taker to do?

A) provide an answer to each question even when the test taker does not know the answer
B) choose from one of two or more words/phrases that appear unrelated but equally acceptable
C) choose one of several answers that appear equally acceptable
D) respond to the item by telling a story or drawing a picture

Question 36

Which one of the following response sets may have a cultural basis as well as being an individual inclination?

A) social desirability
B) acquiescence
C) faking
D) random responding

Question 37

Which one of the following types of questions provides a freedom of response that facilitates assessing higher cognitive behaviors such as analysis, synthesis, and evaluation?

A) essay
B) forced choice
C) multiple choice
D) true/false

Question 38

Which test format would the stem, "Which one word from each of the following pairs of words is most descriptive of you?" be part of?

A) short essay format
B) multiple-choice format
C) true/false format
D) forced choice format

Question 39

Which one of the following formats presents an incomplete sentence such as "I feel happiest when I am . . ."?

A) essay
B) multiple choice
C) sentence assessment
D) sentence completion

Question 40

Which one of the following formats would be best to use when the test takers are likely to answer dishonestly?

A) multiple choice
B) forced choice
C) true/false
D) structured interview

Question 41

Describe the testing universe, the target audience, and the purpose of the test.

Question 42

Develop a mock test plan, including defining the construct the test measures, choosing the test format, and specifying how to score the test. Explain each decision you make in developing the test plan.

Question 43

What are the three models for scoring discussed in your textbook? Define and give an example of each.

Question 44

"Consider the time necessary for the response" is a guideline for writing ______.

A) multiple-choice questions
B) forced choice questions
C) projective questions
D) essay questions

Question 45

What are the differences between objective and subjective test items? What are the strengths and weakness of each?

Question 46

Which one of the following settings would be most appropriate for piloting a test developed to screen new outpatients at a mental health clinic?

A) college students in an introductory psychology class
B) college students at an on-campus mental health clinic
C) adults who come to a mental health clinic for treatment
D) adults who come to several mental health clinics for treatment

Question 47

When subjective test formats are used and the test universe covers a wide range of topics, evidence of validity based on the test's content usually ______.

A) increases significantly
B) increases somewhat
C) remains stable
D) decreases

Question 48

Which one of the following jobs would most likely require a portfolio be presented as part of the selection process?

A) sales executive
B) entry level accountant
C) human resources clerk
D) commercial artist

Question 49

What should the test developer do when the data from a pilot test suggest that the assessment instrument is not reliable or valid?

A) conduct another pilot study with different participants in a different setting
B) interview the respondents to find out why the test did not work
C) make changes to improve reliability and validity before allowing the test user to administer it
D) make the necessary changes and afterward conduct another pilot test with different participants

Question 50

Specific requirements for the test administration location such as privacy, quiet, and comfortable chairs, tables, or desks should be part of the instructions for ______.

A) test takers
B) test administrators
C) test developers
D) scorers

Question 51

Which one of the following would NOT be considered a complex item format?

A) portfolio
B) essay question
C) simulation
D) performance assessment

Question 52

Which one of the following question formats is relatively easy to develop but takes a great amount of time to score?

A) multiple choice
B) essay
C) forced choice
D) projective

Question 53

Which one of the following is a specific guideline for writing multiple-choice items?

A) make all responses similar in detail and length
B) use words such as "always" or "never" to distinguish distractors
C) use inclusive distractors
D) avoid having one choice that is clearly correct

Question 54

Explain the multiple-choice format. What are the components of a multiple-choice item? What are the strengths and weaknesses of the multiple-choice format? Make up a multiple-choice question to use as your example.

Question 55

How are the instructions for the test taker typically delivered?

A) test taker orally reads a prepared script written by the test developer
B) test administrator plays an audio recording of the test developer reading the instructions
C) test taker reads a script written by the test developer
D) test administrator orally reads a prepared script written by the test developer

Question 56

What is the purpose of a pilot test?

A) market a new test to potential test users
B) increase reliability and validity
C) get respondents reactions to test questions
D) find out how well the test performs

Question 57

When a test developer tries to offset the effects of acquiescence by balancing positive statements with negative statements, the test will require ______.

A) reverse scoring
B) cumulative scoring
C) ideographic scoring
D) computer scoring

Question 58

A scientific investigation of a new test's reliability and validity for its specified purpose is called a ______.

A) pilot test
B) reliability and validity study
C) construct-validation study
D) field test

Question 59

Test developers may prefer to use a simulation rather than a performance assessment under which of the following circumstances?

A) when face validity is a critical issue
B) when the behaviors measured may be dangerous to perform
C) when the test developer needs to measure complex behaviors
D) when the test developer expects that test bias may be a problem

Question 60

A pilot test provides ______ data about the test.

A) quantitative
B) qualitative
C) quantitative and qualitative
D) administrator and test taker

Question 61

Describe how a pilot test should be set up and specify the types of information that should be collected.

Question 62

Explain the importance of conducting a pilot test. Give examples of what can happen when the test developer does not conduct a pilot test.

Question 63

Discuss three types of response bias. Give examples.

Question 64

Discuss at least four guidelines for writing multiple choice and true/false items. Give an example of each guideline.