Deck 6: Training Evaluation

Question 1

A firm that aims at improving readily observable outcomes such as productivity by collecting related data at periodic intervals both before and after training is typically applying the _____ evaluation design.

A) Solomon four-group
B) pretest/post-test
C) return on investment
D) time series

Answer 1

D

Question 2

Which of the following is a threat to internal validity?

A) Characteristics of trainees influence program effectiveness.
B) Being evaluated causes participants to try harder in training program.
C) Trainees pay extra attention to material on a test before training.
D) Training group differs from comparison group on individual differences that influence outcomes.

Answer 2

D

Question 3

Behavior or skill-based outcomes are best measured by _____.

A) surveys
B) interviews
C) focus groups
D) observations

Answer 3

D

Question 4

Which of the following types of training outcomes are collected when trainees are asked to rate their level of satisfaction with the trainer at the end of a training program?

A) Result
B) Cognitive
C) Reaction
D) Behavior-based

Question 5

Appropriate training outcomes need to be discriminative which implies that:

A) tests given to employees before and after a training program should differ.
B) trainees should be asked to take a reliable test that includes items for which the meaning or interpretation change over time.
C) trainees' performance on the outcome should actually reflect true differences in performance.
D) different employees should be given different tests for measuring their performance on the same outcome.

Question 6

_____ refers to the process of collecting the outcomes needed to determine whether training is effective.

A) Training evaluation
B) Program design
C) Performance appraisal
D) Needs assessment

Question 7

If trainers are interested in the generalizability of a study's results to other groups and situations,then they are said to be interested in the _____ of the study.

A) outcome practicality
B) criterion relevance
C) external validity
D) outcome believability

Question 8

Formative evaluation involves collecting qualitative data about a training program from trainees mainly through:

A) their opinions and feelings about the program.
B) measures of performance such as volume of sales.
C) tests and ratings of their behavior.
D) the return on investment method.

Question 9

If a manager believes that a training program is valuable and hence provides higher ratings of job performance to those trainees who attended the training program,then the training outcomes will lack relevance due to _____.

A) criterion contamination
B) criterion deficiency
C) criterion irreducibility
D) criterion maturation

Question 10

Which of the following is an affective outcome collected in a training evaluation?

A) Learners' satisfaction with training.
B) Customer service orientation.
C) Increased production.
D) Reduced costs of production.

Question 11

Typically,_____ are used to assess cognitive outcomes.

A) attitude surveys
B) observations
C) focus groups
D) pencil-and-paper tests

Question 12

If a firm measures its sales volume before and after a training program,it is typically assessing the _____ outcome.

A) reaction
B) behavior-based
C) learning
D) results

Question 13

According to Kirkpatrick's framework for categorizing training outcomes,_____ outcomes are level 1 criteria that are often called class or instructor evaluations.

A) learning
B) reaction
C) skill-based
D) affective

Question 14

The _____ training outcome is collected to determine whether trainees are using training content back on the job.

A) reaction
B) learning
C) cognitive
D) result

Question 15

The evaluation process ideally begins with:

A) developing outcome measures.
B) choosing an evaluation strategy.
C) developing measurable learning objectives.
D) conducting a needs analysis.

Question 16

Which of the following is true of return on investment (ROI)?

A) It is referred to as level 3 evaluation in Kirkpatrick's framework for categorizing training outcomes.
B) It refers to comparing sales activity before and after a training program.
C) Dollar value of productivity divided by training costs reflects return on investment.
D) The method of measurement for return on investment typically involves tests, surveys, and interviews.

Question 17

Which of the following statements best differentiates formative evaluation from summative evaluation?

A) Formative evaluation mainly involves collecting quantitative data, whereas summative evaluation involves collecting qualitative data.
B) Formative evaluation takes place on the completion of training, whereas summative evaluation takes place during program design and development.
C) Formative evaluation focuses on how to make a training program better, whereas summative evaluation helps to determine the extent to which trainees have changed after training.
D) Formative evaluation includes measuring the monetary benefits that a company receives from a training program, whereas summative evaluation measures beliefs and opinions of the trainees.

Question 18

Which of the following statements is true of random assignment?

A) It assigns employees to a training program without considering their individual differences.
B) It results in an unequal distribution of individual characteristics such as age, gender, and motivation in the comparison group and the training group.
C) It increases the effects of employees dropping out of the study (mortality).
D) It increases the differences between the training group and comparison group in ability, knowledge, skill, or other personal characteristics.

Question 19

Which of the following statements is true of comparison groups?

A) The use of a comparison group poses a threat to internal validity.
B) Comparison group consists of a group of employees who do not attend the training program.
C) Employees in a comparison group have personal characteristics that are different from other trainees.
D) Use of a comparison group in training evaluation increases the possibility that changes found in the outcome measures are due to factors other than training.

Question 20

_____ refers to the ease with which training outcome measures can be collected.

A) Reliability
B) Practicality
C) Acceptability
D) Relevance

Question 21

Evaluation designs without pretest or comparison groups are most appropriate when:

A) information regarding training effectiveness is not needed immediately.
B) companies are interest in determining how much change has occurred.
C) a company has a strong orientation toward evaluation.
D) a company is only interested in whether trainees have achieved a certain proficiency level.

Question 22

_____ is a time period in which participants no longer receive training intervention.

A) Regression
B) Mortality
C) Reversal
D) Maturation

Question 23

Reaction and cognitive outcomes do not help determine how much trainees actually use training content in their jobs.

Question 24

Formative evaluation usually involves collecting quantitative data through tests or objective measures of performance.

Question 25

ROI analysis is best suited for training programs that are:

A) attended by few employees.
B) inexpensive and have limited visibility.
C) focused on an operational issue.
D) one-time events.

Question 26

A comparison group refers to a group of employees who participate in the evaluation study but do not attend the training program.

Question 27

In Kirkpatrick's five-level framework for categorizing training outcomes,the levels indicate the importance of the outcomes.

Question 28

Results belong to the level 1 criteria in Kirkpatrick's framework for categorizing training outcomes.

Question 29

The Hawthorne effect refers to employees in an evaluation study performing at a low level because of the attention they receive.

Question 30

No evaluation design can ensure that the results of the evaluation are completely due to training.

Question 31

_____ refers to concrete examples of the impact of training that show how learning has led to results that a company finds worthwhile and managers find credible.

A) Utility analysis
B) Success case
C) Return on expectation
D) Outcome practicality

Question 32

Which of the following statements is true of Solomon four-group evaluation design?

A) It is mainly used by companies that are uncomfortable with excluding certain employees or that intend to train only a small group of employees.
B) It is an evaluation design in which pre-training outcomes are completely ignored.
C) It combines both pretest/post-test comparison group and post-test-only control group design.
D) Relative to the other evaluation designs it is more economical and takes lesser time to conduct.

Question 33

Evaluation designs without pretest or comparison groups are most appropriate when companies are interested in determining how much change has occurred in trainees.

Question 34

A reliable test includes items for which the meaning or interpretation does not change over time.

Question 35

Outcome measures are perfectly related to each other.

Question 36

If instructional objectives identify business-related outcomes such as increased customer service,then reaction outcomes should be used in the evaluation.

Question 37

The believability of study results refers to external validity.

Question 38

Pilot testing refers to the process of previewing the training program with potential trainees and managers or with other customers.

Question 39

A pencil-and-paper test is ideal for measuring skill-based outcomes.

Question 40

The influence of training is largest for organizational performance outcomes and weakest for financial outcomes.

Question 41

Return on investment (ROI)is used to show a training program's cost effectiveness before it has been delivered.

Question 42

Describe the evaluation process for training.

Question 43

A training activity dashboard does not provide a perspective of future learning.

Question 44

What conditions require a company to consider a rigorous evaluation design (pretest/post-test with comparison group)?

Question 45

How does workforce-analytics help determine the value of learning activities?

Question 46

Once the costs and benefits of the training program are determined,ROI is calculated by dividing return or benefits by costs.

Question 47

Identify the ways to minimize threats to validity.

Question 48

Why is it important to evaluate a training program?

Question 49

The Solomon four-group design combines the pretest/post-test comparison group and the post-test-only control group design.

Question 50

It is necessary to limit return on investment analysis only to certain training programs as it can be costly.