Deck 11: Training Evaluation

Question 1

The "pre-post design with control group" data collection method answers two basic questions: Did trainees change and Was training responsible for that change

Answer 1

True

Question 2

Causal evaluations require more sophisticated experimental and statistical procedures versus the less sophisticated descriptive evaluations.

Answer 2

True

Question 3

Those organizations that perform training evaluations tend to focus their metrics on evaluating training benefits associated with return on investment (ROI),with far fewer focusing on behaviour and reaction metrics.

Answer 3

False

Question 4

Studies of training professionals show the more sophisticated training evaluations are rarely used because they are perceived to be too complicated to implement,take too long to design,and simply cost too much money.

Question 5

When measuring trainees' "reactions" to a training program,trainers often use questionnaires,focus groups,and interviews.

Question 6

The main reason procedural learning measures are NOT often used is centred on the level of complexity required to properly assess procedural learning.

Question 7

Declarative learning refers to the acquisition of facts and information and is by far the most frequently assessed learning measure.

Question 8

The COMA model and the Decision-Based Evaluation model are the most widely used training evaluation methods.

Question 9

Measures of self-efficacy tend to focus on a trainee's perceived level of support in transferring newly learned skills to the job.

Question 10

One of the primary advantages of the Decision-Based Evaluation model over the Kirkpatrick model is that it allows different variables to be measured depending on the target of the evaluation process.

Question 11

For both self-report and observational data,it is important that the measures focus on specific behaviours as opposed to general behaviours because specific behaviours are more likely to be valid and accurate.

Question 12

Kirkpatrick's hierarchical model has four levels.Of these four levels,a survey provided to trainees after training is often associated with collecting data for calculating behavioural changes at work.

Question 13

Training evaluation is a process that helps organizations determine if expected training outcomes actually materialize.

Question 14

One of the main implications of the Kirkpatrick model of evaluation is the interdependencies among the four levels of evaluation.In practical terms,"(L1)should cause trainee learning (L2),which in turn should cause behaviour (L3),and so on."

Question 15

A question such as "Has the training program resulted in payoffs for the organization" is typically associated with summative evaluations according to Worthen and Sanders.

Question 16

According to the Conference Board of Canada,only 46 percent of Canadian organizations undertook some type of training evaluation in 2010.However,in 2012-13 there was evidence to suggest Canadian companies were improving on the 2010 evaluation results.

Question 17

Formative evaluations provide data about the worthiness or effectiveness of a training program.

Question 18

Training evaluation is undertaken for a variety of reasons.Some of the most salient include identifying the costs and benefits of a particular training initiative and determining if expected training outcomes materialized.

Question 19

One of the more rigorous evaluation technique involves measuring the level of satisfaction by trainees after completing a training program.

Question 20

Affective reaction measures the level of perceived usefulness of a training program.

Question 21

Which type of learning measure is associated with positive training transfer

A)declarative learning
B)procedural learning
C)organizational learning
D)observational learning

Question 22

Which of the following statements is NOT congruent with conclusions in theChapter-opening vignette on Bell Canada

A)classroom training seemed to impact learning outcomes the most
B)the traditional training method was ineffective
C)changing the instructors had no effect on learning outcomes
D)the use of a well-developed instruction booklet had greater effect

Question 23

Which of the following statements best describes the state of training evaluation in Canada according to the Conference Board of Canada

A)more than 90 percent of Canadian companies assessed training in some fashion in 2012-13
B)the level of sophistication of training evaluation methods has decreased over the last few years
C)Canadian organizations are increasingly aware of the importance of training evaluation and are investing more resources toward evaluating programs
D)the most common methods of training evaluations are results and behaviour focused

Question 24

Company ABC noticed an increase in customer satisfaction after employees completed a new customer satisfaction training program.As the VP of human resources,you have been asked to determine if in fact the training program led to the increase in customer satisfaction results.Which of the following training evaluation approaches would you take

A)formative evaluation
B)summative evaluation
C)descriptive evaluation
D)causal evaluation

Question 25

For the variable of organizational results,which of the following questions is typical of one used in a training evaluation survey

A)How much has the organization's turnover rate decreased as a result of training?
B)How many times has the manager recognized employees for extra effort displayed?
C)How many of your training managers can identify Kirkpatrick's four levels of evaluation?
D)How satisfied were you with the content of the training program and how much will you apply on the job?

Question 26

What is the most significant advantage of conducting learning tests with trainees

A)When budgets are being reduced, learning tests give objective evidence that they enhance performance and so training should continue.
B)If trainees consistently score low on some aspect of the course, they will need to do more work.
C)If insufficient time has been devoted to developing learning tests, the results could be arbitrarily inflated.
D)In cases of accidents or litigation, the employer can prove the employee was trained adequately.

Question 27

The COMA evaluation model suggests measurement of variables that fall into four categories rather than relying only on declarative and reaction learning outcomes.Which of the following is NOT one of those four categories

A)cognitive variables
B)organizational payoff variables
C)motivation variables
D)attitudes variables

Question 28

Which two types of "reaction measures" have received the most attention in the literature

A)descriptive and causal
B)perceived and anticipated
C)declarative and procedural
D)affective and utility

Question 29

You are putting together a short report addressing why many training programs are poorly evaluated by organizations beyond Kirkpatrick's "level one." Which of the following is NOT likely to be in your report

A)most training managers have been trained on how to properly evaluate programs and senior management often request it
B)management does not request training evaluation and does not provide funding for it
C)trainers and human resource departments (HR) may not be interested in the results, as outcomes may reflect poorly on the HR department
D)lack of time for training evaluation

Question 30

Immediate post-training questionnaire measures remain very popular with trainers.Which of the following is NOT a reason for this popularity

A)they are easy to collect and analyze, and are easily understood by managers and employees
B)they provide trainers with immediate feedback
C)they are an easy and convenient mechanism for measuring levels 1, 2, and 3 of the Kirkpatrick evaluation model
D)they are an easy and convenient mechanism for measuring affective trainee reactions

Question 31

Which of the following is NOT a criticism of Kirkpatrick's hierarchical evaluation model

A)several authors have challenged the validity of the hierarchical nature of the model
B)the model does not offer a solution if and when a program fails to deliver on objectives
C)the model requires that all training evaluation processes use the same variables
D)the model offers a fairly straightforward framework for evaluating training

Question 32

Which training evaluation model is dominant in both research and practical experience

A)Kraiger's hierarchical model
B)Kirkland's hierarchical model
C)Gagné's hierarchical model
D)Kirkpatrick's hierarchical model

Question 33

Your company wants to use the observation method to measure post-training behaviour changes.What advice would you provide your company about using this method

A)instruct management to limit contact with trainees
B)instruct management to focus on specific behaviours displayed by trainees
C)allow management to use judgment in rating performance
D)instruct management to focus on general behaviours displayed by the trainee

Question 34

What are the two broad categories of barriers to training evaluation

A)formative and summative
B)pragmatic and political
C)Kirkpatrick and COMA
D)reaction and behavioural-based

Question 35

Which of the following is NOT a basic approach for measuring post-training behaviour

A)observations
B)production data
C)focus groups
D)self-reports

Question 36

Your company has just spent $100 000 on a new training program for new employees.You have been asked to calculate the return on investment on this new training program.Which level of Kirkpatrick's general model would you use to inform your answer

A)Level 1
B)Level 5
C)Level 4
D)Level 3

Question 37

According to the textbook,which of the following statements is NOT correct when looking at the meaning and purpose of training evaluation processes

A)diagnose the strengths and weakness of a program and pinpoint needed improvements
B)use evaluation information to justify and reinforce the value and credibility of the training function to the organization
C)determine the cost benefits of a program and help ascertain which program or training technique is most cost-effective
D)determine if the training program has given trainees full top management support via objective metrics

Question 38

Which of the following is a key advantage of the Decision-Based Evaluation (DBE)model over the other two evaluation models described in the textbook

A)The DBE model has significant empirical evidence supporting it.
B)Unlike the other models, the DBE model provides the specific variables to be used.
C)DBE allows the program evaluator to modify the variables to be used in the evaluation process to meet specific situations.
D)DBE is much less complex than both COMA and Kirkpatrick's model.

Question 39

Carlos,an instructor at a local university,wants to measure the effectiveness of his training sessions in terms of payoffs for his employer.Which of the following evaluation methods would you recommend to Carlos

A)transfer of training climate evaluation method
B)Kirkpatrick's evaluation scheme, level 1
C)summative evaluation method
D)reaction-based evaluation method

Question 40

Which of the following is considered to be a political barrier to training evaluation

A)senior management may not be stressing the need for training programs to be evaluated against goals and objectives
B)to take resources away to conduct evaluations may be seen as counterproductive
C)proper assessment of a training program requires the use of two precious commodities: time and money
D)taking on the training evaluation task requires knowledge of evaluation models, research design, measurement, questionnaire construction, and data analysis

Question 41

Which of the following evaluation and data collection methods would offer the strongest conclusions on the effectiveness of a training program

A)single-group design
B)pre-post design
C)time series design with control group
D)post-only design

Question 42

You are finding it extremely difficult to link organizational bottom-line improvements to a specific training program.What is an acceptable process that you may use to justify the training program given the lack of linkage between training and organizational outcomes

A)return on investment
B)return on expectations
C)return on training sunk costs
D)return on training expenses

Question 43

Which common shortcoming is shared by all three descriptive training evaluation models described in the textbook

A)the results and data collected are simply not applicable to most training programs
B)the collection of the data is almost always very expensive and time consuming
C)the results and data collection methods do not account for external variables that may contaminate results
D)implementation of descriptive evaluation models requires mastery of statistical analysis techniques

Question 44

What kind of research design would you use if you had very limited resources and were concerned only with measuring improvements in a group of trained employees

A)quantitative design
B)experimental design
C)quasi-experimental design
D)non-experimental design

Question 45

Transfer motivation is critical in supporting transfer of training content.One technique uses the expectancy theory measuring three variables: valence,instrumentality,and expectancies.Which of the following statements is associated with the "instrumentality" variable

A)the positive or negative consequences of transfer
B)the attractiveness of transfer outcomes
C)the probability that transfer will result in successful performance
D)the level of management support

Question 46

Explain the terms declarative learning and procedural learning,describe how each one can be assessed,and state which one is most important to training transfer.

Question 47

Which theory is an important technique used to assess motivation to transfer

A)conditioning theory
B)social cognitive theory
C)ERG theory
D)expectancy theory

Question 48

A training manager is concerned with whether the results of training are based on application of learned skills or some other external/internal variable.Which type of data collection design would address these concerns

A)pre-post
B)post only
C)time series
D)time series with control group

Question 49

Which statement reflects the concept of perceived support

A)"I believe my organization will support me when I apply newly learned skills."
B)"I am somewhat confident my manager will support me when I apply newly learned skills."
C)"I believe my manager most likely will support me when I apply newly learned skills."
D)"My manager coaches me on how best to apply newly learned skills."

Question 50

Identify and briefly describe the four categories of the COMA training evaluation model.

Question 51

a)Identify the four levels of Kirkpatrick's hierarchical model of evaluation and briefly explain the concerns with it.
b)What is the fundamental difference between Level 4 and Level 5 (the new addition to the original four levels of Kirkpatrick's model)

Question 52

Compare and contrast the concepts of perceived support and anticipated support.

Question 53

Briefly explain the differences between non-experimental designs,experimental designs,and quasi-experimental evaluation designs.

Question 54

Why is a "post-only" design the most widely used data collection method

A)it is the easiest data collection method
B)it establishes causality
C)it is statistically the most comprehensive data collection method
D)it is most reliable in terms of conclusions

Question 55

Why would management be interested in a revision of Kirkpatrick's model that includes a fifth element,and are there situations when calculating a return on a training investment is not practical

Question 56

J.Bruce Tracey,Scott Tannenbaum,and Michael Kavanagh proposed a scale to measure which of the following organizational issues

A)rewards and incentives linkage to behaviour changes
B)the culture of the organization as it relates to continuous learning
C)the perceived level of management support and peer support
D)the level of overall organizational support