Deck 11: Training Evaluation

Question 1

Reaction measures are usually done via a survey immediately after the training session.

Answer 1

True

Question 2

Declarative learning refers to the acquisition of facts and information and is by far the most frequently assessed learning measure.

Answer 2

True

Question 3

Measures of self-efficacy tend to focus on a trainee's perceived level of support in transferring newly learned skills to the job.

Answer 3

False

Question 4

Evaluating training programs requires knowledge about research design,measurements,and data analysis.

Question 5

The main reason procedural learning measures are NOT often used is centred on the level of complexity required.

Question 6

Observations are the most popular measures of behaviour changes resulting from training initiatives.

Question 7

Descriptive and causal evaluations measure basically the same outcome.

Question 8

The "pre-post design with control group" data collection method answers two basic questions: Did trainees change? And,was training responsible for that change?

Question 9

The COMA model and the Decision-Based Evaluation model are the most widely used training evaluation methods.

Question 10

Affective reaction measures the level of perceived usefulness of a training program.

Question 11

One of the primary advantages of the Decision-Based Evaluation model over the Kirkpatrick model is that it allows different variables to be measured depending on the target of the evaluation process.

Question 12

Decisions to evaluate training programs are often made as a direct result of senior management requests.

Question 13

Training evaluation is undertaken for a variety of reasons.Some of the most salient reasons include identifying the costs and benefits of a particular training initiative and determining if expected training outcomes materialized.

Question 14

Kirkpatrick's hierarchical model has four levels.Recently,Kirkpatrick replaced the fourth level with a measurement of return on investment.

Question 15

Formative evaluations provide data about the worthiness or effectiveness of a training program.

Question 16

Those organizations that perform training evaluations tend to focus their metrics on job performance improvements as a direct result of training.

Question 17

Training evaluation is a process that helps organizations determine if expected training outcomes actually materialize.

Question 18

The American Society for Training and Development reported that organizations rarely perform any kind of training evaluation.

Question 19

Training evaluation is basically a single procedure focused on the level of satisfaction reported by trainees after training.

Question 20

For both self-report and observational data,it is important that the measure focus on specific behaviours as opposed to general ones because specific ones are more likely to be valid and accurate.

Question 21

Which type of learning measure is associated with positive training transfer?

A) declarative learning
B) procedural learning
C) organizational learning
D) observational learning

Question 22

What are the two types of reaction measures that are usually used in a training evaluation?

A) descriptive and causal
B) perceived and anticipated
C) declarative and procedural
D) affective and utility

Question 23

For the variable of organizational results,which question below is typical of one used in a training evaluation survey?

A) How much has the organization's turnover rate decreased as a result of training?
B) How many times has the manager recognized employees for extra effort displayed?
C) How many of your training managers can identify Kirkpatrick's four levels of evaluation?
D) How satisfied were you with the content of the training program and how much will you apply on the job?

Question 24

Which of the following is NOT a basic approach for measuring post-training behaviour?

A) observations
B) production data
C) multiple-choice tests
D) self-reports

Question 25

According to the Conference Board of Canada,which of the following statements best describes the state of training evaluation in Canada?

A) more than 50 percent of Canadian companies assessed training in some fashion in 2010
B) the level of sophistication of training evaluation methods has increased over the last few years
C) Canadian companies appear to be disillusioned with training evaluation results
D) the most common methods of training evaluations are results and behaviour focused

Question 26

Which of the following is considered to be a pragmatic barrier to training evaluation?

A) Senior management may not be stressing the need for training programs to be evaluated against goals and objectives.
B) Training departments may be assessed based on the number of programs offered. To take resources away to conduct evaluations may be seen as counterproductive.
C) Some trainers may feel that poor results may reflect poorly on the trainer's performance.
D) Some trainers use conflict of interest arguments to forgo training evaluations.

Question 27

The textbook offers four ways the COMA model improves upon Kirkpatrick's hierarchical model.Which of the following is NOT one of those improvements?

A) much more useful for formative evaluations
B) improves upon measures related to summative evaluations
C) the measures used for evaluation are causally related to training success
D) adds to normal reaction surveys by including more variables

Question 28

If management wants to use observation as an assessment method to measure behaviour transfer,how would a human resources representative instruct management so as to mitigate the main issue associated with observation techniques?

A) instruct management to limit contact with trainees
B) instruct management to focus on specific behaviours displayed by trainees
C) allow management to use judgment in rating performance
D) instruct management to focus on general behaviours displayed by the trainee

Question 29

What is the most significant advantage of conducting learning tests with trainees?

A) When budgets are being reduced, learning tests give objective evidence that they enhance performance and so training should continue.
B) If trainees consistently score low on some aspect of the course, they will need to do more work.
C) If insufficient time has been devoted to developing learning tests, the results could be arbitrarily inflated.
D) In cases of accidents or litigation, the employer can prove the employee was trained adequately.

Question 30

What is the dominant training evaluation model in both research and practical experience?

A) Kraiger's hierarchical model
B) Kirkland's hierarchical model
C) Gagné's hierarchical model
D) Kirkpatrick's hierarchical model

Question 31

Why are immediate post-training reaction measures so common?

A) Empirical evidence suggests that reaction measures are the most correlated with performance after training.
B) Trainers feel that post-training reaction measures results provide an accurate measure of a trainer's abilities and performance.
C) They are easy to collect and analyze and are easily understood by both managers and trainees.
D) Management wants to make sure value is extracted from their training investments.

Question 32

What are the two broad categories of barriers to training evaluation?

A) formative and summative
B) pragmatic and political
C) on-the-job and off-the-job
D) reaction and behavioural-based

Question 33

Carlos,an instructor at a local university,wants to measure the effectiveness of his training sessions.Which of the following evaluation methods would you recommend to Carlos?

A) transfer of training climate evaluation method
B) ISD evaluation method
C) summative evaluation method
D) reaction-based evaluation method

Question 34

What is the concept that uses data to demonstrate a linkage between training and post-training behaviour?

A) formative evaluations
B) summative evaluations
C) descriptive evaluations
D) causal evaluations

Question 35

According to the textbook,which of the following statements is NOT correct when looking at the meaning and purpose of training evaluation process?

A) Diagnose the strengths and weakness of a program and pinpoint needed improvements.
B) Use the evaluation information to justify and reinforce the value and credibility of the training function to the organization.
C) Determine the cost benefits of a program and to help ascertain which program or training technique is most cost-effective.
D) Determine if the training program has given the trainees full top management support.

Question 36

Which of the following is NOT a criticism of Kirkpatrick's hierarchical evaluation model?

A) Several authors have challenged the validity of the hierarchical nature of the model.
B) The model does not offer a solution if and when a program fails to deliver on objectives.
C) The model requires that all training evaluation processes use the same variables.
D) The model offers a fairly straightforward framework for evaluating training.

Question 37

A training manager wants to determine what type of payback he/she will receive from a particular training investment.Which level of Kirkpatrick's general model would he/she use?

A) Level 1
B) Level 5
C) Level 4
D) Level 3

Question 38

Which of the following is a key advantage of the decision-based evaluation (DBE)model over the other two evaluation models described in the textbook?

A) The DBE model has significant empirical evidence supporting it.
B) Unlike the other models, the DBE model provides the specific variables to be used.
C) DBE allows the program evaluator to modify the variables to be used in the evaluation process to meet specific situations.
D) Much less complex than both COMA and Kirkpatrick's model.

Question 39

What was the main conclusion arrived at in the Bell Canada vignette introduced in the textbook?

A) changing a trainer had an effect on results
B) lecture-style instruction had an effect on results
C) the usage of instructional booklets had an effect on results
D) Bell Canada needs to pay close attention to training costs

Question 40

The textbook identifies several contributory factors causing companies to forgo training evaluations.Which of the following have NOT been identified?

A) not demanded by trainers and training program designers
B) management does not request it
C) trainers and human resource departments may not be interested in the results
D) lack of time for training evaluation

Question 41

Explain the terms declarative learning and procedural learning,describe how each one can be assessed,and state which one is most important to training transfer.

Question 42

Why is a post-only design the most widely used data collection method?

A) it is the easiest data collection method
B) it establishes causality
C) it is statistically the most comprehensive data collection method
D) it is most reliable in terms of conclusions

Question 43

Which theory is an important technique used to assess motivation to transfer?

A) conditioning theory
B) social cognitive theory
C) ERG theory
D) expectancy theory

Question 44

When subjects are randomly assigned,which of the following is the simplest experimental design?

A) single-group design with control
B) pre-post design with control
C) time series design with control
D) post-only design with control

Question 45

Which of the following are good practices suggested by the textbook for collecting behaviour data?

A) allow sufficient time for trainees to practise and apply skills before collecting data
B) collect data shortly after training to ensure knowledge retention is highest
C) collect data at a single point in time
D) inform trainees when and where you will be collecting behavioural data

Question 46

Identify and briefly describe the four categories of the COMA training evaluation model.

Question 47

Why would management be interested in a revision of Kirkpatrick's model that includes a fifth element?

Question 48

Identify the four levels of Kirkpatrick's hierarchical model of evaluation and briefly explain the concerns with it.

Question 49

Rouiller and Goldstein developed several cues to assess transfer climate.Which of the following is NOT an example of such cues?

A) John, well done with that customer complaint
B) Our trainer was very knowledgeable
C) My organization promotes a "learning culture"
D) My peers are supportive and help me develop my skills

Question 50

Which statement reflects the concept of perceived support?

A) I believe my organization will support me when I apply newly learned skills.
B) I am somewhat confident my manager will support me when I apply newly learned skills.
C) I believe my manager will support me when I apply newly learned skills.
D) My manager coaches me on how best to apply newly learned skills.

Question 51

Briefly explain the differences between non-experimental designs,experimental designs,and quasi-experimental evaluation designs.

Question 52

A training manager is concerned with whether results of training are based on application of learned skills or some other external/internal variable.Which type of data collection design would address his/her concerns?

A) pre-post
B) post only
C) time series
D) time series design with control group

Question 53

What kind of research design would you use if you had very limited resources and were only concerned with measuring improvements in a group of trained employees?

A) quantitative design
B) experimental design
C) quasi-experimental design
D) non-experimental design

Question 54

Compare and contrast the concepts of perceived support and anticipated support.

Question 55

What are the two test items used in the internal referencing strategy developed by Haccoun and Hamtiaux?

A) self-efficacy and self-control
B) quasi-experimental and experimental
C) relevant and irrelevant but germane
D) relevant and irrelevant

Question 56

What is the concept that provides the measurement of a training program's ability to meet managerial goals?

A) return on investment
B) return on expectations
C) return on equity
D) return on objectives