Chapter 6 Training Evaluation

  1. Training effectiveness
    Refers to the benefits that the company and the trainees receive from training.
  2. Training outcomes or criteria
    Refer to measures that the trainer and the company use to evaluate training programs.
  3. Training Evaluation
    Refers to the process of collecting the outcomes needed to determine whether training is effective.
  4. Evaluation Design
    Refers to the collection of information-- including what, when, how, and from whom-- that will be used to determine the effectiveness of the training program.
  5. Formative Evaluation
    Refers to the evaluation of training that takes place during program design and development. Helps ensure that: the training program is well organized and runs smoothly and trainees learn and are satisfied with the program.
  6. Pilot Testing
    Refers to the process of previewing the training program with potential trainees and managers or with other customers.
  7. Summative evaluation
    Refers to an evaluation conducted to determine the extent to which trainees have changed as a result of participating in the training program.
  8. The evaluationg process
    • 1. Conduct a Needs Analysis
    • 2. Develop Measurable Learning Objectives and Analyze Transfer of Training
    • 3. Develop Outcome Measures
    • 4. Choose and Evaluation Strategy
    • 5. Plan and Execute the Evaluation
  9. Kirkpatrick's Four-level Framework of Evaluation Criteria
    • 4. Results: Business results achieved by trainees
    • 3. Behavior: Improvement of behavior on the job
    • 2. Learning: Acquisition of knowledge, skills, attitudes, behavior
    • 1. Reactions: Trainee satisfaction
  10. Reaction Outcomes
    Refer to trainees perceptions of the program, including the facilities, trainers, and content.
  11. Evaluation Outcomes
    • Reactions
    • Learning or Cognitive
    • Behavior and Skills
    • Affective
    • Results
    • Return on Investment
  12. Cognitive Outcomes
    are used to determine the degree to which trainees are familiar with principles, facts, techniques, procedures, or processes emphasized in the training program.
  13. Skill-Based outcomes
    Are used to assess the level of technical or motor skills and behaviors.
  14. Affective Outcomes
    Include attitudes and motivation.
  15. Results
    Are used to determine the training program's payoff for the company
  16. Return on Investment (ROI)
    Refers to comparing the training monetary benefits with the cost of the training.
  17. Direct costs
    Include: salaries and benefits for all employees involved in training
  18. Indirect costs
    Are not related directly to the design, development, or delivery of the training program. General office supplies, facilities, equipment, and related expenses.
  19. Benefits
    Are the value that the company gains from the training program
  20. Criteria relevance
    Refers to the extent to which training outcomes are related to the learned capabilities emphasized in the training program.
  21. Criterion contamination
    Refers to the extent that training outcomes measure inappropriate capabilities or are affected by extraneous conditions.
  22. Criterion Deficiency
    Refers to the failure to measure training outcomes that were emphasized in the training objectives.
  23. Reliability
    Refers to the degree to which outcomes can be measured consistently over time.
  24. Discrimination
    Refers to the degree to which trainees' perfomance on the outcome actually reflects true differences in performance.
  25. Practicality
    Refers to the ease with which the outcome measures can be collected.
  26. Training Evaluation Practices
    Image Upload 1
  27. Training Program Objectives and Their Implications for Evaluation
    Image Upload 2
  28. Threats to validity
    Refer to factors that will lead and evaluator to question either the believability of the study results or the extent to which the evaluation results are generalizable to other groups of trainees and situations
  29. Threats to validity
    • Internal Validity: Company, Persons, Outcome Measures.
    • External Validity: Reaction to pretest, reaction to evaluation, interactions of selection and training, interaction of methods.
  30. Pretraining measure
    Baseline of the outcome.
  31. Posttraining measure
    Measure of outcome after training
  32. Comparison group
    Refers to a group of employees who participate in the evaluation study but do not attend the training program
  33. Hawthorne Effect
    Refers to employees in an evaluation study performing at a high level simply because of the attention they are receiving.
  34. Random Assignment
    Refers to assigning employees to the training or comparison group on the basis of chance.
  35. Comparison of Evaluation Designs
    • Posttest Only
    • Pretest/posttest
    • Posttest only with comparison group
    • Pretest/posttest with comparison group
    • Time series
    • Time series with comparison group and reversal
    • Solomon Four-group
  36. Factors that influence the type of evaluation design
    Change potential, Importance

    Scale, Purpose of training, Organization Culture

    Expertise, Cost, Time frame
  37. Cost Benefit Analysis
    The process of determining the economic benefits of a training program using accounting methods that look at training costs and benefits.
  38. Utility Analysis
    Cost benefit analysis method that involves assessing the dollar value of training based on estimates of the difference in job performance betwee trained and untrained employees, the number of individuals trained, the length of time a training program is expected to influence performance, and the variability in job performance in the untrained group of employees.
  39. Success cases
    Refer to concrete examples of the impact of training that show how learning has led to results that the company finds worthwhile and the managers find credible.
  40. Training Metrics
    Expenditure/employee, learning hours recieved/employee, expenditure as a % of payroll, expenditure as a % of revenue, cost/learning hour received, % of expenditures for external services, Learning hours received/training and development staff member, average % of learning activities outsourced, average % of learning contant by content area, Average % of learning hours provided via different delivery methods.
Card Set
Chapter 6 Training Evaluation
Chapter 6 key terms