Evaluation Criteria: -
Types of criteria that can be used for these pretests and posttests. There are six levels at which training effectiveness can be measured: content validity, employee reactions, employee learning, application of training, business impact, return on investment, and intangible measures.
Content Validity: -
At times, the only way that training can be evaluated is by comparing training content with the knowledge, skills, and abilities required to perform a job. In other words, the content validity of the training can be examined. For example, if a job analysis indicates that knowledge of electronic circuitry is necessary to perform a job, then a seminar that is designed to teach this knowledge would have content validity.
Although content analysis may ensure that a training program is job related, it still does not indicate whether a particular training method is effective. But if a training program is content valid and is conducted by a professional trainer who can document previous success with the method in other organizations, it may be a safe assumption that the training program will be successful. Keep in mind, however, that making such an assumption is acceptable only when actually evaluating the effect of training is not possible because there are too few employees for a proper analysis or there are financial or political constraints on conducting a proper evaluation.
Employee Reactions: -
The most commonly used method to evaluate training is measuring employee reactions to the training. Employee reactions involve asking employees if they enjoyed the training and learned from the training. Employee reactions are important because employees will not have confidence in the training and will not be motivated to use it if they do not like the training process. However, even though positive employee reactions are necessary for training to be successful, positive employee reactions do not mean that training will lead to changes in knowledge or performance.
Because trainee reactions constitute the lowest level of training evaluation, they can often be misleading. For example, most seminars conducted by outside consultants are informative and well presented, so employee reactions are almost always positive, even though the training may not actually affect knowledge or future performance. For example, as shown in Table 8.4, in a meta-analysis by Alliger, Tannenbaum, Bennett, Traver, and Shotland (1997), employee reactions had only a small correlation with learning and application of training.
Employee Learning: -
Instead of using employee reactions as the criterion in evaluating training performance, actual employee learning can usually be measured (Tyler, 2002). That is, if a training program is designed to increase employee knowledge of communication techniques, then creating a test to determine whether an employee actually learned is possible. This test will be administered before training and then again after the training has been completed. The measurements that will be used for the pretest and posttest, as with selection tests, must be both reliable and valid. Thus, if the purpose of a training program is to increase job knowledge, an appropriate job knowledge test must be constructed or purchased. A trainer can spend a great deal of time a creating a training program and evaluating its effectiveness, but the whole training effort will be wasted if measure used to evaluate effectiveness is no good.
The restaurant industry provides many examples of evaluating training effectiveness through employee learning. At Claim Jumpers Restaurants, servers must pass a 100-item menu test before they are released from training. At Rock Bottom Restaurants, learning is measured by requiring new employees to obtain written statements from their peers verifying that they have mastered their new duties; 20%a do not perform well enough to get the required verification. To make the testing process more fun, Bugaboo Creek Restaurants hold scavenger hunts to ensure that trainees know where everything is located.
At times, reliable and valid measures of training effectiveness are difficult to obtain. Perhaps a good example of this is seen with the human relations seminars that are common to training programs. Typically, an outside consultant conducts a seminar on a topic such as “better communication skills” or “calming irate customers.” A seminar may run from 2 hours to 2 days
in length. Once completed, however, it is important to measure the effectiveness of the seminar training.
Application of Training: -
Another criterion for evaluating the effectiveness of training is the degree of application of training, or the extent to which employees actually can use the learned material. Learning and memorizing new material is one thing, and applying it is another. For example, if employees learn how to deal with angry customers, their ability to apply this material can be measured by observing how they treat an angry customer while they are actually working. Application of training is often measured though supervisor ratings or through the use of the secret shoppers.
Business Impact: -
The fifth criterion that can be used to evaluate a training program’s effectiveness is business impact. Business impact is determined by evaluating whether the goals for training were met. For example, a restaurant such as Buffalo Wild Wings conducts a training program designed to increase sales of “top shelf” liquor. A week after the training, if sales of liquors such as Knob Creek bourbon and Absolute vodka increase, and sales of liquors such as Pepe Lopez tequila and Aristocrat vodka decrease, the training would be considered successful because it has had the desired business impact.
Return on investment: -
The sixth criterion for evaluating the success of a training program is return on investment (ROI). That is, after accounting for the cost of the training, did the organization actually save money? For example, imagine that a bank trains its tellers to cross-sell Visa cards. The tellers rate the training session as being enjoyable (employee reactions), all of the employees pass a test on sales techniques (knowledge), and sales attempts increase by 30% (application). The ROI approach would then ask, “If we spent $5,000 training the tellers, how much more revenue was brought in as a result of the training?” If the answer to the question is more than the amount spent on training, then the program would be considered a success. For example, Parry (2000) demonstrated that training employees how to conduct effective meetings at Southwest Industries cost $15,538 to implement but saved the organization $820,776 in the first year after the training.
