Q.5 Briefly explain how we can evaluate training result.
Ans. One important characteristic of industrial psychology is its reliance on research. Evaluating training results is a good example of this reliance. Because training programs can be costly in both time and money, it is essential that they be evaluated to determine if they can be improved, should continue to be offered, and whether it significantly increases performance or affects positive changes in behavior.
Research Designs for Evaluation: -
There are many ways to evaluate the effectiveness of a training program, and two factors differentiate the various methods. The first involves practicality, and the second is concerned with experimental rigor. Although scientifically rigorous research designs are preferred, their use is not always possible. Yet a practical research design without scientific rigor yields little confidence in research findings.
The most simple and practical of research designs implements a training program and then determines whether significant change is seen in performance of job knowledge. To use this method, performance or job knowledge must be measured twice. The first measurement, a pretest, is taken before the implementation of training. The second measurement, a posttest, is taken after the training program is complete. A diagram of this simple pretest—posttest design is as follows:
Pretest ® 4 Training ® 4 Posttest
Although this method is fairly simple, its findings are difficult to interpret because there is no control group against which the results can be compared. That is, suppose a significant difference in performance is seen between the pretest and the posttest. If a training program has occurred between the two tests, it would be tempting to credit the training for the increase. The increase, however, may have resulted from other factors, such as changes in machinery, in motivation caused by non-training factors, or in managerial style or philosophy.
Likewise, suppose no significant increase in performance is observed between pretest and posttest. The natural conclusion might be that the training program did not work. Without a control group, that interpretation is not necessarily correct. The same changes noted above for an increase may have caused a decrease in performance in this second case. Thus, it is possible that the training program actually did increase performance but that other factors reduced it, which resulted in no net gain in performance from training.
To overcome these problems, a control group should be used (Kearns, 2001). For training purposes, a control group consists of employees who will be tested and treated in the same manner as the experimental group except that they will not receive training. The control group will be subject to the same policy, machinery, and economic conditions as the employees in the experimental group who receive training. The diagram for a pretest/posttest control group design looks like this:
Experimental group : Pretest ® Training ® Posttest
Control group : Pretest ® Posttest
The big advantage this second design has is that it allows a researcher to look at the training effect after controlling for outside factors.
R. R. Donnelley & Sons increased their annual commissions by $22,000, and the company increased profits by
$34 million. The company was obviously pleased but was worried that the increased performance could have been due to something other than training. So it compared the results with a control group of employees who had not received training. The commissions of the control employees increased by $7,000 over the same period. Thus, the net effect of the training was still sizable—$15,000 per employee—but not as high as the $22,000 originally thought. The control group allowed the company to control for such factors as increased sales agent experience and new company promotions.
Even though this design is an improvement on the first, it too has its drawbacks. First, except for training manipulation, it is almost impossible to treat a control group the same as the experimental group. Control groups often consist of workers at other plants or on other shifts at the same plant. Such groups are used because there often is no alternative. But the fact that they are in different environments reduces confidence in the research findings.
Even if employees in the same plant on the same shift can be randomly split into control and experimental groups, problems will still exist. The most glaring of these involves the possibility that because the two groups are close to one another, the training effect for the experimental group will spill over to the control group. Employees in the control group also may resent not being chosen for training. This resentment alone may lead to a decrease in performance by employees in the control group. Finally, it is possible that the untrained employees will pressure the newly trained employees to revert to the “old way” of doing things.
With both of the above designs, the pretest itself presents a problem. That is, the mere taking of a test may itself lead to increases in performance. Because of this, a rather complicated method called the Solomon four-groups design can be used (Campbell a Stanley, 1963). With this design, one group will undergo training but will not take the pretest, a second group will undergo training but will take the pretest, a third group will not undergo training but will take the pretest, and a fourth group will neither undergo training nor take the pretest. The diagram for this design is as follows:
Group 1 Training ® Posttest
Group 2 Pretest ® Training ® Posttest
Group 3 Pretest ® Posttest
Group 4 Posttest
This design allows a researcher not only to control for outside effects, but also to control for any pretest
effect. This is the most scientifically rigorous of the research designs used to evaluate training, but even it
has a serious drawback: It is often not practical. That is, four groups of employees must be used, two of which do not receive training. Thus, to use this design at one organization or plant, ideally a relatively large number of employees must be available and kept from discussing the training with one another.
