Three Levels of Evaluation: OK, Bad, or Good

When asked to evaluate something, most people’s first response is to consider whether is succeeded or failed, with “better than just success” a usual consideration. When asked to evaluate on a one to five scale, they will simplify the scale to five being excellence, four being success, and everything lower being an indication of some level of annoyance. My hardly scientific review of the evaluations collected by colleagues and myself, over many years, indicates that the average falls between 4.2 and 4.7. The amazing thing is that managers will try to use these biased numbers to actually evaluate an instructor to a finer resolution than the results contain.

As an experiment, ask people, “In a scale of one to five where five is the best, what does three mean?” You will probably get an answer like, “It means average.” Then ask the second question, “When you have been asked to rate something from one to five, how often have you selected one, two, or three?”

Numbering the values during the evaluation process add a level of abstraction that hinders accurate results. If the values are labeled “1, 2, 3” people have to interpret what the numbers mean. If you give them “Poor, Successful, Excellent” as a set of choices, you will probably get more accurate results, particularly if you explain that “successful” will not get the instructor fired.

Would you rate a system using “0, 1, 2” the same as a system using “1, 2, 3”?

It is possible to get a fairly valid five step evaluation, by giving three possibilities, “Poor, Successful, Excellent” by adding an optional remark field. If the remark is more than a restatement of the selection, then you can probably rate it as a beyond excellent or poor. If a person is willing to take the time to write something meaningful, then it means more than just checking a box.

From a training point of view this brings up the problem of asking any question during training, “How do you ask a question so the question is testing knowledge rather than cultural interpretations?”