Levels of Evaluation

The Levels of Evaluation

Evaluation: It's a tough one for sure. I noticed long ago when I was teaching new ROTC instructors how to teach, that college textbooks often came with a testbank book. Upon seeing this, I thought this was a great idea, as it would alleviate the instructor from writing test questions and provide quality test items. Well, I was sorely wrong. I looked over many college textbooks over the next few months and their accompanying test bank booklets. To put it bluntly, these test banks were quite worthless. All of the ones I reviewed had two serious flaws. One, all of the questions were at the Knowledge level or at best, at low Comprehension level. These junior and senior level classes should have asked for more than just Knowledge or low Comprehension, yet the test banks never touched on the higher levels of Bloom's Taxonomy. Second, most of the questions were almost and some exact, verbatim from the text. Worse than that, they were not tied to the lesson objective through any kind of Sample of Behavior or Competency. After some additional research, it ocurred to me that having a a test bank booklet readily available along with the very expensive text book greatly enhanced the sales volume of the text book as the instructor was relieved of the tedious task of test question development. I did more research among my college professor colleagues and quickly determined that not only did they not know how to develop good student evaluations, but had little interest in doing so. I'm sure in my heart that there are many college instructors who want to develop and administer quality student assessments, I've just not seem many. All of those I have met interested in quality student assessments had degrees in education, ISD or similar areas. That simply indicates to me, somewhere along the ling, they were taught the importants of student assessments.

I developed a saying during these years, "Every student deserves the opportunity to score 100% on every assessment", but they will not. Students will self stratify if the assessment is valid. I have taught many people the importance of conducting accurate student assessments and I will provide a short introduction to that on this page. Let's start by reviewing the traditional "4 Levels of Evaluation".

I'm going to assume you are at least familiar with Donald Kirkpatrick's work on the 4 Levels of Evauation. If not, I recommend going to the Kirkpatrick website and becoming intimately versed on this seminal work. It is the foundation for so much in the world of student evaluation. You can find their work here http://kirkpatrickpartners.com/ or on many other websites. Just "Bing" or "Google" if you prefer, the 4 levels and you will find plenty of information on this process.

My focus in this writing is on the additional levels. Level 5, is commonly called Return On Investment, by its leading proponent, Jack Phillips. You can learn all you ever wanted to know about Return On Investment from the ROI Institute, located here: http://www.roiinstitute.net/ Having attended several of Jack and Patti Phillips' workshops and certification courses, I can highly recommend their work if you are in need of developing a Return On Investment.

The other levels on my chart above start at Level 0. I call it level 0 in order to not juggle the other well establlished levels of evaluation and also because it should come before the actual delivery of any learning event. Level 0 is a curriculum review process which follows the JADDIE ISD model and requires content validation and also delivery validation prior to the actual delivery of curriculum to students.

The last level on my chart is Level 6, which of course is actually Level 7 in total. This is an annual curriculum review that incorporates all of the data from the other 6 levels. Even if you are not doing Level 1, 2, 3, 4, or 5, Level 6 is essential to determine the accuracy and validity of your curriculum.