THE FOUNDATION OF GOOD CURRICULUM DEVELOPMENT AND EVALUATION
A basic understanding of the Instructional Systems Development (ISD) process is required to understand how to develop any education or training content, including evaluation content. This step-by-step approach ensures an understandable product is presented to the student for their learning environment. The basic instructional systems development process consists of the following master phases:
Each of these master phases contain many task steps that, when combined, create a product the learner can deal with appropriately. Many steps are unique to particular learning products; that is, some steps are only used for Face to Face (F2F) products, some are only used for eLearning products and so on. Using the appropriate steps ensures courses designed for trainees are instructionally sound, accessible, tailored to specific performance needs, and capable of being produced in an efficient and streamlined manner.
There are a variety of well-known ISD processes, but almost all use some form of the process known as ADDIE for the five master phases of Analysis, Design, Development, Implementation, and Evaluation. Like so many practitioners of the art of learning, I too have developed my own version of ISD that has served me well in a myriad of curriculum development efforts. I have long been a proponent of using a system, most any system, when developing curriculum, but when I was going through my Project Management Professional certification classes, I found what I feel is the best way to approach curriculum development and thus, curriculum evaluation.
According to the Project Management Body of Knowledge (called the PMBOK), there are 5 phases of project management. And what is curriculum building if not a project? The 5 phases of project management are:
Monitor and Control
It occurred to me that this sounded similar to many of the ISD models I have read. It also always bothered me that there was no “planning” phase to the traditional ISD model, although some people use the PADDIE model, adding a "Planning Phase" to the traditional ADDIE model. So after thinking about this, it occurs to me that the Execution Phase of the PMBOK model is really the heart of project management and that is actually the ADDIE model. I have combined the two models into my idea of an “ideal” (at least for now) version of ISD. Since there are many, many versions of ISD models, and most are named for their creators, I have taken the liberty of naming this model, JADDIE (Joe’s ADDIE). Here is the graphic I’ve developed to demonstrate how the ADDIE and PMBOK models are complementary in the JADDIE model.
Any process requires actors to act upon that process and JADDIE is no exception. At a minimum, two skill sets are required, an Instructional Systems Developer (ISD) and a Subject Matter Expert (SME). There are occasions where one person possesses both skill sets, but not often. Regardless, these two skill sets are the absolute minimum required to build a learning product. Additional people often required include: a graphics specialist, an evaluation specialist, a course director, an eLearning specialist, which may include special software skills (such as Adobe Captivate, Camtasia, or Learning Management System skills).
We begin our explanation of the JADDIE by entering the PMBOK portion of the model at the INITIATE phase. Like every step in the JADDIE model, this phase can be simple or complex, depending on the extent of the project. Do you need one class or a full course with many classes? As the effort is initiated that is one of the very first decisions to be made.
In the PLAN phase, we need to draft a broad statement of plan to determine general objectives and resources.
In the EXECUTE phase, we really get into the traditional meat of the JADDIE model and conduct a needs analysis, learning event design, development, implementation and evaluation plan. These are the traditional phases of ISD as executed in traditional manner so no further explanation is needed here.
The next major phase is the MONITOR & CONTROL phase where we complete the processes of the traditional ADDIE model. We provide reports to Stakeholders, Champions, and other relevant leaders.
The final phase is the CLOSE phase where we complete our documentation of the curriculum, harvest lessons learned and archive our products. Any content that must be off loaded to other storage and retrieval systems is appropriately stored for reuse.
Of course there is much more content in the JADDIE model that discussed here, however that is the subject of a different workbook. For our purposes in this workbook on student evaluation, we are only concerned with knowing we have a model to guide our efforts. Here is a simple numerical framework that you can easily follow.
Instructional Design Management Plan Structure
1.1 Develop charter/Initiate Plan
1.2 Identify stakeholders, champions, key personnel
2.1 Develop management plan
2.2 Develop work breakdown schedule
2.3 Develop instructional framework/formats
2.4 Determine Management Functions
2.5 Determine Support Functions
2.6 Determine Administration Functions
2.7 Determine Delivery Functions
3.1 Analyze Requirements
3.1.1 Determine instructional requirements
3.1.2 Conduct target audience analyses
3.1.3 Conduct resource analyses
3.2 DESIGN CURRICULUM
3.2.1 Design objectives, competencies, visual aids, student materials
3.2.2 Design assessment methods
3.2.3 Research and design new materials
3.2.4 Select instructional methods
3.3 DEVELOP COURSEWARE
3.3.1 Develop syllabus
3.3.2 Develop lesson plans and support materials
3.3.3 Develop student materials based on current build
3.3.4 Finalize assessments
3.3.5 Develop validation plan (first run course)
3.3.5 Finalize instructional materials
3.4 Implement Instruction
3.4.1 Perform final checks prior to instruction
3.4.2 Conduct instruction
3.5.1 Conduct Level 2 evaluations, Practical Exercises and competency based quizzes
3.5.2 Conduct Level 1 evaluations, immediately post class
3.5.3 Conduct Level 3, 4 and 5 evaluations as appropriate
4.0 MONITOR AND CONTROL
4.1 Ensure course success; provide reports to stakeholders, etc.
5.1 Harvest Lessons Learned
5.2 Terminate existing version of courseware when directed by stakeholders and champion base on validated evaluation results
5.3 Offload useful content to new version of course if warranted by evaluation