Formative Evaluation Checklist

One-to-one formative evaluation is usually performed during initial design and development of ICW. It is an informal evaluation method used to identify problems in lesson elements and make changes early when it's least expensive. This checklist should be used by a sampling of students, instructional designers and specialists, subject matter experts, and training managers.

The instructional adequacy component evaluates the extent to which ICW lessons provide the necessary kind of support for learning and meeting the instructional objectives. There are four areas of instructional adequacy that may be addressed: design, procedures, clarity, and efficiency. Use the table below to assess these areas.

Area

Topics

Is the course content logical and well organized?

Are the course objectives clearly defined?

Does the course content help to achieve the objectives?

Are examples meaningful and helpful?

Are questions presented clearly?

Is the feedback for incorrect answers supportive and corrective?

Does answering the questions help achieve the objectives?

Are important points emphasized?

Was the amount of information presented in each lesson comfortable?

Is the sequence of information within each lesson logical?

Are the lessons the right length?

Are lessons free of gender, race, religion, or ethnicity bias?

Do media elements fairly represent composition of career field in terms of gender and race?

Procedures

Do the number of interactions in each lesson make learning easier?

Are installation instructions easy to follow?

Are instructions on how to run the program easy to follow?

When a student is asked to respond, does the method used to respond make it easy to select the correct answer?

Do students enjoy the course and want to continue learning via computer?

Clarity

Is the text easy to understand?

Are course lessons attractive to look at?

Do graphics make lessons easier to learn?

Is sound and speech clear; not too soft or loud?

Efficiency

Are there sufficient numbers of examples and practice items included in the lessons?

Do the lessons help students learn the material?

This component refers to what the lesson actually looks like. Areas of concern include font types, color palettes, and graphic styles. In the past, cosmetic adequacy has been the primary focus of formative evaluations, taking precedence over instructional adequacy. Although important, cosmetic adequacy is not as important as ensuring that the content taught is correct and effective. In general, there are two areas assessed within the cosmetic adequacy component: appearance and clarity. Use the table below to assess these areas.

Area

Topics

Appearance/clarity

Is it easy to see how the information on the screen is organized?

Are graphics too simple or too detailed?

Is animation interesting without being distracting?

Is the text free of typographical errors?

Are colors and graphics attractive and not distracting?

Do colors and graphics aid in learning?

Are screens consistent and easy to use?

This component refers to the actual courseware itself. Topics that are addressed are: Is the software running as programmed? Are there any problems with the hardware or software? Does the lesson run like it was designed to run? The formative evaluation of this component may take the form of system testing and evaluation within three main areas of design, procedures, and proficiency. Use the table below to assess these areas.

Area

Topics

Design

Has the domain of appropriate responses been adequately anticipated?

Is information displayed accurately?

Are lesson components logically and systematically designed?

Procedures

Are procedures for evaluating student input and performance adequate?

Proficiency

Are lessons free from programming and conceptual loops?

Do lessons run efficiently (i.e., memory usage)?

This component addresses whether lessons are compatible with the styles of instructors and students, easily incorporated into existing curriculum activities and structures, and compatible with lesson activities and procedures already in place. There are two areas of curriculum adequacy: design and procedures. If curriculum adequacy is a concern during the design and development phases, the table below lists suggested topics that may be assessed during formative evaluations.

Area

Topics

Design

To the extent feasible and advisable, have instructor and user preference been included in the design?

Can lessons be used as the basis for additional and related lesson development?

Do lessons contain information likely to become quickly obsolete?

Do lessons contain options that require specific hardware or software considerations?

Procedures

Do lessons offer flexibility in how they can be used?