Unit 2.  Writing Objectives and Test Items

Writing Objectives

You may use the topical statement format used by the Air Force Institute for Advanced Distributed Learning (AFIADL) or learning (behavioral) objective format. No matter which format you choose—learning objective or topical statement—each lesson must have at least one objective.

Learning objectives

If you and your organization decide to use learning objectives, you may want to read up on how to develop them in AF Manual 36-2236, Guidebook for Air Force Instructors.

If you think back to your training in ISD standards, a learning objective states a behavior that is discrete and observable. It is a precise statement of the skills, knowledge, or attitudes you expect the student to be able to demonstrate after instruction. Here are a couple of examples of simple learning objectives:

Topical statements

A topical statement (TS) serves as an objective and a lesson heading. Topical statements must appear at the beginning of a lesson. They also must be listed on the menu page, as they help to show the scope of the unit and allow students to quickly find the lessons they need to restudy after taking a course exam.

In essence, the verb is generally "implied" in TS format, so you don't have to be concerned with choosing the "proper" task verb (e.g., state, cite, specify, identify, describe) to show level of student response. Yet the wording of the proficiency code key should steer (but not dictate) the wording of the topical statement.

Write topical statements that relate directly to the STS. Be creative by varying your approach—ask questions, appeal to the student's imagination or curiosity, use dashes in a statement, etc. For example—

Lowercase proficiency codes

Lowercase proficiency codes relate to task knowledge. A topical statement for a lesson on task knowledge probably should begin with a gerund (an "ing" form of a verb used as a noun) or a "how to" statement:

But, "how to" statements need not always indicate steps. For instance, the lesson for the topical statement—How to select a drill bit—may cover only factors to consider, such as drill pitch, hardness, and so on.

Uppercase proficiency codes

Uppercase codes are used for subject knowledge. A topical statement for a lesson on subject knowledge likely will be stated as a title (noun) or as a question.

Bear 33 Ventilator (USAF Photo)

As we said, AFIADL uses the topical statement format. We capitalize the first letter of the first word, and lowercase other words unless they are proper nouns. We use no end punctuation unless the topical statement asks a question. Here are a few more examples of topical statements:

Marking proficiency codes

If you are developing a career development course (CDC), you must mark the proficiency codes of the lessons in your volumes. You can place the codes in any format that is convenient for you (paper chart, a floppy disk, or embedded in your screens). If you embed them in your text screens, make sure they are transparent to students and don't interfere with your authoring software. Indicate in a transmittal letter where AFIADL can find the codes (and how to access them, if embedded). By reviewing the proficiency codes, your instructional systems specialist (ISS) at AFIADL will help you ensure proper coverage of each lesson and test item.

Selecting ` and Writing Interactive Test Items

Objective

Test Item(s)

In this section we give you a primer on selecting (plus a little help on constructing) the type of test item to use for interaction. Interactive questions (IQ) give your students immediate feedback and self-evaluation on small amounts of information. IQs are embedded in your courseware. They involve students in the learning process, engaging them in concepts and ideas. IQs are not the same as questions used for testing, which are discussed later.

How do you decide where in a lesson to place these questions? You place them where they're needed! That may sound simplistic, but their placement depends on a number of factors. The main factor is testing to determine achievement of the objective(s). Of course, you'll want to sample behavior that leads to that achievement. Another factor is the amount of material being offered, its complexity, and the amount of interaction you desire for your students.

You should have at least two interactive questions per lesson. If you ask a series of questions, try not to have more than four in a row. Here's an example of a multiple-choice question.

 

Your counterparts writing CDCs and SCs to be delivered in hard copy develop what we call self-test questions. They're placed at the end of each section. You don't need a group of self-test questions at the end of a section since your interactive questions take the place of self-test questions.

You can use supply-type (short-answer) or selection-type (matching or multiple-choice) interactive questions. The type of test questions you construct depends on the learning outcomes to be measured. Multiple-choice items tend to provide the highest-quality items and can measure a variety of learning outcomes. However, sometimes a short-answer item is more appropriate, especially for numeric or alphanumeric answers. At other times, matching makes more sense, especially where the student is asked to click and drag elements to a graphic.

Short-answer items

A short-answer (fill-in-the-blank) question asks students to supply, rather than select, the answer. This type can be used when you want students to recall a fact, idea, or principle, or calculate a problem. It's difficult, though, to phrase a question to limit the number of possible answers (not to mention spelling variations). So, you may want to limit your use of short answer questions.

Having said that, there are times when short-answer items are appropriate. In those cases, use these construction tips:

To measure higher levels of learning

When you provide a situation that asks students to draw from everything they have learned thus far in a lesson, you help them apply their knowledge and show they understand the information as a whole. Another advantage of the problem/situation question is that you can measure learning in a variety of ways. You can change the variables to alter situations so that the students can make decisions about the same facts by evaluating different conditions. You can create problems to cover such areas as mathematics, troubleshooting, and planning. This kind of question is effective, efficient, and flexible. Use these guidelines to develop problem/situation items:

	Make the problem realistic, practical, and applicable to your career field.
	Be sure the information is accurate.
	Give enough data for students to find a solution.
	Do not introduce unnecessary data as distracters.

Problem/situation examples

Here is a representative math problem that might appear at the end of the lesson (or group of lessons), the objective of which is to convert decimal numbers to binary, octal, and hexadecimal notations.

In this case, you would probably want to give feedback for each column. If the student tries and fails twice to properly calculate the binary conversions, you would give the answers with an explanation of how to solve the problem (or link back to that discussion in the text). Then ask another similar, but different, question.

A mathematical problem, in particular, forces students to do operations accurately and in a definite sequence. The simplest form of mathematical problem provides a body of data and requires the direct application of one or more related formulas. For example, look at these two sets of questions:

	Make the situation realistic.
	Make it relevant to the principles or ideas developed in the text.
	Make it new. Don't repeat situations discussed in the text.
	Be clear and specific.
	Make sure the situation gives a good basis for the solution, but limit it so that it doesn't pose too many problems.
	Develop several problems for each situation.
	Make each problem independent of the solution to any other problem.

The amount of information in each situation is determined by the number of problems and the difficulty of the material you're teaching. Situation questions are based on actual or hypothetical circumstances or data. They can be simple or complex and can provide the basis for one or more problems, depending on the purpose you want the questions to serve.

Problem/situation questions have many advantages in distance learning. They approach real life more closely than other types of questions and help you put students in the job. Items that force students to use all the text and multimedia presented, as well as reasoning ability, make good troubleshooting exercises.

Matching items

Use matching questions to show comparison, identification, or definition of basic facts. Matching questions have two columns of related topics. Items in the columns can be textual or graphic. Students match items from one column that best relate to items listed in another column, either by typing in a selection or clicking and dragging. Apply these rules when you write matching items:

	Give simple and direct instructions.
	Use homogeneous item groupings.
	For text-only questions, put longer items in the left-hand column; shorter items on the right.
	Use parallel syntax. (Any item in one column should seem to go with any item in the other.)
	Use plausible distracters, but don't try to trick students.
	Be sure entire matching item fits on one screen.

True-false items

True-false items must be always true or always false. This makes them very difficult to construct, while still measuring what you want them to measure. Students have a 50-50 chance of guessing the correct answer; and when students correctly identify an item as false, you aren't sure the student knows the correct answer. For all of these reasons, we recommend you NOT use true-false items.

Having said that, we realize there are times when a switch, for instance, is either ON or OFF. To avoid the 50 percent guessing factor, include other parts of the entire procedure (not just switch operation) in the options of a multiple-choice item.

Multiple-choice items

Properly written, a multiple-choice item can teach terminology, facts, principles, and applications. It's especially effective as a means of requiring students to select the most important information from an array of data. Most of your interactive questions should be multiple-choice items.

A multiple-choice question consists of two parts: a stem and options. The options consist of one correct response and two or more distracters. (Try to avoid two-option questions.) These terms have specialized meanings in test development.

Stem

The stem is the statement of requirement. The stem either asks a question, directs students to do something (imperative), or begins a sentence that students must complete by selecting the right word or phrase (incomplete statement). Generally, it's best to use the question or imperative type of stem.

	Don't repeat key words from stem to options. A key word appearing in both the stem and an option can reveal the correct answer.
	Write options that are grammatically consistent with the stem.
	Avoid "all of the above" and "none of the above."
	Don't use such options as "both a and c" and "neither a nor c"
	Arrange numeric options in ascending or descending order.
	Be consistent with punctuation.

Correct response

Make one option clearly the correct response. Make sure the lesson supports the correct response.

Distracters

To an unknowledgeable person, each of the other options must look as if it could be the correct response, though it is not. In fact, the value of your multiple-choice items depends largely on your skill in writing convincing but incorrect distracters. Make distracters plausible.

Building Feedback for Interactive Test Items

Feedback is the computer's response to choices (responses) made by students. Feedback gives students a feel for how well they are progressing through a lesson, thereby increasing their confidence levels. It also reinforces the subject matter. The two important elements to consider are content and format.

For now, we're just touching on the subject of feedback for interactive test items. Future plans may involve other dimensions of interaction, like the quality and depth of the feedback and conditional branching.

You must send your itempool in a .doc file. The file name for the itempool is U0.DOC (that is U-ZERO.DOC). The mechanics of developing an itempool are presented here.

1. Create a new file.
2. Name the file U0.DOC. This file may have formulas created in Microsoft® Equation Editor.
3. On the first line, type the course number and volume, such as 2A177-01.
4. On the next line, type topicid and then the first topical statement number and a period. You may use uppercase or lowercase, but you must use zeros for numerals, not capital Os.
5. On the next line, type beginitem ans=a (b, c, or d) endid.
6. On the next line, type or copy from another file the item stem and the four options. Confirm that the answer cited is correct.
7. On the next line, type enditem.
8. Leave the next line blank.
9. Repeat appropriate steps above for the remaining test item bank questions.

You may type comments or instructions in between "enditem" and "beginitem" using brackets ([[[), as in the examples that follow. The beginning of the test item bank (U0.DOC) should look something like this:

2A177-01
topicid001.
beginitem ans=b endid
Which of these is not a benefit of stock funding depot-level reparables (DLR)?
a. Reductions in levels of required inventory.
b. Increased carcass return rates.
c. Decreased carcass return rates.
d. Increased material availability.
Enditem

[[[use next item on CE
beginitem ans=c import=i endid
Under the depot-level reparables (DLR) stock fund concept, what price is charged to your unit funds when a serviceable asset is issued from supply?
a. Standard price.
b. Carcass price.
c. Exchange price.
d. None; there is no charge.
enditem

topicid002.
beginitem ans=d endid
Under the depot-level reparables (DLRs) stock fund concept...etc.

Developing UREs and CE

After your ISS develops a URE, it is sent to you for coordination. You also receive an answer key and validation report. At that time, check each item carefully, especially the answer key. You may telephone your concurrence or changes to the ISS.

After all of your volumes have been reviewed, an ISS develops your CE, and we mail it to you for coordination. You receive two forms, two keys, and two validation reports. Check these documents very carefully. By catching errors at this stage, you save everyone valuable time and effort.

At present, AFIADL is printing course exams "on demand;" that is, when requested by the education offices. While not instantaneous, this process allows us to keep CEs up to date with changes or deletions.

Main Menu

Glossary

Webmaster

Continue