Strategic Leadership and Decision Making

APPENDIX

Tactics and Techniques for Evaluating Consensus Team Decision Making

This appendix is a compilation of tactics and techniques for evaluating consensus team decision making and contain the following annexes:

Annex 1: Questions for Conducting An After-Action Review

Annex 2: Team Process Evaluation

The annexes provide a set of tactics and techniques for evaluating the use of the Consensus Team Decision Making Model, and of the Tactics and Techniques in Decision Making. The annexes in this appendix are the "how-to" for implementing the evaluation of the consensus team decision-making model.

GENERAL

Many teams are accustomed to evaluate their performance systematically and regularly. Sometimes the results are measurable and dramatic: a patient dies during heart surgery and the post-mortem reveals that it was the result of a particular effect that was not the fault of the surgical team, though it may be something that could be noted for the future. The pitching coach of the Baltimore Orioles charts each pitch during the course of a game, not only counting the number of pitches, but whether it was a curve ball, a fast ball, and so on, and exactly what the situation was in the game at that time. And while there may be some differences in evaluating decision making teams in industry or in government, the fact that their work is important--it can affect policy, finances, national security, and so on--should lend some urgency to devising ways in which evaluation of the decision making process is measured in those environments, or that it is at least observed and recorded anecdotally.

We have noted in previous chapters the importance of evaluation to the decision making process. There are two kinds of evaluation described: one is a simple after-action review (annex 1) that could be used immediately after the conclusion of a team decision making meeting to get a general set of impressions and immediate feedback from a meeting; the other evaluation is one that is specifically designed to yield information on the leader of a team using some form of consensus decision making. Whatever instrument and timing you choose to use, the objective is to obtain information that could be used in organizing and conducting future meetings. Each of the evaluations will be discussed.

AFTER ACTION REVIEWS

A. TEAM EVALUATION. The most likely evaluation is apt to be an evaluation of the team by its own members, or in some cases by a process observer in concert with the team itself. Annex 2 of this appendix provides a set of observations that could be used to facilitate such an evaluation. Even though a set of leading questions is provided in annex 1, it is important to observe that there may be no singular format, nor list of questions for conducting an after-action review. That may be because no two teams, or decision making situations are the same. Thus, one of the tasks of the leader, or someone designated as a process observer, is to select questions based upon the circumstances, team composition, and observed decisionmaking process. Samples of leading questions that can be used in AAR's are itemized in the annex 1; this can be administered anonymously to the whole group at the conclusion of a session, and the inputs can be analyzed by the team leader and the team in preparation for subsequent sessions. This is a qualitative evaluation instrument, and is dependent on the feelings and perceptions of the participants. The accumulation of many comments that might suggest the presence of the same phenomenon by members of the team is a cause for adjusting the process model in some way. For example, if too little time is allowed at the end for review, you might adjust the "management element" of the model; or, if some people feel that they had little opportunity to participate in the discussion, some parts of the conceptual component might be adjusted for the next team meeting.

Former Army Chief of Staff Gordon Sullivan, describes an AAR as it was used to "create a structured way of facilitating learning from complex experiences that are often very ambiguous." ("Hope is Not a Method," Gordon R. and Michael V. Harper, What Business Leader's Can Learn from America's Army, Random House, New York, 1996). General Sullivan describes an AAR as a step in helping the Army to learn about what it was doing, and how to do it better. In his view, it is something that takes place after every significant event. Someone who has been with the team, sits down with them and they discuss what happened. It requires an awareness of what did happen, what should have happened, and what could have been done differently to achieve the objectives in a more efficient way. It takes time; it needs to be handled in a non-threatening way; and, it is hard. But the outcome yields a much improved sense of what happened. And, if this leads to improved performance, then the investment seems worthwhile. The context of the AAR, in General Sullivan's view, is organizational improvement through learning.

While annex 2 provides a means for a paper and pencil evaluation, it can be done using, for example, a GroupWare systems environment; both can be paired with an oral feedback session. If the leader is committed to team improvement through learning, there is no limit to the ways to gather data and have the team consider its performance.

B. LEADER EVALUATION. At times there may be a particular value in having the team, or at least some members of the team, evaluate the performance of the leader; annex 2 provides such a format. An aspect of this format is that it provides a quantitative dimension to other information gathered by using only annex 1. Annex 2 seeks to gauge the conduct and performance of the team leader more discretely than other commentary might allow. Such an evaluation of the leader might be done during times of training, or even in actual practice if someone is concerned about the progress of the team and wishes to have some more direct feedback to the leader about his or her performance.

There are three distinct considerations that should be kept in mind; each is put in the form of a question:

A leader has, or can have, profound effects on a team. More than any other member of the team, the leader (by definition, an input) can influence the decision making process. Process is a function of inputs. Therefore, any evaluation of the team leader is a "process input." The idea of evaluating the team leader's behaviors may be very threatening to many people, especially those who have matured inside very structured, hierarchical organizations. If anything can put leaders outside their comfort zones, it is being evaluated. However, what we're interested in evaluating is not the leader so much as the leader's behavior.

A team leader, in the scientific and academic sense, possesses a set of values, skills, attributes, prefer-ences,knowledge, and experiences. When facilitating the decision making process, the leader displays his set of values, skill, etc., through his behavior.

Because we can observe a leader's behavior, we can measure it. This is much easier to do in an objective manner than with an amorphous group of people who form a team. The level of complexity is much reduced, and observations can be focused.

One way of answering this question is to consider the framework offered by the model; that is, the tri-partite notions of High Conceptual Level, Prudent Consensus Appraoch, and Vigilant Decision Management. There are certain aspects of each of these areas that can be framed into specific questions. We want to measure the leader's performance, to increase everyone's sensitivity to the pillars, and to focus on certain areas so that they are adverted to during actual decision making situations. As a reminder they are:

-High Conceptual Level

Did the team leader adequately identify and define the problem(s)?

Did the team leader specify major tasks and products?

Did the team leader specify desired and undesirable outcomes?

Did the team leader specify priorities and adjust as necessary?

Did the team use all appropriate frames of reference?

Did the team leader specify a temporal perspective?

Did the team address short-term and long-term consequences?

Did the team examine and clarify assumptions?

Did the team war-game outcomes and consider all consequences?

Did the team use divergent and convergent thinking?

Did the team consider the moral implications of the decision?

-Prudent Consensus Approach

Did the team leader define roles and functions?

Did the team include all affected team members in the debate?

Did the team compensate for any absent or overworked member?

Did the team leader avoid being autocratic by sharing power?

Did the team leader micro-manage?

Did team members provide each other with full access to information?

Did the team keep communications channels open?

Did the team focus on mutual problems/issues, not personalities?

Did the team address winning issues first?

-Vigilant Decision Management

Did the team track real time information?

Did the team build multiple, simultaneous alternatives?

Did the team rely on expert advice?

Did the team leader try for consensus, but not delay the decision?

Did the team meet goals on schedule?

Did the team leader redirect tasks as required?

Did the team leader plan for and use a final review period?

Did the team leader schedule pauses for introspective looks at process and product?

In a particular scenario, and at the conclusion of a meeting, all participants will be asked to complete a peer rating feedback sheet ; a sample is provided in annex 2 on the performance of the leader. The leader will evaluate him/herself also. The evaluation sheet is focused on measuring how well the leader managed the group's decision making process. Someone will be designated to collect all of the leader evaluation sheets and collate the responses, and subsequently to share the results with the leader in a private feedback session.

SUMMARY

Evaluation can provide meaningful developmental feedback to the team as a whole and to the team leader as an individual. The goal of the feedback in consensus team decision making is to change both the team members', and the leader's, focus from output to process. If we focus feedback efforts on process, essentially creating an additional feedback path, we concern ourselves with output as a function of process. To a large extent, process is a function of inputs. Therefore, any evaluation of the team leader is indirectly but influentially concerned about process. Because we can observe a leader's and a team's behaviors, we can measure them.

Annex 1:

Questions for Conducting an After-Action Review

TEAM PERFORMANCE EVALUATION

SURVIVING IN A MISMANAGED GROUP

MANAGING CONFLICT

Annex 2

Team Process Evaluation

 

Definitely

Not Definitely

     
High Conceptual Level

Yes

Yes

Sure

No

No

Did the team leader

Adequately identify and define

the problem(s)?

1

2

3

4

5

Did the team leader specify major

tasks and products?

1

2

3

4

5

Did the team leader specify

desired and undesirable

outcomes?

1

2

3

4

5

Did the team leader specify

priorities and adjust

as necessary?

1

2

3

4

5

Did the team leader specify a

sequence of steps and solution/

process techniques?

1

2

3

4

5

Did the team use all

appropriate frames

of reference?

1

2

3

4

5

Did the team leader specify a

time horizon or perspective?

1

2

3

4

5

Did the team address short-term and long-term consequences?

1

2

3

4

5

Did the team examine and

clarify assumptions?

1

2

3

4

5

Did the team war-game

outcomes and consider all

consequences?

1

2

3

4

5

Did the team use divergent and

convergent thinking?

1

2

3

4

5

Did the team consider the

moral implications of the

decision?

1

2

3

4

5

Prudent Consensus Approach          
Did the team leader define roles

and functions?

1

2

3

4

5

Did the team include all affected

team members in the debate?

1

2

3

4

5

Did the team compensate for

any absent or overworked

member?

1

2

3

4

5

Did the team leader avoid

being autocratic by sharing

power and authority?

1

2

3

4

5

Did the leader avoid

Micro management?

1

2

3

4

5

Did team members provide

each other with full access to

information?

1

2

3

4

5

Did the team keep

communications

channels open?

1

2

3

4

5

Did the team focus on mutual

problems/issues, not

personalities?

1

2

3

4

5

Did the team address winning

issues first?

1

2

3

4

5

Vigilant Decision Management          
Did the team track real time

information?

1

2

3

4

5

Did the team build multiple,

simultaneous alternatives?

1

2

3

4

5

Did the team rely on expert

advice?

1

2

3

4

5

Did the team leader try for

consensus, but not delay the

decision?

1

2

3

4

  

5

Did the team meet goals on

schedule?

1

2

3

4

  

5

Did the team leader reorder

tasks as required?

1

2

3

4

5

Did the team leader plan for

and use a final review period?

1

2

  

3

  

4

  

5

Did the team leader schedule

pauses for introspective looks at

process and product?

1

2

3

4

5

 

| Return to Top | Return to Contents |