The nation that makes a great distinction between its scholars and its warriors will have its thinking done by cowards and its fighting done by fools.
--- Thucydides
Fantasy, abandoned by reason, produces impossible monsters; united with it, she is the mother of the arts and the origin of marvels.
--- Goya
When a task cannot be partitioned because of sequential constraints, the application of more effort has no effect on the schedule. The bearing of a child takes nine months, no matter how many women are assigned.
--- Frederick P. Brooks, Jr., The Mythical Man-Month
Some ideas are so stupid that only intellectuals could believe them.
--- George Orwell
Intelligence is like a four-wheel drive. It allows you to get stuck in more remote places.
--- Garrison Keillor
Do one thing every day that scares you.
--- Eleanor Roosevelt
My ability to keep cool in a crisis is based entirely on not knowing all the facts.
--- Garrison Keillor
The one common experience of all humanity is the challenge of problems.
--- R. Buckminster Fuller
The intuitive mind is a sacred gift and the rational mind is a faithful servant.
We have created a society that honors the servant and has forgotten the gift.
--- Albert Einstein
Integrity without knowledge is weak and useless, and knowledge without integrity is dangerous and dreadful.
--- Samuel Johnson
Knowledge is of two kinds. We know a subject ourselves, or we know where we can find information upon it.
--- Samuel Johnson
Iron rusts from disuse,
stagnant water loses its purity and in cold weather becomes frozen;
so does inaction sap the vigors of the mind.
--- Leonardo da Vinci
Thinking is the hardest work there is, which is the probable reason why so few engage in it.
--- Henry Ford
In the field of observation, chance favors the prepared mind.
--- Louis Pasteur
We shall not cease from exploration, and the end of all our exploring will be to arrive where we started and know the place for the first time.
--- T. S. Eliot
The other terror that scares us from self-trust is our consistency; a reverence for our past act or word, because the eyes of others have no other data for computing our orbit than our past acts, and we are loath to disappoint them.
...
A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines. With consistency a great soul has simply nothing to do. He may as well concern himself with his shadow on the wall. Speak what you think now in hard words, and to-morrow speak what to-morrow thinks in hard words again, though it contradict every thing you said to-day. — 'Ah, so you shall be sure to be misunderstood.' — Is it so bad, then, to be misunderstood? Pythagoras was misunderstood, and Socrates, and Jesus, and Luther, and Copernicus, and Galileo, and Newton, and every pure and wise spirit that ever took flesh. To be great is to be misunderstood.
--- Ralph Waldo Emerson, in his essay “Self-Reliance”
Overall/General Resources
Fit to Think: Conceptual, Critical, and Creative Thinking, briefing by Grant Hammond, Air War College
- Action Learning Guide (local copy), from Presidential Management Fellows (PMF), posted by OPM - check especially the appendices for items/tools such as inference ladder, four frames, conflict management, stakeholder mapping, force field analysis, and more
Mind Tools, a GOOD site for quick pass at a whole slew of stuff -- problem solving, analysis, memory improvement, creativity, stress management, time management, planning skills, comm skills, improved reading skills, & shareware tools
- How We Think, 1910, by John Dewey
- "Knowing" - self, enemy, situation - the Art of War 2000 (local copy) - a quick, easy read from Navy CIO with topics such as types of cognitive capabilities, and knowing yourself as an agent of change
- Learning Organization Doctrine: Roadmap for Transformation, US Army Corps of Engineers - definitions, models, outcomes, etc. (local copy, PDF)
Creativity and Innovation
If the world were perfect, it wouldn’t be.
--- Yogi Berra
Creativity is thinking up new things. Innovation is doing new things.
--- Theodore Levitt
Anxiety is the hand maiden of creativity.
--- T. S. Eliot
The foolish and the dead alone never change their opinions.
--- James Russel Lowell
He that will not apply new remedies must expect new evils; for time is the greatest innovator.
--- Francis Bacon
In times of change, learners inherit the earth, while the learned find themselves equipped to deal with a world that no longer exists.
--- Eric Hoffer
The world owes all of its onward impulses to men ill at ease. The happy man inevitably confines himself within ancient limits.
--- Nathaniel Hawthorne
I want to stay as close to the edge as I can without going over. Out on the edge you see all kinds of things you can't see from the center.
--- Kurt Vonnegut, Jr., writer
In the age of information sciences, the most valuable asset is knowledge, which is a creation of human imagination and creativity. We were among the last to comprehend this truth and we will be paying for this oversight for many years to come.
--- Mikhail Gorbachev, 1990
Innovation by definition will not be accepted at first. It takes repeated attempts, endless demonstrations, monotonous rehearsals before innovation can be accepted and internalized by an organization. This requires 'courageous patience'.
--- Warren Bennis
- See John Boyd and the OODA loop - especially the articles and briefings by Osinga
- See also intuition
- See also Creative Problem-Solving (CPS) process
- See also Innovation Adoption-Diffusion on Transformation of War page
- See also Innovation Adoption-Diffusion on Future Studies page
CreatingMinds.org - with principles, techniques, tools, etc.
Creativity Techniques - short descriptions of a whole passel of techniques
- Roots of Innovation (local copy, 2.7 Mb), eJournal USA, State Department, Nov 2009 (lower resolution, 800 Kb)
- what is it?
- which cultures foster it?
- the global geography of innovation
- how do complementary skills help?
- secrets of collaboration
- 2009 innovation index by country ranking
- Developing Creative and Critical Thinkers (local copy), by Allen and Gerras, in Military Review, Nov-Dec 2009
- Brain and Creativity Institute, University of Southern California
- The mission of the Brain and Creativity Institute is to gather new knowledge about the human emotions, decision-making, memory, and communication, from a neurological perspective, and to apply this knowledge to the solution of problems in the biomedical and sociocultural arenas.
- From Stone to Silicon: A Brief Survey of Innovation, by Husick, Foreign Policy Research Institute, Oct 2008 - top 25 innovations of all time
- Toward More Innovative Program Management (local copy), by Perino, in Defense Acquisition Review Journal, Feb-Mar 2005 - results of research into the science and psychology of innovation - using MBTI and FourSight assessment tools - includes formula for MBTI Creativity Index
- Stimulating Innovation (local copy), by Kostoff, Office of Naval Research (DOC file)
- Science and Technology Innovation (local copy), by Kostoff, Office of Naval Research - compares literature-based and workshop-based approaches for stimulating innovation (DOC file)
- Communication, Management Benchmark Study (local copy), Dept of Energy -- includes chapters on networking, alliances, organizational culture, and innovation
Innovation, especially organizational, short chapter, covers a lot
Creativity, how to cultivate, short chapter, covers a lot
Leadership: Creativity and Innovation, by William Klemm, from AU-24, GOOD broad coverage of ideas
Innovation and the Military Mind, by Air Vice-Marshal R. A. Mason, from AU-24
The Creative Leader, by Kendall, from AU-24
- Innovation: from Getting It to Getting It Done (local copy) - briefing by Kao, from OSD Transformation website
- Leadership: Strategies for Personal Success - Student Manual (local copy), FEMA
- Managing Multiple Roles for the Company Officer
- Creativity
- Enhancing Your Personal Power Base
- Ethics
- Leadership: Strategies for Personal Success - Instructor Guide (local copy), FEMA
Creativity Web, resources for creativity and innovation
- 10 Kick Starts to Your Creativity
- Creativity Basics
- The Serendipity Equations, by Figueiredo and Campos, posted by the Naval Research Lab
- Searching the Unsearchable: Inducing Serendipitous Insights, by Campos and Figueiredo, posted by the Naval Research Lab
- Beyond Productivity: Information Technology, Innovation, and Creativity, National Academies Press, 2003 - addresses issues such as "what makes people creative" and "how creative people work"
- The International Center for Studies in Creativity at Buffalo State College
- The TRIZ Journal for people interested in the TRIZ method of creativity and innovation
- Creativity in the Workplace, links to resources
- Creativity for Life, living creatively
- The Innovation Journal
- American Creativity Association
- European Association for Creativity and Innovation (EACI)
Emotional Intelligence
Emotional Intelligence - Implications for All U.S. Air Force Leaders, by Latour and Hosmer, in Air & Space Power Journal
- Gov Online Learning Center - with courses on ethics, management, risk assessment, decision-making, critical thinking, problem-solving, strategic planning, etc.
- Consortium for Research on Emotional Intelligence in Organizations
- "Leadership That Gets Results," by Daniel Goleman, in Harvard Business Review,
- describes six leadership styles: Coercive, Authoritative, Affiliative, Democratic, Pacesetting and Coaching
- Managing with Emotional Intelligence, at LeadershipAdvantage.com
- Emotional Intelligence As a Leadership Skill - course at the Federal Executive Institute
- From Army Management Staff College (AMSC), Dr Ursula Lohmann
- Emotional Competence and Leadership (local copy), USCG one-page summary
- Consortium for Research on Emotional Intelligence in Organizations
- EQ.org directory of internet Emotional Intelligence sites
- Emotional Intelligence at HayGroup.com - just an intro to the concept
- Emotional Intelligence Test - one example
- EQ Emotional Intelligence tests online - mixed quality, try several and glean tidbits from the narratives
Brainstorming
- See also Kipling for six places to start - what, why, when, how, where, and who
- Rules for Brainstorming (local copy), from the U.S. Army Corps of Engineers
- Brainstorming Guidelines, Naval Safety Center
- The Idea Center, with IdeaFisher software, commercial brainstorming tool, with modules for strategic planning, speeches/presentations, conflict resolution, and creative writing
Memory Skills
Concept Maps
Cognitive Skills
Cognitive Bias
- See also fallacies in logic
- Criminal Investigation Failures: Avoiding the Pitfalls (Part One) (local copy), by Rossmo, in Law Enforcement Bulletin, Sep 2006 - discusses various types of cognitive bias
- Cognitive Biases listed at Wikipedia
Psychology of Intelligence Analysis (local copy), by Heuer, 1999, for CIA
-- very good examination of many elements of critical thinking, with examples (PDF version)
- Check out Part III - Cognitive Biases
- Countering Terrorism: Integration of Practice and Theory (local copy), overview of conference at FBI Academy, Feb 2002
- from Appendix 8: Psychology of Bias
These investigators found that there is a general bias, based on both innate predispositions and experience, in animals and humans, to give greater weight to negative events or attributes. This is evident in four ways:
(a) negative potency (negative entities are stronger than the equivalent positive entities),
(b) steeper negative gradients (the negativity of negative events grows more rapidly with approach to them in space or time than does the positivity of positive events),
(c) negativity dominance (combinations of negative and positive entities yield evaluations that are more negative than the algebraic sum of individual subjective evaluations would predict), and
(d) negative differentiation (negative entities are more varied, yield more complex conceptual representations, and engage a wider response repertoire).
The authors review this taxonomy, with emphasis on negativity dominance, including literary, historical, religious, and cultural sources, as well as the psychological literatures on learning, attention, impression formation, contagion, moral judgment, development, and memory. They suggest that one feature of negative events that make them dominant is that negative entities are more “contagious” than positive entities.
Critical Thinking
Not everything that can be counted counts,
and not everything that counts can be counted.
--- Albert Einstein
If a man will begin with certainties, he shall end in doubts.
But if he will be content to begin with doubts he shall end in certainties.
--- Francis Bacon
Trying to get people to reason in a way that is not natural for them
is like trying to teach a pig to sing.
You don't accomplish anything and you annoy the pig.
--- E. Jeffrey Conklin & William Weil
- See argumentative and persuasive communication
- See media affecting decision making
- Developing Creative and Critical Thinkers (local copy), by Allen and Gerras, in Military Review, Nov-Dec 2009
- A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis (local copy), posted by CIA, March 2009
- This primer highlights structured analytic techniques—some widely used in the private sector and academia, some unique to the intelligence profession. It is not a comprehensive overview of how intelligence officers conduct analysis. Rather, the primer highlights how structured analytic techniques can help one challenge judgments, identify mental mindsets, stimulate creativity, and manage uncertainty. In short, incorporating regular use of techniques such as these can enable one to structure thinking for wrestling with difficult questions.
- Critical Thinking - tools to test/measure
- The Critical Thinking Rubric, Critical Thinking Project, Washington State University - assesses skill at each step of the problem solving process, including ability of individual to identify and consider influence by the context of the issue
- Seven dimensions to be assessed:
- Identifies and summarizes the problem/question at issue (and/or the source's position).
- Identifies and presents the STUDENT'S OWN hypothesis, perspective and position as it is important to the analysis of the issue.
- Identifies and considers OTHER salient perspectives and positions that are important to the analysis.
- Identifies and assesses the key assumptions.
- Identifies and assesses the quality of supporting data/evidence and provides additional data/evidence related to the issue.
- Identifies and considers the influence of the context on the issue. [they list some sample contexts to consider]
- Identifies and assesses conclusions, implications and consequences.
- Holistic Critical Thinking Scoring Rubric, by Facione and Facione
- California Critical Thinking Skills Test (CCTST)
- Test of Every Day Reasoning (TER)
- Thinking Critically about Critical Thinking: a Fundamental Guide for Strategic Leaders (local copy), by Gerras, U.S. Army War College, June 2006
Critical Thinking and Intelligence Analysis (local copy, 1.4 Mb low res), by Moore, National Defense Intelligence College (NDIC) occasional paper no. 14, March 2007 (local copy, 12.1 Mb high res) - includes generic and intel-specific discussion, as well as an appendix which is the NSA's Critical Thinking and Structured Analysis Class Syllabus
- The Cognitive Bases of Intelligence Analysis (local copy, 6 Mb file), by Thompson et al, U.S. Army Research Institute for the Behavioral and Social Sciences, Jan 1984
- Critical Thinking For The Military Professional, by Guillot, in Air & Space Power Chronicles
- CRITO (local copy), a step-by-step method for creating a critical analysis essay,
courtesy of Dr. David K. Johnson, Massachusetts College of Liberal Arts
- C - State a Conclusion or claim
- R - State Reasons or evidence meant to convince the reader
- I - Test the Inference, or argument
- T - Test the Truth of the R
- O - Construct the strongest imaginable Objections, and respond to them
- Risk Perception & Strategic Decision Making: General Insights, a New Framework, and Specific Application to Electricity Generation Using Nuclear Energy (local copy), by Brewer, Sandia National Labs, Nov 2005 - some good generic discussion included, such as ten critical thinking processes, with examples for each
- Active Learning Strategies to Promote Critical Thinking, by Walker, William Paterson University - a good summary of educational references regarding critical thinking
- Tools for Improving Your Critical Thinking
- Training Critical Thinking for Tactical Command (local copy), NATO report, Apr 2004
- The Miniature Guide to Critical Thinking: Concepts & Tools, by R. Paul and L. Elder, The Foundation for Critical Thinking, 2002
- Summary notes of above book
- A Rulebook for Arguments, by Anthony Weston
- Critical Thinking: What It Is and Why It Counts, by Facione, Santa Clara U. - 2010 update is available
- Strategies for Teaching Critical Thinking (local copy), an ERIC Digest
CriticalThinking.org - with 35 Dimensions of Critical Thought, GOOD series of short articles on affective and cognitive strategies, incl such items as listening critically
Evaluating Critical Thinking Skills (local copy), from U.S. Naval Academy's Center for Teaching and Learning -- criteria-based evaluation guide
Psychology of Intelligence Analysis (local copy), by Heuer, 1999, for CIA -- very good examination of many elements of critical thinking, with examples (PDF version)
CIA Compendium of Analytic Tradecraft Notes (local copy) - "CIA has made this edition [1995] of the compendium available to the public to help shed light on how the Directorate of Intelligence meets the daily challenges of providing timely, accurate, and rigorous analysis to intelligence consumers"
Manual of Job-Related Thinking Skills (local copy), Department of Homeland Security - including deductive reasoning, reasoning with sets, inductive reasoning about real-world events, and statistical reasoning - includes quizzes throughout
- How Critical Thinking Shapes the Military Decision Making Process (local copy), by Usry, Naval War College, 2004
- A lack of Combatant Commander (COCOM) critical thinking in the Military Decision Making Process (MDMP) is a causal factor in military failure at the operational level. However, critical thinking can improve the MDMP of the COCOM. This paper analyzes the effects of critical thinking on the combatant commander’s decision making process by: defining critical thinking; illustrating its impact on intuitive and analytical decisions; demonstrating barriers to critical thinking and proposing practical ways to use critical thinking in the MDMP.
The Role of Rhetorical Theory in Military Intelligence Analysis - A Soldier’s Guide to Rhetorical Theory (local copy), by Mills, AU Press
Critical Thinking on the Web, "a directory of quality online resources"
Developing Thinking Skills: Critical Thinking at the Army Management Staff College (AMSC) (local copy), by Eichhorn
- Assessing Curriculum via Critical Thinking (local copy, PDF, 300 kb), by Eichhorn, AMSC - includes various models, procedures, lists, etc.
(PPT, 1.1 Mb - with transition effects, etc.)
- U.S. Army Research Institute (ARI) products - see also ARI decision making products
- Conceptualizing Multicultural Perspective Taking Skills (local copy), by Rentsch et al, ARI report, Nov 2007
- Program Evaluation Metrics for U.S. Army Lifelong Learning Centers (LLCs): Extension to Military Operational Specialty(MOS)-Based LLCs (local copy), by Cianciolo, ARI report, May 2008
- Building Cultural Capability for Full-Spectrum Operations (local copy), by Abbe, ARI report, Jan 2008
- Cross-Cultural Competence in Army Leaders: A Conceptual and Empirical Foundation (local copy), by Abbe et al, ARI report, Oct 2007
- FOCUS: A Model of Sensemaking (local copy), by Sieck et al, ARI report, May 2007
- Performance in Non-Face-to-Face Collaborative Information Environments (local copy), by Schaab et al, ARI report, Jan 2007
- Leader Experience and the Identification of Challenges in a Stability and Support Operation (local copy), by Ben-Yoav Nobel et al, ARI report, July 2006
- Developing Adaptive Proficiency in Special Forces Officers (local copy) - "focuses particularly on the topics of mental adaptability, interpersonal adaptability, and leading an adaptable team"
- Training Critical Thinking Skills for Battle Command (local copy), Workshop Proceedings
- Training Critical Thinking for the Battlefield - Vol 1 - Basis in Cognitive Theory and Research (local copy)
- A Cognitive Framework for Battlefield Commanders' Situation Assessment (local copy) , Ft Leavenworth Field Unit
- Overview of Practical Thinking Instruction for Battle Command (local copy)
- Distance Learning: A Way of Life-Long Learning (local copy), by Belanich et al, Army Research Institute (ARI) Special Report 63, Sep 2005
- Practical Thinking - Innovation in Battle Command Instruction (local copy)
- Toward an Understanding of Team Performance and Team Cohesion Over Time Through the Lens of Time Series Analysis (local copy), by McIntyre et al, ARI Research Note 2003-07
- Tacit Knowledge and Practical Intelligence: Understanding the Lessons of Experience (local copy), by Hedlund et al, ARI Research Note 2003-04
- Tacit Knowledge in the Workplace (local copy - warning, 12 Mb file), by Sternberg et al, ARI report, Mar 1999
- Developing Effective Military Leaders: Facilitating the Acquisition of Experience-Based Tacit Knowledge (local copy), by Matthew et al, ARI report, Apr 2005
- The Influence of Trainee Gaming Experience and Computer Self-Efficacy on Learner Outcomes of Videogame-Based Learning Environments (local copy), by Orvis et al, ARI report, June 2005
- Task Difficulty and Prior Videogame Experience: Their Role in Performance and Motivation in Instructional Videogames (local copy), by Orvis et al, ARI report, June 2007
- Exploring the Interaction of Implicit and Explicit Processes to Facilitate Individual Skill Learning (local copy), by Sun and Mathews, ARI report, May 2005
- Utility of Game Instructions (local copy), by Chen, ARI report, Apr 2003
- Carnegie Hall: An Intelligent Tutor for Command-Reasoning Practice Based on Latent Semantic Analysis (local copy)
- An Overview of Automaticity and Implications for Training the Thinking Process (local copy)
- Descriptive Models of Military Decision Making (local copy), by Klein et al, ARI Research Note 90-93, 1990
- Advanced Team Decision Making: A Developmental Model (local copy), by Zsambok, Klein et al, ARI publication of research done for ICAF, Aug 1992
- Based on our observations of numerous tactical and strategic decision making teams, and on a review of relevant literature, Klein Associates derived three key components of advanced team decision making: team self identity, team conceptual level, and team self monitoring. The model contains ten key behaviors critical to team development in these components.
- ARI newsletters
- Training Small Unit Leader Adaptive Thinking Skills: Lessons Learned from Combat Training Center Exercises (local copy)
- Common Thinking and Judgment Problems Observed in Small Unit Leaders
- Difficulty reacting to novel situations – unable or slow to “think on your feet”
- Overly reactive. Difficulty in thinking like the enemy
- Focused on making quick decisions before fully analyzing the situation
- Inadequate monitoring of the situation or responding to cues in the environment
- Difficulty translating orders from higher echelons into actions
- Tunnel vision
- Overly focused on distracters; not on the big picture
- Difficulty shifting roles (e.g., combat to mediation within the same mission)
- Limited cultural awareness and necessary communication skills (specifically in dealing with
civilians)
- Critical Thinking as Dialogue: a New Approach to Training Critical Thinking (local copy), ARI Newsletter, October 2004 (whole issue (3 Mb))
- Critical Thinking Training for Army Schoolhouse and Distance Learning (local copy), ARI Newsletter, August 2003 - "High Pay-off Skills to deal with the uncertain" - includes diagram of Conceptual Model of Critical Thinking
- Training Adaptive Leaders (local copy), ARI Newsletter, May 2003 - "train a performance - a thinking performance" - using the Think Like a Commander vignettes to improve ability to identify critical information
- Tacit Knowledge for Military Leadership (local copy), ARI Newsletter, Spring 2001 - "Tacit knowledge scores, a better predictor of leadership effectiveness"
- Training Critical Thinking Skills for Battle Command: How to Think, Not What to Think (local copy), ARI Newsletter, Spring 2001 - "Improve critical thinking to improve battle command tactical performance"
- Training and Assessment of Decision-Making Skills in Virtual Environments (local copy), ARI Newsletter, Spring 2001 - "This research showed a linkage between decision-making and situation awareness"
- Critical Logistics Thinking Skills (local copy), by Dehrer, for U.S. Army Logistics Management College
- Center for Critical Thinking, aka Critical Thinking Community
- Library of articles on critical thinking, incl GOOD series of short articles ranging from Socratic teaching to assessing thinking skills
- Other resources include the following
- Valuable Intellectual Traits
- Intellectual Humility
- Intellectual Courage
- Intellectual Empathy
- Intellectual Integrity
|
- Intellectual Perseverance
- Faith in Reason
- Fairmindedness
|
- Universal Intellectual Standards "are standards which must be applied to thinking whenever one is interested in checking the quality of reasoning" - "following are the most significant"
- Clarity
- Accuracy
- Precision
- Relevance
|
- Depth
- Breadth
- Logic
|
- Dartmouth Writing Program support materials - including development of argument
- Fundamentals of Critical Reading and Effective Writing
- Critical Thinking Across the Curriculum Project, Longview Community College
- Mission: Critical, tutorial on critical thinking, San Jose State University
- Institute for Critical Thinking (ICT), Montclair State U.
- Graduate Program in Critical and Creative Thinking (CCT), U. of Mass. Boston
- Of the Liberty of Thought and Discussion, Chapter 2 from On Liberty, by John Stuart Mill
- Rudyard Kipling, from "The Elephant's Child" in Just So Stories
I keep six honest serving-men
(They taught me all I knew);
Their names are What and Why and When
And How and Where and Who.
[ed. - answering those six questions is often a good place to begin]
- Occam's Razor, published 1328 (also spelled Ockham)
- also known as the Law of Parsimony or Principle of Economy or Principle of Simplicity
- Principle from William of Occam (logician and Franciscan friar, c. 1285-1350 or 1280-1349) -- stating that given multiple theories to explain a set of observations the simplest explanation is to be preferred
- "Pluralitas non est ponenda sine neccesitate" or "non sunt multiplicanda entia praeter necessitatem" - "Entities should not be multiplied unnecessarily."
- "Never multiply explanations or make them more complicated than necessary. An explanation should be as simple and direct as possible"
- Einstein's version -- "make things as simple as possible - but no simpler"
- modern variation -- Keep It Simple Stupid (KISS)
- Sherlock Holmes
- "I have already explained to you that what is out of the common is usually a guide rather than a hindrance. In solving a problem of this sort, the grand thing is to be able to reason backward. That is a very useful accomplishment, and a very easy one, but people do not practise it much. In the everyday affairs of life it is more useful to reason forward, and so the other comes to be neglected. There are fifty who can reason synthetically for one who can reason analytically." [in A Study in Scarlet, 1887]
- "How often have I said to you that when you have eliminated the impossible, whatever remains, however improbable, must be the truth?" [in The Sign of Four, 1890]
- "That process," said I, "starts upon the supposition that when you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth. It may well be that several explanations remain, in which case one tries test after test until one or other of them has a convincing amount of support." [in The Case Book of Sherlock Holmes, 1927]
- "It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts." [in The Adventures of Sherlock Holmes, 1892, "A Scandal in Bohemia"]
- "I ought to know by this time that when a fact appears to be opposed to a long train of deductions, it invariably proves to be capable of bearing some other interpretation." [in A Study in Scarlet, 1887]
- Applying the Science of Learning: Using the Principles of Cognitive Psychology To Enhance Teaching and Learning (local copy), testimony before Congress by Halpern
- Promoting Critical Thinking In Professional Military Education, an Air University research paper
- Improving Leadership through Better Decision Making: Fostering Critical Thinking, an Air University research paper
- Appendix A discusses the Watson-Glaser Critical Thinking Appraisal - which is composed of a series of five tests
- inference test - "gauges ones’ ability to discern the degree of truth or falsity of inferences drawn from the data that you are given"
- recognition of assumptions test - samples the "ability to recognize unstated assumptions in statements"
- deduction test - "samples the ability to reason properly when given a statement of premise; to recognize the relation of implication between propositions"
- interpretation test - samples the "ability to weigh evidence and make valid generalizations"
- evaluation of arguments test - "samples the ability to distinguish strong and relevant arguments from ones which are weak or irrelevant"
- Critical Thinking I, lesson plan at SAMS, Leavenworth (local copy)
- Critical Thinking II, lesson plan at SAMS, Leavenworth (local copy)
- Collaborative Learning Enhances Critical Thinking, article by Gokhale
- Articles on Critical Thinking, annotated bibliography
- Logic Tutor, by Green - FREE online tutorial system on logic
- Think Critically about What You Find on the Web
- Write a Critical Book Review
Science and Reason
How Science Works (local copy), by Goodstein, in Reference Manual on Scientific Evidence, Second Edition, Federal Judicial Center, 2000 - compares Francis Bacon’s Scientific Method, Karl Popper’s Falsification Theory, Thomas Kuhn’s Paradigm Shifts, and more
- Steps of the scientific method (from CDC site)
- Name the problem or question
- Form an educated guess (hypothesis)
of the cause of the problem and make
predictions based upon the hypothesis
- Test your hypothesis by doing an
experiment or study (with proper controls)
- Check and interpret your results
- Report your results to the scientific
community
- Scientific Method Man, article in Wired, Sep 2004 - discussing "verifier approach" to problem solution, as used by Gordon Rugg
- With the verifier approach, Rugg begins by asking experts to draw a mental map of their field. From there, he stitches together many maps to form an atlas of the universe of knowledge on the subject. "You look for an area of overlap that doesn't contain much detail," he says. "If it turns out there's an adjoining area which everyone thinks is someone else's territory, then that's a potential gap."
- His approach is built on the observation, noted as far back as the 1970s, that experts tend to cut to the chase. In their zeal to get to an answer, they make many little mistakes. (A recent study of work published in Nature and British Medical Journal, for example, found that 11 percent of papers had serious statistical errors.) Experts unknowingly fudge logic to square data with their hypotheses. Or they develop blind spots after years of working in isolation. They lose their ability to take a broader view. If all this is true, he says, think of how much big science is based on flawed intuition.
The verifier method boils down to seven steps:
- 1) amass knowledge of a discipline through interviews and reading;
- 2) determine whether critical expertise has yet to be applied in the field;
- 3) look for bias and mistakenly held assumptions in the research;
- 4) analyze jargon to uncover differing definitions of key terms;
- 5) check for classic mistakes using human-error tools;
- 6) follow the errors as they ripple through underlying assumptions;
- 7) suggest new avenues for research that emerge from steps one through six.
- Additional resources on the Verifier Approach
Socratic Method & Asking Questions
Problem Solving
The one common experience of all humanity is the challenge of problems.
--- R. Buckminster Fuller
I am enthusiastic over humanity's extraordinary and sometimes very timely ingenuities. If you are in a shipwreck and all the boats are gone, a piano top buoyant enough to keep you afloat may come along and make a fortuitous life preserver. This is not to say, though, that the best way to design a life preserver is in the form of a piano top. I think we are clinging to a great many piano tops in accepting yesterday's fortuitous contrivings as constituting the only means for solving a given problem.
--- R. Buckminster Fuller
Intellectuals solve problems; geniuses prevent them.
--- Albert Einstein
We can't solve problems by using the same kind of thinking we used when we created them.
--- Albert Einstein
The biggest problem in the world could have been solved when it was small.
--- Witter Bynner
There is always an easy solution to every human problem—neat, plausible, and wrong.
--- H.L. Mencken
If a problem has no solution, it may not be a problem, but a fact - not to be solved, but to be coped with over time.
--- Shimon Peres
- See also Decision Making
- See also Critical Thinking
- School of Advanced Air and Space Studies (SAASS), Air University
- School of Advanced Military Studies (SAMS), U.S. Army
- School of Advanced Warfighting (SAW), Marine Corps University
- Joint Advanced Warfighting School (JAWS), National Defense University
- Naval Operational Planner Course (NOPC), Naval War College
- The Four Horsemen of the Problem Solving Apocalypse (local copy), by Coppola, in U.S. Army Medical Department Journal, Jul-Aug 1997
- Decision Making and Problem Solving (local copy), from Army ROTC
- Innovative Problem Solving in USAF Officer PME Curriculum, ACSC research paper
- Problem Solving -- related theories
- Decision Making and Problem Solving (local copy), self-study course from FEMA
- Osborn-Parnes Creative Problem-Solving (CPS) process
- Traditional Six-Step Problem Solving Process
- Identify and Select the Problem
- Analyze the Problem
- Generate Potential Solutions
- Select and Plan the Solution
- Implement the Solution
- Evaluate the Solution
- Influence of Anonymity in a Group Problem-Solving Environment, research paper, Air Force Institute of Technology (AFIT)
- Planning Primer, U.S. Army Corps of Engineers, 23 pages -- includes discussion of problem-solving process (local copy)
- Planning Manual, U.S. Army Corps of Engineers, 321 pages -- includes discussion of problem-solving process (local copy)
- Six-Step Problem Solving Process, as used by Army Corps of Engineers
- Identifying problems and opportunities
- Inventorying and forecasting conditions
- Formulating alternative plans
- Evaluating alternative plans
- Comparing alternative plans
- Selecting a plan
- Techniques for Effective Decision Making, by Mind Tools
- Problem Solving and Decision Making: Consideration of Individual Differences - Using the Myers-Briggs Type Indicator (MBTI), by Huitt
Root Cause Analysis
Analysis and Decision Making
When you get to the fork in the road, take it.
--- Yogi Berra
Short term thinking drives out long term strategy, every time.
--- Nobel Prize-winning economist Herbert Simon
- See Problem Solving above
- See Critical Thinking above
- See Intuition below
- See Ethical Decision Making
- See Media Affecting Decision Making
- See Game Theory below
- See Tactical Decision Games contrasting intuitive and analytic approaches
- See John Boyd and the OODA loop
- See combat leadership, including key historical decisions
- Decision Making -- related theories
- The Art of Design: a Design Methodology (local copy), by Banach, Military Review, Mar-Apr 2009
- Designing Decision Support Systems To Help Avoid Biases & Make Robust Decisions, With Examples From Army Planning, by Chandrasekaran, The Ohio State University, 1 Dec 08 - posted by DTIC
- Strategic Decision Games: Improving Strategic Intuition (local copy), by DeFoor, Joint Advanced Warfighting School, 23 Apr 2007
- Net Assessment: A Practical Guide, by Bracken, in Parameters, Spring 2006
- Net assessment is one of the principal frameworks for analyzing the national security strategy of the United States. It has been used by the Department of Defense for many years. Understanding net assessment—what it is and what it can do—is important for two reasons. First, it has general application to many military issues. Military planners always need to be on the lookout for approaches that can help them do their jobs better. Net assessment should be in that tool kit.
- Shaping the Next One Hundred Years: New Methods for Quantitative, Long-Term Policy Analysis, by Lempert, Popper, and Bankes; RAND report, 2003
- A sophisticated reader ought to view with great skepticism the prospect of answering questions about the long-term future. The checkered history of predicting the future — from the famous declarations that humans would never fly to the Limits to Growth study to claims about the “New Economy” — has dissuaded policymakers from considering the effects of their decisions more than a few months or years ahead. However, today’s choices will significantly influence the course of the twenty-first century. New analytic methods, enabled by modern computers, may transform our ability to reason systematically about the long term. This report reviews traditional methods of grappling with the morrow, from narratives to scenario analysis, which fail to address the multiplicity of plausible long-term futures. The authors demonstrate a quantitative approach to long-term policy analysis (LTPA). Robust decision methods enable decisionmakers to examine a vast range of plausible futures and design near-term, often adaptive, strategies to be robust across them. Reframing the question “What will the long-term future bring?” as “How can we choose actions today that will be consistent with our long-term interests?” these methods provide powerful analytic support to humans’ innate capacity for “what-if-ing.”
- Reforming Pentagon Strategic Decisionmaking (local copy), by Lamb and Lachow, INSS, Strategic Forum No. 221, July 2006
- Theory of Effectiveness Measurement (local copy), by Bullock, AFIT dissertation, Sep 2006
- Effectiveness measures provide decision makers feedback on the impact of deliberate actions and affect critical issues such as allocation of scarce resources, as well as whether to maintain or change existing strategy. Currently, however, there is no formal foundation for formulating effectiveness measures. This research presents a new framework for effectiveness measurement from both a theoretical and practical view. ... Finally, the pragmatic nature of the approach is illustrated by measuring the effectiveness of a notional, security force response strategy in a scenario involving a terrorist attack on a United States Air Force base.
- Cultural Differences
- Cultural Factors in Complex Decision Making, by Strohschneider, in Online Readings in Psychology and Culture, Center for Cross-Cultural Research, Western Washington University, Bellingham, Washington USA, 2002
- Decision Making in Individualistic and Collectivistic Cultures, by Güss, in Online Readings in Psychology and Culture, Center for Cross-Cultural Research, Western Washington University, Bellingham, Washington USA, 2002
- Risk Perception & Strategic Decision Making: General Insights, a New Framework, and Specific Application to Electricity Generation Using Nuclear Energy (local copy), by Brewer, Sandia National Labs, Nov 2005 - some good generic discussion included, such as ten critical thinking processes, with examples for each
- Command Decision-Making: Experience Counts (local copy), by Wolgast, Army War College paper, 2005
- Lee’s Mistake: Learning from the Decision to Order Pickett’s Charge (local copy), by Gompert and Kugler, Defense Horizons number 54, Aug 2006
- Custer in Cyberspace (local copy), by Gompert and Kugler, Defense Horizons number 51, Feb 2006
- One of the consequences of the network revolution and corresponding distribution of authority is that many more persons up and down the ranks will be making combat decisions than compared to the days of centralized command and control. Power is migrating from headquarters “to the edge.” Therefore, it is essential to foster battle-wisdom
not just for senior officers but also for the junior officers and noncommissioned officers leading units in the field.
- Gladwell, Malcolm. Blink: The Power of Thinking without Thinking. New York: Little, Brown, 2005
- Book review of Gladwell's Blink (local copy), by Norton, in Naval War College Review, Winter 2006
- Future Warfare and the Decline of Human Decisionmaking, by Adams, in Parameters, Winter 2001
- In short, the military systems (including weapons) now on the horizon will be too fast, too small, too numerous, and will create an environment too complex for humans to direct. Furthermore, the proliferation of information-based systems will produce a data overload that will make it difficult or impossible for humans to directly intervene in decisionmaking. This is not a consideration for the remote science-fiction future. Weapons and other military systems already under development will function at increasingly higher levels of complexity and responsibility--and increasingly without meaningful human intervention.
- The Recogition-Primed Decision Model: an Alternative to the MDMP for GWOT (local copy), by Bushey and Forsyth, in Field Artillery, Jan-Feb 2006
- The Recogition-Primed Decision Model (local copy), by Ross et al, in Military Review, Jul-Aug 2004
- "Klein, S. Wolf, Laura G. Militellio, and Carolyn E.Zsambok show that skilled decisionmakers usually generate a good COA [course of action] on their first try. J.G. Johnson and M. Raab replicated this finding, extending it to show that when skilled decisionmakers abandon their initial COA in favor of a later one, the subsequent COA’s quality is significantly lower than the first one."
- Cultural Barriers to Multinational C2 Decision Making (local copy), by Klein et al, for 2000 Command and Control Research and Technology Symposium
- Extending Naturalistic Decision Making to Complex Organizations: A Dynamic Model of Situated Cognition, by Shattuck and Miller, Naval Postgraduate School, 2006
- Naturalistic decision making (NDM) has become established as a methodological and theoretical
perspective. It describes how practitioners actually make decisions in complex domains. However, NDM
theories tend to focus on the human agents in the system. We extend the NDM perspective to include the
technological agents in complex systems and introduce the dynamic model of situated cognition.
- A Literature Review of Analytical and Naturalistic Decision Making (local copy), by Zsambok, Beach, and Klein, for Naval Command, Control and Ocean Surveillance Center, Dec 1992 - examines many decision models/strategies
- Experts, expertise, and experience (local copy), course module from NOAA
- What do experts do so well that others don't?
- Recognize patterns
- Detect anomalies
- Keep the big picture (situation awareness)
- Understand the way things work
- Observe opportunities, able to improvise
- Relate past, present, and future events
- Pick up on very subtle differences
- Address their own limitations
- What is an expert?
- Novice - Lives in the moment. Can’t recognize complex relationships. Produces limited options.
- Routine Expert - Great at everyday stuff, strong procedural knowledge. Runs into trouble when problems are ill-structured or novel.
- Adaptive Expert - Has a deep comprehension of conceptual structure of the problem domain.
- 25 hours of well done simulations can achieve the same effect as 2 years of experience (or much more)
- The Future of Simulation Technology for Law Enforcement: Diverse Experience with Realistic Simulated Humans (Local Copy), by Forsythe, in the FBI Law Enforcement Bulletin, Jan 2004
- Automation and Expertise: What is expertise and how can automation work against it? (local copy) briefing by Quoetone, at WDM III Workshop, NOAA (PDF)
- NASA's ASK Talks with Dr. Gary Klein - use of storytelling, even internally, to improve decision making and problem solving and development/use of expertise
- The Internet and Congressional Decisionmaking (local copy), a CRS report prepared for the Chairman, House Rules Committee, 19 Sep 2001
- Military Decisionmaking Process (MDMP)
- Implications of Modern Decision Science for Military Decision-Support Systems, by Davis, Kulick, and Egner, RAND, 2005
- Military Decision-Making Process (MDMP) (local copy), chapter 5 of FM 101-5 Staff Organization and Operations
- Is It Time to Abandon the Military Decisionmaking Process (local copy), by McLamb, in Military Review, Mar-Apr 2002
- US Army Decisionmaking: Past, Present, and Future (local copy), by Paparone, in Military Review, Jul-Aug 2001
- How Critical Thinking Shapes the Military Decision Making Process (local copy), by Usry, Naval War College, 2004
- A lack of Combatant Commander (COCOM) critical thinking in the Military Decision Making Process (MDMP) is a causal factor in military failure at the operational level. However, critical thinking can improve the MDMP of the COCOM. This paper analyzes the effects of critical thinking on the combatant commander’s decision making process by: defining critical thinking; illustrating its impact on intuitive and analytical decisions; demonstrating barriers to critical thinking and proposing practical ways to use critical thinking in the MDMP.
- Descriptive Models of Military Decision Making (local copy), by Klein et al, ARI Research Note 90-93, 1990
- Combined Forces Command-Afghanistan Management Decision Model (local copy), DoD IG, July 2005
- Considering the request for assistance and the proposed application of the MDM, the DoD IG Team created a “universal” model that can be used to assess the capability
and capacity of any organization founded on democratic principles and the “Rule of Law.” For the purposes of this model, the definition of “organization” includes the full spectrum of organizational structures—from a nation-state to any type of government-, military-, or business-related entity. Consequently, the DoD IG Team built the attached model and identified the high-level elements common to most organizations that should be considered to establish a viable, stable, and selfsustaining institution. This document, therefore, includes the Management Decision Model, instructions on how to use the model, and a comprehensive description of the various elements embedded in the model’s architecture.
- Between Discipline and Intuition: the Military Decision Making Process in the Army's Future Force (local copy), a 2004 SAMS paper by Vowell
- Groupthink
- Hive Mind and Groupthink: the Curse of the Perfect IPT (local copy), by Hewson, in Defense AT&L, Nov-Dec 2005
- Groupthink, Politics, and the Decision to Attempt the Son Tay Rescue, by Amidon, in Parameters, Summer 2005
- Effects of Groupthink on Tactical Decision-Making (local copy), a 2001 SAMS paper by Johnson
- Groupthink – The Dark Side of Teaming and How To Counteract It! (local copy), by Kaut, TACOM-ARDEC TQM Officer
- New Challenges, New Tools for Defense Decisionmaking
, RAND report, 2003
- Foundations for Reasoning in Cognition-Based Computational Representations of Human Decision Making (local copy), by Senglaub et al, Sandia National Laboratories, Nov 2001
- Decision Navigation: Coping with 21st-Century Challenges in Tactical Decisionmaking (local copy), by Gyllensporre, in Military Review, Sep-Oct 2003
- Tactical Decision Making (local copy), course by Marine Corps Institute
- Decision Making and Problem Solving (local copy), from Army ROTC
- Evaluation of Cross-Cultural Models for Psychological Operations: Test of a Decision Modeling Approach (local copy, 4 Mb), USAF Research Lab, June 2001
- Expeditionary Air Force Leaders: Cognitive Skills for the Naturalistic Battlespace
, by Thomas, Grable, and Stratton, in Air & Space Power Chronicles - including discussion of Recognition-Primed Decision-making (RPD)
- SPAWAR Systems Center (SSC), U.S. Navy
- Decision Support Systems for Coalition Operations: Final Report (local copy), Technical Report 1886, Aug 2002
- Decision-Making in a Dynamic Environment: The Effects of Experience and Information Uncertainty (local copy), Technical Report 1832, Aug 2000
- Cognitive Aspects of Decision-Making: Project Summary (local copy), Technical Report 1830, July 2000
- Cultural Variation in Situation Assessment: Influence of Source Credibility and Rank Status (local copy), Technical Report 1829, July 2000
- Tactical Decision-Making Under Uncertainty: Experiments I and II (local copy), Technical Report 1821, Apr 2000
- A Cognitive Model for Exposition of Human Deception and Counterdeception (local copy), Technical Report 1076, Oct 1987
- U. S. Army Research Institute for the Behavioral and Social Sciences - see also ARI critical thinking products
- Collaboration and Self Assessment: How to Combine 360 Assessments to Increase Self-Understanding (local copy), by Psotka et al, ARI Report, Mar 2007
- Overview of Practical Thinking Instruction for Battle Command (local copy)
- Guidelines for Leaders to Consider When Making Decisions (local copy)
- Making Decisions in Natural Environments (local copy, 5.5 Mb), by Klein, ARI Research, Feb 1997
- Investigations of Naturalistic Decision Making and the Recognition-Primed Decision Model (local copy), by Klein and Calderwood, ARI Research Note 96-43
- The Development of Knowledge Elicitation Methods for Capturing Military Expertise (local copy), by Klein, ARI Research Note 96-14
- Predicting Rapid Decision-Making Processes Required by the Dismounted Objective Force Warrior (local copy, .7 Mb)
- Emotional Synthetic Forces (local copy)
- The objective of this research was to make the decision-making process of complex agents less predictable and more realistic, by incorporating emotional factors that affect humans.
- Measures of Platoon Leader Situation Awareness in Virtual Decision-Making Exercises (local copy)
- Assessing decision-making skills in virtual environments (local copy, 2.42 Mb)
- Decision-Centered MOUT Training for Small Unit Leaders (local copy, 2.08 Mb)
- Evaluating an Approach to MOUT Decision Skills Training (local copy, 1.66 Mb)
- Training Future Force Leaders to Make Decisions Using Digital Information (local copy, 2.3 Mb), Dec 2003, ARI report
- "... digital information such as video sensors and detailed map overlays will replace probabilistic cues from the environment."
- "While the decisions themselves do not differ, the decision-making process and the information used are different with digital information than with traditional cues. Decision making with digital information is more analytical ...."
- "Spatial orientation will become a key issue and skill in the electronic battlefield."
- The Case for a Joint Military Decisionmaking Process (local copy), by Anderson and Slate, in Military Review, Sep-Oct 2003
- Decision Making and the Levels of War (local copy), by Killion, in Military Review, Nov-Dec 2000
- Developing the Capacity for Decisive Action (local copy), by Krawchuk, in Military Review, Nov-Dec 2000 - 2000 MacArthur Writing Award 1st Place
- Decision Making and Problem Solving (local copy), self-study course from FEMA
- Decision Making in a Crisis (local copy), self-study course from FEMA
- Applying Dialectic to Acquisition Strategy (local copy), by Peeler, in Acquisition Review Quarterly, Spring 1997 - includes discussion of
- Kant's Hierarchy of Reason
- Hegel's Dialectic
- Karl Popper
- Simultaneous Antitheses
- General Robert E. Lee
and Modern Decision Theory, by Gilster, in AU Review, Mar-Apr 1972, including discussion of battle of Chancellorsville, and brief discussion of
- Lanchester Equations
- Bayes’ Theorem
- Von Neumann-Morgenstern Utility Theorem
- NATO’s Decision-Making Procedure (local copy), CRS report
- NATO Decisionmaking: Au Revoir to the Consensus Rule? (local copy), by Michel, Strategic Forum 202, Aug 2003
- FOCUS: A Model of Sensemaking (local copy), by Sieck et al, ARI report, May 2007
- Sensemaking Symposium, Final Report (local copy), DODCCRP
- The Ecology of Uncertainty: Sources, Indicators, and Strategies for Informational Uncertainty (local copy), by Schunn et al, Naval Research Lab
- The Center for Decision Research, U. of Chicago - including downloadable articles/papers
- Decision Sciences Institute (DSI), multidisciplinary international association dedicated to advancing knowledge and improving instruction in all business and related disciplines
- DoD Information Analysis Centers (IACs)
- Effective decision-making processes for the Joint Force Commander (local copy), from the Air Land Sea Bulletin, at the Air Land Sea Application Center
- Are You a Good Decision Maker? (local copy), from SBA -- includes "Ten Steps to Wise Decision-Making" and "Common Decision-Making Mistakes"
- The Human Side of Decision Making (local copy), from FAA -- brief examination of the human traits that interfere with decision-making
- Ladder of Inference (local copy), from NIH - "a model that describes an individual's mental process of observing situations, drawing conclusions and taking action"
- Intuitive Policing - Emotional/Rational Decision Making in Law Enforcement (local copy), by Pinizzotto et al, in the FBI Law Enforcement Bulletin, February 2004
- Making Ethical Decisions - A Practical Model (local copy), by Schafer, in the FBI Law Enforcement Bulletin, May 2002
- Cultivating Intuitive Decisionmaking (local copy), by Krulak, in Marine Corps Gazette, May 1999, as posted on the USMC Commandant's Page
- War in the Pits: Marine-Futures Traders Wargame (local copy), NDU Strategic Forum 61, by West
- Marine generals and colonels vs futures traders in decisionmaking wargame
- "The traders' OODA loop, executed at much higher speed, is ISAA: Information, Sort by Priority, Act, Assess"
- Virtual Stress (local copy), in Marines Online, senior Marines vs futures traders in decision making wargame
- The Collapse of Decisionmaking and Organizational Structure on Storm King Mountain (local copy), by Putnam, USDA Forest Service, 1995
- Numerous studies show no matter how many factors are important, the human mind normally can handle only about seven factors (e.g., seven-digit telephone numbers).
- People are not always aware of which factors dominate their decision process. Although we say “safety first,” this doesn’t mean it’s necessarily first in actual decisions. Also, people are seldom aware of the few factors they actually are processing so tend to be overconfident in their decisionmaking ability.
- Studies also show that our linear thinking tends to underestimate hazards, particularly if the hazard is increasing at a logarithmic or exponential rate as can happen on the fireline. An example would be estimating rates of fire spread. A computer would give the better decision in a heartbeat. People would tend to underestimate the rate of spread and have difficulty deciding on an appropriate course of action. And so it is important to understand the limits of how we process information and common types of errors that can occur.
- Eliciting Knowledge from Experts: A Methodological Analysis, by Hoffman et al, NASA Human-Factors workshop paper
- Revisiting the Abilene Paradox - Is Management of Agreement Still an Issue?, by Deiss - brief discussion of the mismanagement of agreement, where nobody in a group wants to do an activity, but the group does it because each member thinks all the others want to do it
- Strategic Leadership and Decision Making (local copy), book from NDU
- Intuition: an Imperative of Command (local copy), by Rogers, in Military Review - examines relevance of intuition to decision making in the context of warfighting on the modern battlefield
- Decisionmaking Theory (local copy), Marine Corps Doctrine Publication 6
- "the intuitive approach is more appropriate for the vast majority of typical tactical or operational decisions-decisions made in the fluid, rapidly changing conditions of war when time and uncertainty are critical factors, and creativity is a desirable trait"
- Note 18. Intuitive decisionmaking more appropriate for the vast majority of tactical/operational decisions: A 1989 study by Gary A. Klein (based on 1985 observations) estimated that decision makers in a variety of disciplines use intuitive methods 87 percent of the time and analytical methods 13 percent of the time. Evidence now suggests that this
study was actually biased in favor of analysis. More recent studies estimate the breakdown at more nearly 95 percent intuitive to 5 percent analytical. G. A. Klein, "Recognition-Primed Decisions" in William B. Rouse (ed.), Advances in Man-Machine System Research (Greenwich, CT: Jai Press, 1989); G. L. Kaempf, S. Wolf, M. L. Thordsen, and G. Klein, Decision Making in the Aegis Combat Information Center (Fairborn, OH: Klein Associates, 1992); R. Pascual and S. Henderson, "Evidence of Naturalistic Decision Making in Command and Control" in C. Zsambok and G. Klein (eds.), Naturalistic Decision Making, forthcoming publication (Hillsdale, NJ: Lawrence Erlbaum Associates); Kathleen Louise Mosier, Decision Making in the Air Transport Flight Deck: Process and Product, unpublished
dissertation (Berkeley, CA: University of California, 1990).
- Decision Making Theory (local copy), Naval Doctrine Publication 6, Naval Command and Control
- "The intuitive approach is clearly more appropriate for the fluid, rapidly changing environment of combat, when time and uncertainty are critical factors."
- Strategic Decisionmaking in the Information Age, Army War College Strategic Leadership Workshop (local copy), incl DOCs and PPTs
- Naturalistic Decision Making (local copy), "an attempt to understand how humans actually make decisions in complex real-world settings," by Klein and Klinger, 4 pages, in Human Systems IAC Gateway, vol XI, nbr 3 -- article contrasts naturalistic with recognition-primed decision (RPD) model
- Strategies of Decision Making (local copy), by Klein, in Military Review, May 1989
- Executive Decision Making (local copy, including bite-size pieces for faster loading), handbook from the Naval War College
- Case Studies in Policy Making and Implementation (local copy, including bite-size pieces for faster loading) - handbook from the Naval War College
- Syllabus for National Security Decision Making, Naval War College
- A Framework for Military Decision Making under Risks (abstract) (full copy), by Schultz, School of Advanced Airpower Studies (SAAS) - includes discussion of Eisenhower's decision regarding Operation Market Garden - see especially Chapter 2, Military Decision Making and Prospect Theory -
- Risk-Based Decision Making (RBDM), U.S. Coast Guard Research & Development Center, Groton, CT
- Applying Risk-based Decision-making Methods/Tools to U.S. Navy Antiterrorism Capabilities (local copy), by Mitchell and Decker, at Homeland Security Symposium and Exposition, 27 May 2004
- Decision Making and the Availability Heuristic - a self test of how publicity affects your reasoning
- Problem Solving and Decision Making: Consideration of Individual Differences - Using the Myers-Briggs Type Indicator (MBTI), by Huitt
- Command Decisions, collection of essays on the key decisions by both sides in both theaters of WW II
- Some Commercial Tools
- Techniques for Effective Decision Making, at Mind Tools
- Decision-Making, by Kent, in AU Review, May-June 1971 - addressing the problems and use of analysis
- More On Decision-Making, by Blythe, in AU Review, Jan-Feb 1972 - follow-on and response to above article
- Organizations
- Decision Analysis Society
- European Association for Decision Making (EADM)
- Society for Judgment and Decision Making
- Society for Risk Analysis
- Center for the Decision Sciences, Columbia University
- Institutes and Schools
- Decision Consortium, U. of Mich.
- Josephson Institute of Ethics
- Journals
- Organizational Behavior and Human Decision Processes, journal
- Journal of Behavioral Decision Making
- Journal of Risk and Uncertainty
- Medical Decision Making
- Society for Medical Decision Making (SMDM)
- Medical Decision Making journal
- Clinical Decision Making, MIT
- Gaming and Decision Theory
- Coevolution: Studying the Dynamics of Competition, and Gauss: Scenario Translation (local copy), by McDonald, of SAIC, at Project Albert International Workshop 5, posted by USMC Warfighting Lab - includes out-thinking, out-doing, reflexive control, OODA loop, recursive anticipation, co-adaptation, adaptive mental model, and more
- Game Theory.net, "a resource for educators and students of game theory"
- Stability Modeling and Game-Theoretic Considerations, Los Alamos National Labs (local copy)
- Decision-Theory for Crisis Management, Final Report, ARPA (local copy)
- A Framework for Military Decision Making under Risks (abstract) (full copy), by Schultz, School of Advanced Airpower Studies (SAAS) -- see especially Chapter 2, Military Decision Making and Prospect Theory
- A Complex Adaptive System (CAS) Approach to Public Policy Decision Making, by Eoyang, Yellowthunder, and Ward
- Using Decision Procedures to Accelerate Domain-Specific Deductive Synthesis Systems (local copy), by Baalen and Roach, NASA
- Tools for Decision Analysis: Analysis of Risky Decisions, probabilistic modeling, by Arsham, U. of Baltimore
- Methods for Eliciting Strategic Knowledge (Tables) (local copy), prepared for NATO Defense Group
Assumption-Based Planning
- The Importance of "Wild Card" Scenarios (local copy), by Dewar, for National Intelligence Council NIC 2020 project
- Assumption-Based Planning: A Planning Tool for Very Uncertain Times, by Dewar, Builder, Hix, and Levin, a RAND Monograph Report,
- other RAND reports concerning Assumption-Based Planning
- Assumption-Based Planning: a Tool for Reducing Avoidable Surprises, by Dewar, a RAND report, printed by Cambridge University Press
Game Theory
Uncertainty
- Journal of Risk and Uncertainty
- The Ecology of Uncertainty: Sources, Indicators, and Strategies for Informational Uncertainty (local copy), by Schunn et al, Naval Research Lab
- Decisionmaking Theory (local copy), Marine Corps Doctrine Publication 6
- "the intuitive approach is more appropriate for the vast majority of typical tactical or operational decisions-decisions made in the fluid, rapidly changing conditions of war when time and uncertainty are critical factors, and creativity is a desirable trait"
- Certain Uncertainty: Inoculating for Surprise, by Beaumont, in Air University Review
- Improving CIA Analytic Performance: Strategic Warning (local copy), by Davis, occasional paper for CIA's Improving CIA Analytic Performance: Strategic Warning - includes discussion of "Substantive Uncertainty and Strategic Warning" and "Analytic Tradecraft for Managing Substantive Uncertainty" and "Averting Strategic Surprise through Alternative Analysis"
Complex Systems
- See Chaos & Complexity on Theory page
- Leadership and Systems Thinking (local copy), by Reed, in Defense AT& L, May-June 2006 [AT & L = Acquisition, Technology & Logistics]
- The Department of Defense is a large and complex social system with many interrelated parts. As with any system of this type, when changes are made to one part,
many others are affected in a cascading and often unpredictable manner. Thus, organizational decisions are fraught with second- and third-order effects that result in
unintended consequences. “Fire and forget” approaches are rarely sufficient and are sometimes downright harmful.
- Peter Senge submits, in The Fifth Discipline, that systems thinking provides just the type of discipline and toolset needed to encourage the seeing of “interrelationships rather than things, for seeing patterns of change rather than static ‘snapshots.’” Senge argues that this shift of mind is necessary to deal with the complexities of dynamic social systems.
- Complexity, Conflict Resolution, and How the Mind Works, by Jones and Hughes, in Conflict Resolution Quarterly, Summer 2003
- New England Complex Systems Institute (NECSI)
- MIT Center for Coordination Science (CCS)
- MIT Artificial Intelligence Lab
- Overcoming the 90% Syndrome: Iteration Management in Concurrent Development Projects, by Sterman
Deductive and Inductive Reasoning
- See Holmes' comments above
- Essays and Arguments: A Handbook on Writing Argumentative and Interpretative Essays, by Johnston, May 2000, in public domain
Manual of Job-Related Thinking Skills (local copy), Department of Homeland Security - including deductive reasoning, reasoning with sets, inductive reasoning about real-world events, and statistical reasoning - includes quizzes throughout
Statistics and Trace Evidence: The Tyranny of Numbers (local copy), by Houck, in Forensic Science Communications, FBI - discusses induction and deduction and their application in establishing evidence - see especially the section "How Do We Know All Ravens Are Black?"
- quick definitions at an NIH site listing desired job skills
- Deductive Reasoning - Able to apply general rules to specific problems to come up with logical answers, including deciding whether an answer makes sense.
- Inductive Reasoning - Able to combine separate pieces of information, or specific answers to problems, to form general rules or conclusions. This includes coming up with a logical explanation for why seemingly unrelated events occur together.
- Janusian Thinking and Acting (local copy), by Paparone and Crupi, in Military Review, Jan-Feb 2002
- The authors maintain that the current U.S. approach to military operations-strategic, operational, and tactical-is too linear for today's contemporary operating environment. They argue that future warfighters must move beyond linear thought and action to a realm of thinking and acting that recognizes and accepts paired yet opposite ideas and actions
- "Instead of ruling out alternative hypotheses, Janusian thinking calls on us to embrace contradictions as naturally occurring phenomena. When we create insights for thinking and acting from the Janusian framework, we achieve remarkable explanatory power over the nature of human information processing."
- The seats of reason? An imaging study of deductive and inductive reasoning, by Goel et al, Dept of Psychology, York U., North York, Ontario, CA -- abstract posted by National Library of Medicine
We carried out a neuroimaging study to test the neurophysiological predictions made by different cognitive models of reasoning. Ten normal volunteers performed deductive and inductive reasoning tasks while their regional cerebral blood flow pattern was recorded using [15O]H2O PET imaging. In the control condition subjects semantically comprehended sets of three sentences. In the deductive reasoning condition subjects determined whether the third sentence was entailed by the first two sentences. In the inductive reasoning condition subjects reported whether the third sentence was plausible given the first two sentences. The deduction condition resulted in activation of the left inferior frontal gyrus (Brodmann areas 45, 47). The induction condition resulted in activation of a large area comprised of the left medial frontal gyrus, the left cingulate gyrus, and the left superior frontal gyrus (Brodmann areas 8, 9, 24, 32). Induction was distinguished from deduction by the involvement of the medial aspect of the left superior frontal gyrus (Brodmann areas 8, 9). These results are consistent with cognitive models of reasoning that postulate different mechanisms for inductive and deductive reasoning and view deduction as a formal rule-based process.
PMID: 9175134 [PubMed - indexed for MEDLINE]
- Deductive Logic, by St. George Stock, posted by Project Gutenberg
Dialectical Reasoning
- The Dialectical Dimension of the Moral Military Decision Making - an Idealistic Approach, by Stadler, for JSCOPE 2000 Conference
- Dialectical Versus Empirical Thinking: Ten Key Elements of the Russian Understanding of Information Operations, by Thomas, CALL Publication 98-21
- The Links between Science, Philosophy, and Military Theory:
Understanding the Past, Implications for the Future , by Pellegrini, for SAAS
- The effect of cultural differences in fear of isolation on dialectical reasoning, by Kim and Markman, U. of TX
Intuition
The intuitive mind is a sacred gift and the rational mind is a faithful servant.
We have created a society that honors the servant and has forgotten the gift.
--- Albert Einstein
- See also creativity and innovation
- See tactical decision games on the Simuations page, contrasting intuitive and analytic approaches
- See also - Recognition-Primed Decision Model - references in the Decision-Making section above
- See also situation awareness
- Strategic Decision Games: Improving Strategic Intuition (local copy), by DeFoor, Joint Advanced Warfighting School, 23 Apr 2007
- Reforming Pentagon Strategic Decisionmaking (local copy), by Lamb and Lachow, INSS, Strategic Forum No. 221, July 2006
- Lee’s Mistake: Learning from the Decision to Order Pickett’s Charge (local copy), by Gompert and Kugler, Defense Horizons number 54, Aug 2006
- Custer in Cyberspace (local copy), by Gompert and Kugler, Defense Horizons number 51, Feb 2006
- When conditions are complex and dynamic, time is short, and critical information is available, the key to making good decisions is to blend intuition with reasoning—more specifically, reliable intuition with timely reasoning.
- From "The Personal Relevance of Great Campaigns" - by Bird, 22 Feb 2001, Command and General Staff Officer's Course
- Carl Von Clausewitz explains that events in warfare are surrounded by uncertainty, and that there are few universal truths. Because of this, leaders must sort through this “fog” to find the truth, often a daunting endeavor that’s permeated by chance. The commander must sift through this information and decide what pieces are relevant and require action. Clausewitz specifically refers to the capability of the mind to discriminate information allowing quick, correct decisions. He describes this ability as coup d’oeil, “the quick recognition of a truth that the mind would ordinarily miss or would perceive only after long study and reflection.”2 Napoleon faced such a scenario in the battles of Jena-Auerstadt. With limited information, he turned an entire field army in place to seek decisive battle with the Prussians and won the day. [2Carl Von Clausewitz, On War, ed. and trans. by Michael Howard and Peter Paret, (Princeton, NJ: Princeton University Press, 1976), p. 100-102. ]
- Coup D'Oeil: Strategic Intuition in Army Planning (local copy), by Duggan, Strategic Studies Institute (SSI), Nov 2005
- Command Decision-Making: Experience Counts (local copy), by Wolgast, Army War College paper, 2005
- Intuition: an Imperative of Command (local copy), by Rogers, in Military Review - examines relevance of intuition to decision making in the context of warfighting on the modern battlefield
- Tactical Intuition (local copy), by Reinwald, in Military Review, Sep-Oct 2000
- Intuition: An Instantaneous Backup System?, by Mrazek, in Air University Review
- Decisionmaking Theory (local copy), Marine Corps Doctrine Publication 6
- "the intuitive approach is more appropriate for the vast majority of typical tactical or operational decisions-decisions made in the fluid, rapidly changing conditions of war when time and uncertainty are critical factors, and creativity is a desirable trait"
- Note 18. Intuitive decisionmaking more appropriate for the vast majority of tactical/operational decisions: A 1989 study by Gary A. Klein (based on 1985 observations) estimated that decision makers in a variety of disciplines use intuitive methods 87 percent of the time and analytical methods 13 percent of the time. Evidence now suggests that this
study was actually biased in favor of analysis. More recent studies estimate the breakdown at more nearly 95 percent intuitive to 5 percent analytical. G. A. Klein, "Recognition-Primed Decisions" in William B. Rouse (ed.), Advances in Man-Machine System Research (Greenwich, CT: Jai Press, 1989); G. L. Kaempf, S. Wolf, M. L. Thordsen, and G. Klein, Decision Making in the Aegis Combat Information Center (Fairborn, OH: Klein Associates, 1992); R. Pascual and S. Henderson, "Evidence of Naturalistic Decision Making in Command and Control" in C. Zsambok and G. Klein (eds.), Naturalistic Decision Making, forthcoming publication (Hillsdale, NJ: Lawrence Erlbaum Associates); Kathleen Louise Mosier, Decision Making in the Air Transport Flight Deck: Process and Product, unpublished
dissertation (Berkeley, CA: University of California, 1990).
- Decision Making Theory (local copy), Naval Doctrine Publication 6, Naval Command and Control
- "The intuitive approach is clearly more appropriate for the fluid, rapidly changing environment of combat, when time and uncertainty are critical factors."
- The Warning Process and the Role of Intuition (local copy), course module from NOAA
- Intuitive Policing - Emotional/Rational Decision Making in Law Enforcement (local copy), by Pinizzotto et al, in the FBI Law Enforcement Bulletin, February 2004
- Cultivating Intuitive Decisionmaking (local copy), by Krulak, in Marine Corps Gazette, May 1999, as posted on the USMC Commandant's Page
- War in the Pits: Marine-Futures Traders Wargame (local copy), NDU Strategic Forum 61, by West
- Marine generals and colonels vs futures traders in decisionmaking wargame
- "The traders' OODA loop, executed at much higher speed, is ISAA: Information, Sort by Priority, Act, Assess"
- Virtual Stress (local copy), in Marines Online, senior Marines vs futures traders in decision making wargame
- Intuitive people worse at detecting lies, by Young, NewScientist.com, 18 Mar 2002
- People who think of themselves as being intuitive make worse lie detectors than those who do not trust in a "gut instinct", according to new research.
- One possible explanation is that intuitives in fact rely on common misconceptions about how to spot a liar, he says.
Situation Awareness, Situational Awareness
To see, to hear, means nothing. To recognize (or not to recognize) means everything.
--- Andre Breton
- See also Intuition above
- See also Predictive Battlespace Awareness (PBA)
- Individual Situation Awareness (local copy), course module from NOAA
- Situation Awareness Definition (from Mica Endsley, 1988)
- Level 1 - Perception of the elements in the
environment within a volume of space
- Level 2 - Comprehension of their meaning
- Level 3 - Projection of their status in the near future
- Team Situation Awareness (local copy), course module from NOAA
- Situation Awareness Demons: the Enemies of
Situation Awareness (local copy), course module from NOAA
- Discusses the eight demons laid out by Endsley et al, in "Designing for Situation Awareness"
- Attentional Tunneling
- Requisite Memory Trap
- Workload, Anxiety, Fatigue, and Other Stressors (WAFOS)
- Data Overload
- Misplaced Salience
- Complexity Creep
- Errant Mental Models
- Out-of-the-Loop Syndrome
Ye Olde Brain, and Its Workings
The destiny of every human being is decided by what goes on inside his skull when confronted with what goes on outside his skull.
--- Dr. Eric Berne
- Department of Brain and Cognitive Sciences, MIT
- MIT CogNet: the Brain Sciences Connection
Left-Brain/Right-Brain (local copy), from Army ROTC "Foundations for Success"
Psychology of Intelligence Analysis (local copy), by Heuer, 1999, for CIA -- very good examination of many elements of critical thinking and mental processes, with examples (PDF version)
- Complexity, Conflict Resolution, and How the Mind Works, by Jones and Hughes, in Conflict Resolution Quarterly, Summer 2003
- Semiotic Fundamentals of Information Processing in Human Brain (local copy), by Perlovsky, Air Force Research Lab
- The paper discusses a mathematical nature of signs and symbols, and relates it to information processing and understanding, structure of the mind and brain, learning, and pattern recognition.
- Quantum Theory and the Role of Mind in Nature (local copy), by Stapp, Lawrence Berkeley Laboratory
- On Quantum Theories of the Mind (local copy), by Stapp, Lawrence Berkeley Laboratory
- A Quantum Theory of the Mind-Brain Interface (local copy), by Stapp, Lawrence Berkeley Laboratory
- The Evolution of Consciousness (local copy), by Stapp, Lawrence Berkeley Laboratory
- Values and the Quantum Conception of Man (local copy), by Stapp, Lawrence Berkeley Laboratory
- additional work by Stapp
Miscellaneous
[return to top]