Concept.  This course explores the connection between quantitative analysis and policy formulation.  The course will cover many types of analyses, not only the classic kinds of defense analysis – cost-effectiveness and combat models – but also budget, manpower, and cost analyses.  Although aimed at students going into national security positions, the analytical methods and approaches covered in the course are broadly applicable.

The purpose is not to make quantitative analysts of students but to make them intelligent consumers of analytical products.  Why bother with quantitative analysis?  Because in policy debates some numbers beat no numbers every time.  Without some appreciation of quantitative techniques, policy analysts are at a severe disadvantage.

The intention is practical and concrete, to aid in making better decisions in the real world.  The course aims to be directly useful in a student’s next job and in getting that job.

Structure.  The course uses case studies, historical examples, readings, and exercises to investigate how quantitative analysis is used in deciding major policy issues.  Teaching is through a combination of lecture and discussion.  Every class has a case, “caselette”, or exercise to give students a chance to apply to a concrete situation both their own experience and what they learned from the readings.

Sessions are grouped into four general areas: analysis and policy, analysis of conflict, acquisition of weapons, and resource allocation.

There are no prerequisites.  Some knowledge of military history and defense policy can be moderately helpful, but previous course experience shows little connection between such knowledge and the final grade.  No prior knowledge of quantitative analysis is needed nor are any special mathematical skills required.  However, a high level of discussion participation will be expected of every student.

Course themes:

• Good analysis can produce insights that inform decision making … and decision-makers want help.
• Analysis is accessible to everyone, not just those with quantitative analysis degrees.
• What a number means is not always obvious; it is important to understand what is inside.
• Assumptions about model inputs drive outputs.
• Multiple analyses, not a single number, illuminate an issue.
• Analysis does not produce certainty, but it can clarify choices.
• Policy analysis without quantitative analysis is sterile.

Contact information

(W) 202-775-3109; Cell: 703-915-5229

Office: Center for Strategic and International Studies, 1616 Rhode Island Ave, Washington DC (literally one block from SAIS Rome Building)

E-mail: mcancian@csis.org; mcancian@comcast.net. Send emails to both addresses, if you can, to be sure I see it and can respond promptly.

Office hours: A phone call or a Zoom session any time

                      Before class

                      At office by appointment (but easy to arrange)

Not applicable.

Readings.  No texts to buy, no packet of readings.  Readings are mostly on EReserves and total about 100 pages of fairly dense material each week.  They include many classics in the analytical literature and cover a wide range of sources – governmental organizations like GAO, think tanks like RAND and Institute for Defense Analyses, academic experts. Readings also include some dissenting literature that questions fundamental assumptions. All materials will be available online through EReserves or at the library.

Course prerequisites: None

Auditors:  Auditors will be allowed if class size is below 20, BUT auditors must be prepared every week so they can participate in the discussion. 

Electronic sources.  This course will make extensive use of Canvas and the EReserves system.

Professor’s notes.  These will be provided before each class to explain the purpose of the readings and how to approach that week’s discussion and exercise.

Grading

(1) Class participation (20% of final grade).  Participation is important and expected. 
(2) Two two-page memos (20% of final grade each, total 40%).  These are structured as a memo to a senior official analyzing a real-world problem.
(3) Final examination (40% of final grade), 3 hours, essays and problems covering the semester’s material.

Collaboration.  Collaboration is encouraged in preparing for class and in class exercises.  However, the papers and exam must be individual efforts.

A schedule for the papers and exam will be provided in the first class and posted on Canvas.

ANALYSIS AND POLICY

1. Introduction: Analysis and Leadership

Purpose:

• First 45 minutes of class: course description, requirements, overview; then,
• Using two case studies, discussion of the role of analysis in decision-making and the implications that has for the quality of decisions, the responsibilities of senior policy officials, and civil-military relations.

Elements:

• How, and how much, to relay on the judgments of line organizations
• The appropriate role of resources in decision-making
• The leader’s role: oversight v. advocacy, leadership v. management, referee v. player
• Information and analysis as levers of bureaucratic power

Readings:

• Alain Enthoven and K. Wayne Smith, How Much is Enough, 1971 (republished 2005), p.1-8, 31-47, 60-71, 113-116
• Peter T. Tarpgaard, “McNamara and the Rise of Analysis in Defense Planning,” Naval War College Review, autumn, 1995, p.67-87
• Charles A. Stevenson, SECDEF: The Nearly Impossible Job of Secretary of Defense, 2006, p.3-5, 181-188

Cases (distributed in class):  Determining Military Aid for Ukraine, Setting the size of the Army; 

Purpose: To discuss how analysis works, the process by which it is conducted, and the tradeoffs involved.

• The history of Operations Research/Systems Analysis
• How the process of analysis can affect the outcome
• The relationship between analysts and decisionmakers
• Keeping score – picking the right measures of effectiveness
• How model structure affects the result

Readings:

• E.S. Quade, Analysis for Public Decisions (Third Edition), p.13-97, 127-170 (not on ERes)
• Stockfish, Models, Data and War, GAO, Ch. 2 and Appendix I
• Wagner, Mylander, and Sanders, Naval Operations Analysis, 1997, p.1-24

http://www.mod.uk/NR/rdonlyres/65F3D7AC-4340-4119-93A2-20825848E50E/0/sdr1998_complete.pdf

3a. Analysis Case Study: Strategic Mobility

Purpose: To apply our discussion of the analysis process to a real world example.

Reading: Robert Ronstadt, The Art of Case Analysis, p.1-22
Joint Publication 4-01 The Defense Transportation System, III-1 – III-19

Case: “Congressionally Mandated Mobility Study (CMMS)” and “CMMS Update”

3b.  Communicating in the Bureaucracy/Writing Effective Memos

Purpose: To discuss what makes an effective memo, in preparation for the first paper assignment.

Readings: Perry M. Smith, Assignment Pentagon, (5th edition, 2020), [p.27-82]

               Hank Staley, Tongue and Quill, p.21-86 [Not on ERes]

               Professor Eliot Cohen, “Some Thoughts on Writing”

               Edward Tufte, The Visual Display of Quantitative Information, p.9-51

    Optional: The Tongue and Quill, Air Force handbook 33-337, http://static.e-publishing.af.mil/production/1/saf_cio_a6/publication/afh33-337/afh33-337.pdf

ANALYSIS OF CONFLICT

Purpose: To consider the assumptions built into campaign scenarios in models, and their effects on policy and budgets.

Elements:

• Assumptions about campaign structure and their implications
• Decisions about where, who, and how to fight
• Victory conditions and conflict termination
• Warning, deployment, and the outbreak of conflict
• Campaign phases
• Conventional campaigns and counterinsurgency campaigns
• Implications of these assumptions for budgets and force structure

Readings:

• Frostic and Bowie, “Conventional Campaign Analysis of Major Regional Conflicts”, in P. Davis, New Challenges for Defense Planning, RAND, 1994
• Davis, Hillestad and Crawford, “Capabilities for Major Regional Conflicts”, in Khalilzad and Ochmanek, Strategy and Defense Planning for the 21st Century
• Joint Operations, Joint Pub 3-0, 2018, “Termination” I-7, I-8; “Joint Operational Model/Phasing” V-7 – V-14
• Field Manual 3-24 Insurgencies and Counter Insurgencies (2014), p.4-13 – 4-16 and David Galula, Counterinsurgency Warfare – Theory and Practice, Praeger, 1963 p.107-135 (both contained in a single file on ERes)

Exercise: Klingner v. Adams: Alternative Approaches to a Conventional Conflict in Korea


5.  Combat Models: What are they and how do they work?

Purpose: to examine the structure, dynamics, and assumptions inside combat models - a non-technical discussion- how these affect outcomes, and how outcomes affect policy

Elements:

• Lanchester's equations made easy
• Firepower scoring
• Campaign combat models
• Rate of advance calculations
• Overview of JICM – a commonly used theater combat model

Readings:     

• Lanchester, FW, Aircraft in Warfare, ch. 5 and 6 (“The Principle of Concentration”)
• Lepingwell, J.W.R., “The Laws of Combat? Lanchester Reexamined”, International Security, Summer 1987, p. 89-135

• Michael O’Hanlon, The Science of War, Ch. 2 Modeling Combat and Sizing Forces”, 2009, p.63-85, skim 85-140
• Wm Krondak et al., Unit Combat Power, International Symposium on Military Operational Research, 2007, p.1-11

• Optional: George Akst, “Campaign Modeling”, Military Operations Research Society, March 2013; Trip Barber, “Joint Warfare Analysis”, Military Operations Research Society, March 2014

6. Wartime Operations: Attrition

Purpose: Using a simple mathematical formula, to gain insight into the dynamics of attrition and then to consider their policy implications

Elements:

• The mathematics of attrition
• Applying the mathematics to operational situations and understanding the implications of the result
• Attrition in ground and air combat
• Naval barrier operations
• Air defense intercepts
• Munitions accuracy (CEPs) 

Readings:

• D. Rowntree, Probability Without Tears, p. 1-74 [not on ERes]
• Army Ground Forces: The Procurement and Training of Ground Troops, p.181-225
• Williamson Murray Strategy for Defeat: The Luftwaffe 1933-1945 (also, revised version, Luftwaffe) ch. V (optional), VI, Appendix 2 and 3 (p. 209-263, 341-343) [ch. VI on ERes]
•  Steven Ambrose, Citizen Soldiers, ch. 11 “Replacements and Reinforcements” (p.273-289)
• Optional: Charles Wheelan, Naked Statistics, Ch.5, “Basic Probability”, p.68-89; Memphis Belle  (video)

Case: Recruiting for the Waffen S.S.


7. Force Structure: What do we have, and why do we have it?

Purpose: To discuss why nations maintain armed forces and the factors that shape their design

Elements:

• Review of force structure by service
• Issues in counting – what counts, what does not, and why it makes a difference
• Peacetime v. wartime demands on military forces
• Threats – nations, timelines, weapons
• Forward deployments and rotation bases
• “Tooth-to-tail” – how much combat power, how much support
• Allies and Host Nation Support (HNS)

Readings:

• Adam Talaber, The US Military’s Force Structure: A Primer, Congressional Budget Office, 2021, Summary, p.17-107
• Enthoven and Smith, How Much is Enough?, p. 132-142 in ch. 4 “NATO Strategy and Forces”
• R. Danzig, Driving in the Dark: Ten Propositions about Prediction and National Security, Center for a New American Security, 2011, esp. p.5-17 (on ERes and http://www.cnas.org/drivinginthedark)
• [one reading on the FY 2024 budget and defense strategy, pending publication, TBD]
• Optional: Mark Cancian, Force Structure on the National Defense Strategy, CSIS, https://www.csis.org/analysis/force-structure-national-defense-strategy-highly-capable-smaller-and-less-global
  

Case: How Many Ships Are in the Chinese Navy?

8a. Force Structure: Designing a Navy

Readings:

• Review Adam Talaber, The US Military’s Force Structure: A Primer, Congressional Budget Office, 2021, “Department of the Navy,” p.45-79
• CNO Navigation Plan 2022, ADM John Gilday, Chief of Naval Operations, July 2022, esp. p. 9-11, https://www.navy.mil/Press-Office/Press-Releases/display-pressreleases/Article/3105576/cno-releases-navigation-plan-2022/
• Optional: Defense Primer: Naval Forces, Congressional Research Service, April 2022, https://crsreports.congress.gov/product/pdf/IF/IF10484/22
  

WEAPONS ACQUISITION

8b. Weapons Acquisition: How can analysis help us decide what to buy?

Purpose: To discuss how quantitative (and non-quantitative) considerations go into generating requirements for new acquisition systems

Elements:

• Analysis v. politics in setting requirements
• Threat analysis
• Scenario development
• Analysis of alternatives
• Impact of performance specifications

8b. Case: “Trident: Setting the Requirements” (KSG case C15-88-802.0)

Readings:

• David Sorenson, The Process and Politics of Defense Acquisition, 2009. ch. 2 “The Current American Defense Acquisition Process” and ch. 4 “The Politics of American Weapons Acquisition” (p. 28-65, 87-125)
• DOD Instruction 5000.02T, Operation of the Defense Acquisition System (January 2022). Main text, Enclosure (8) “Affordability”, Enclosure (9) “Analyses of Alternatives,” Enclosure (10) “Cost Estimating and Reporting” http://www.dtic.mil/whs/directives/corres/pdf/500002p.pdf
• J. Ronald Fox, Defense Acquisition Reform 1960-2009, 2014, p. 5-33
• Info video of Trident launch on course web site

Purpose: To discuss the different ways to measure cost, the different ways of developing a cost estimate, and how different costing procedures produce different results.

Elements:

• What is cost?
• The purpose and process for developing cost estimates
• Why estimating costs on large military and civilian projects is so difficult
• Types of costs: fly away, unit procurement, total program, life cycle
• Types of estimates: parametric (top-down), work breakdown structure (bottom-up)
• Learning curves; average v. marginal costs; overhead and production rates

Readings:

• GAO, Cost Estimating and Assessment Guide, 2009, ch. 1,2,4 (p.5-25, 31-51)
• E.S. Quade, Analysis for Public Decisions, (Third Edition), p.108-123 (section on cost)

• Defense Management Systems College Teaching Notes
  • o “Introduction to Cost Analysis”
  • o “Cost Estimating Methodologies”
• “Application of Learning Curve Theory to Systems Acquisition
• GH Fisher, Cost Considerations in Systems Analysis, RAND, 1970, p.120-130

Cases: Selected Acquisition Report (SAR) [specific report TBD]

V-22 Costs – A Debate

Elements

• Defining and measuring cost growth
• Causes of cost growth
• Contract structure and its effect on cost
• Ethical dimensions – when does optimism become, in effect, a lie?

Readings:

• An Analysis of Weapons System Cost Growth, RAND, 1993, p.1-53 [Not on Eres, but at http://www.rand.org/pubs/monograph_reports/2006/MR291.pdf]
• David McNichol, Acquisition Policy, Cost Growth, and Cancellations of Major Defense Acquisition Programs, IDA, 2018, Executive Summary (p. 1-24)
• Norman R. Augustine, Augustine’s Laws, (1997), p.104-124
• Franklin C. Spinney, Defense Facts of Life: The Plans Reality Mismatch, Westview Press, 1985, p.125-168
• Martin Wachs, “Ethics and Advocacy in Forecasting Public Policy”, Business and Professional Ethics Journal, Vol. 9, No. 1 & 2, 1990, p.141-157
• Sissela Bok, Lying: Moral Choice in Public and Private Life, Pantheon, 1978, p.11 (bottom)-16

Case: The Big Dig, “Black Hole”, Government Executive, April 2001 w/ excerpts from Dept. of Transportation IG Reports, February 10, 2000, and November 29, 2000; (for background, see www.massdot.state.ma.us/highway/TheBigDig.aspx), Also, “10 years later, did the Big Dig deliver?”, Boston Globe, December 29, 2015

Purpose: To consider the elements of a cost-effectiveness analysis.

Elements: Defining effectiveness quantitatively (static v. dynamic comparisons), equal cost v. equal effectiveness analyses, assumptions imbedded in scenarios, the difficulties in forecasting capabilities, choices in picking Measures of Effectiveness

Readings: 

• Garrett and London, Fundamentals of Naval Operations Analysis, 1970, p.161-173, 182-185
• Stockfish, Models, Data and War, p.38-48
• E.S. Quade, A Critique of Cost Effectiveness, 1975, RAND (7 pages)
• David Novic, Concepts of Cost for Use in Studies of Effectiveness, 1957, RAND

Case: UK Trident Alternatives Review of 2013

RESOURCE ALLOCATION

Purpose: To consider the defense budget from a variety of financial and historical perspectives and, in doing so, uncover the policy judgments behind the numbers.

Elements:

• Top line budget measures (% of federal budget, % of GDP, current dollar, constant dollar, budget projections)
• Composition of the budget (by function, Major Defense Program, appropriation, service shares)
• Historical trends
• Defense budgets of other nations
• Problems of cross-national fiscal comparisons
• Sector impact

Readings:

• Allan W. Lerner and John Wanat, 1992, Public Administration, Ch. 6 “Public Budgeting”, p.120-144
• Gordon Adams and Cindy Williams, Buying National Security: How America Plans and Pays for Its Global Role and Safety at Home, 2010, ch. 5, “Planning, Programming, Budgeting, and Execution in the Department of Defense,” p. 93-119
• DoD FY 2024 Budget Request Overview, [publication expected Feb 2022]
• [An outside analysis of the FY 2024 if available in time]
• Optional: Defense Management: Primer for Senior Leaders, US Army War College, 2017

Case in class: National Defense Budget Estimates for the FY2024 Budget (DOD Comptroller budget backup data, called the “Green Book” because of its cover), Publication expected March 2023, (Assigned sections will be in one file on ERes, full document at TBD) [ch. 1 (overview, tables 1-2,1-4,1-7), 2-1 (total DOD only), 3-2, 5 (definitions, tables 5-1,5-3,5-4,5-10, 5-11,5-12), 6 (tables 6-1,3,5,16 (last page only),17 (last page only),18 (last page only) 6-25, 6-26), ch. 7 (tables 7-2,7-5). 

Purpose: To analyze the characteristics of the four kinds of manpower, their implications for cost and usage, and their resulting effects on national security policy.

Elements:

• Characteristics of the four kinds of manpower: active duty military, reserve military, government civilian, civilian contractor
• Strengths, weaknesses, and policy implications of each type
• Full costing – including retirement, recruiting, allowances, support
• Achieving a balance among the different types of personnel
• Defining “core” governmental activities

Readings (many individual readings but not large in total volume):

Active v. reserve military:

• Comprehensive Review of the Future of the Reserve Component, DoD, 2011, p.15-34
• Commission on the National Guard and Reserves, Final Report, January 31, 2008, p.5-11, 51-87
• Assessing the Army’s Active-Reserve Component Force Mix, Joshua Klimas et al., RAND, 2014

Military v. government civilian

• Replacing Military Personnel in Support Positions With Civilian Employees, Congressional Budget Office, December 2015, p.1-18 https://www.cbo.gov/sites/default/files/114th-congress-2015-2016/reports/51012-Military_Civilian_Mix_1.pdf

Government civilian v. contractors:

• Privatization and the Federal Government: An Introduction, Kevin Kosar, Congressional Research Service, 2006 https://fas.org/sgp/crs/misc/RL33777.pdf

Contractors v. military (a single .pdf file on EReserves):

• Department of Defense’s Use of Contractors to Support Military Operations: Background, Analysis, and Issues for Congress, Moshe Schwartz and Jennifer Church, Congressional Research Service, 2013, p.1-14, skim 15-23
• Optional: Logistics Support for Deployed Military Forces, CBO, 2005, p.ix-xvi (summary), p.1-26

Case: Forces for Homeland Security