U.S. Senate on Model Validation

This link is to report 116-48, The National Defense Authorization Act for Fiscal Year 2020 by the Committee on Armed Services, United States Senate, dated June 11, 2019: https://www.congress.gov/116/crpt/srpt48/CRPT-116srpt48.pdf

While this 609 page report is probably worth reading from cover to cover, that I have not done. On the other hand, pages 253-254 are worth quoting (the bolding in the text is mine):

Evaluation of modeling and simulation used for force planning and theater operational requirements

The committee notes that the Department of Defense uses a large number and variety of computer models and simulations to support decision-making about force structure, resource allocation, war gaming, and priority weapons platforms and technologies to develop and deploy in support of likely operational scenarios. These models are used to develop information to brief decision makers, including the Congress, about, for example, the current state of the balance of forces in the Pacific and European theaters, the outcomes of likely war scenarios, and the need for investments in advanced technologies and new warfighting capabilities.

The committee is concerned that the quality, accuracy, and dependability of these models, given their important role in decision making processes, has not been adequately validated. The committee notes that technical and engineering models used to develop systems such as body armor and missiles are rigorously verified and validated for veracity of assumptions and technical accuracy using real world data. Other models, such as those used in the financial sector, are developed using expertise from a variety of disciplines, including economics, sociology, and advanced mathematics. The committee is concerned that the models used by organizations including the Joint Staff, Office of Net Assessment, war colleges, and service-level planning entities are simplistic by comparison and not subject to the same level of scrutiny.

Therefore, the committee directs the Secretary of Defense to establish an independent team of academic and industry modeling and subject matter experts to review the quality of modeling and simulation used for force planning, war gaming, resource allocation, and other senior leader decision-making associated with implementation of the National Defense Strategy. The team shall review the technical quality of models currently in use, including their ability to simulate as required by application; physics and engineering, socio-economic impact, readiness, global financial markets, politics, and other relevant inputs and outputs. The team shall assess the quality of these models and make recommendations for investments or policy changes needed to enhance and continuously validate current and future modeling and simulation tools employed to enable senior-level decision-making.

The committee directs the Secretary to support the team with expertise as needed from the Joint Staff, Office of Net Assessment, Under Secretary of Defense for Research and Engineering, and other relevant organizations. The Secretary shall ensure that the team has sufficient resources and access to all data and records necessary to perform its analysis. The committee directs the Secretary to deliver a report on the independent team’s assessments and recommendations with any additional comments, and a specific concurrence or non-concurrence for each recommendation, to the Committees on Armed Services of the Senate and House of Representatives no later than December 31, 2020.

A big thank you to the person who brought this to my attention.

Share this:
Christopher A. Lawrence
Christopher A. Lawrence

Christopher A. Lawrence is a professional historian and military analyst. He is the Executive Director and President of The Dupuy Institute, an organization dedicated to scholarly research and objective analysis of historical data related to armed conflict and the resolution of armed conflict. The Dupuy Institute provides independent, historically-based analyses of lessons learned from modern military experience.
Mr. Lawrence was the program manager for the Ardennes Campaign Simulation Data Base, the Kursk Data Base, the Modern Insurgency Spread Sheets and for a number of other smaller combat data bases. He has participated in casualty estimation studies (including estimates for Bosnia and Iraq) and studies of air campaign modeling, enemy prisoner of war capture rates, medium weight armor, urban warfare, situational awareness, counterinsurgency and other subjects for the U.S. Army, the Defense Department, the Joint Staff and the U.S. Air Force. He has also directed a number of studies related to the military impact of banning antipersonnel mines for the Joint Staff, Los Alamos National Laboratories and the Vietnam Veterans of American Foundation.
His published works include papers and monographs for the Congressional Office of Technology Assessment and the Vietnam Veterans of American Foundation, in addition to over 40 articles written for limited-distribution newsletters and over 60 analytical reports prepared for the Defense Department. He is the author of Kursk: The Battle of Prokhorovka (Aberdeen Books, Sheridan, CO., 2015), America’s Modern Wars: Understanding Iraq, Afghanistan and Vietnam (Casemate Publishers, Philadelphia & Oxford, 2015), War by Numbers: Understanding Conventional Combat (Potomac Books, Lincoln, NE., 2017) , The Battle of Prokhorovka (Stackpole Books, Guilford, CT., 2019), The Battle for Kyiv (Frontline Books, Yorkshire, UK, 2023), Aces at Kursk (Air World, Yorkshire, UK, 2024), Hunting Falcon: The Story of WWI German Ace Hans-Joachim Buddecke (Air World, Yorkshire, UK, 2024) and The Siege of Mariupol (Frontline Books, Yorkshire, UK, 2024).
Mr. Lawrence lives in northern Virginia, near Washington, D.C., with his wife and son.

Articles: 1516


  1. Is there some way to follow the progress of the group who are developing the report and perhaps have input as well as review draft reports?

Leave a Reply

Your email address will not be published. Required fields are marked *