No Action on Validation In the 2020 National Defense Act Authorization

Well, I got my hopes up that the Department of Defense modeling and simulation community was finally going to be forced, kicking-and-screaming, to move forward; ensuring that their models were properly validated, not build upon a “base of sand” and making sure they are not assembled like some “house of cards.” This was to come about through four paragraphs in the Senate’s initial markup of the National Defense Authorization Defense Act (NDAA) of 2020 that instructed DOD to assemble a team “ assess the quality of these models and make recommendations…not later than December 31, 2020.”

The original four paragraphs are here:

U.S. Senate on Model Validation

Well, it looks like this is not going to happen !!!

According to a little research done by Shawn Woodford, it turns out that that modeling and simulation validation proposal in the original Senate Armed Services Committee report for the 2020 NDAA dated 11 June 2019 did not make it to the final Senate 2020 NDAA bill, passed on 2 July 2020. The proposal was also not included in the House version. The House and Senate versions are now being reconciled in committee and the final 2020 NDAA will probably be approved soon now that there is a general bi-partisan overall budget agreement. There will be a defense budget, but, it appears that it won’t address validation. There is a slim possibility this could change if it is added back in by committee at the last minute.

The 2020 NDAA SASC Report, 11 June 2019:

The 2020 NDAA S. 1790 SASC final markup, 2 July 2019:

The 2020 NDAA HR 2500 HASC final markup, 12 July 2019:

We would love to know who got those four paragraphs placed into the original Senate NDAA mark-up to start with; and afterwards, why they were then stripped out of the final bill. Clearly someone felt it was important enough to be put in there (as do we). We do not know who those “someone” are. And….who was it that stepped in from wherever and made sure those four paragraphs were removed?

If anyone knows anything further about this, please let us know.


Source for 1991 RAND “Base of Sand” paper:                                                                 .

The phrase “house of cards” we used in a report we did on casualty estimation methodologies (Casualty Estimation Methodologies Studies, 25 July 2005, The Dupuy Institute, page 32). To quote:

In 1991, Paul Davis and Donald Blumenthal employed the term “base of sand” to describe the essential modeling problem of the day. They described one of the core problems as a lack of a vigorous military science.

Unfortunately, this was the responsibility of the operations research community. Understanding military science was part of what ORO [Operations Research Office] was attempting to do in its early days. It was the operations research community who proposed the models, felt they could develop models, sold them to the military, spent the budget and finally produced the models built upon a “base of sand.” As such, they are the community that needs to correct the problem and produce this theory of combat. They are the scientists.

Yet, the problem is bigger than a “base of sand.” This phrase implies that there is a shortfall of data to start with. Yet every complex model (and most of these models are complex) is built from a number of interrelationships within the model. This is even more so for hierarchy of model structures. Each of these inter-relationships, which are often model unique constructs, is often built upon “expert judgment.” Therefore, the “base of sand” does not just start at the bottom, but carries through to each individual function within a model. As such, what has been built upon this “base of sand” is a “house of cards.”

Share this:
Christopher A. Lawrence
Christopher A. Lawrence

Christopher A. Lawrence is a professional historian and military analyst. He is the Executive Director and President of The Dupuy Institute, an organization dedicated to scholarly research and objective analysis of historical data related to armed conflict and the resolution of armed conflict. The Dupuy Institute provides independent, historically-based analyses of lessons learned from modern military experience.
Mr. Lawrence was the program manager for the Ardennes Campaign Simulation Data Base, the Kursk Data Base, the Modern Insurgency Spread Sheets and for a number of other smaller combat data bases. He has participated in casualty estimation studies (including estimates for Bosnia and Iraq) and studies of air campaign modeling, enemy prisoner of war capture rates, medium weight armor, urban warfare, situational awareness, counterinsurgency and other subjects for the U.S. Army, the Defense Department, the Joint Staff and the U.S. Air Force. He has also directed a number of studies related to the military impact of banning antipersonnel mines for the Joint Staff, Los Alamos National Laboratories and the Vietnam Veterans of American Foundation.
His published works include papers and monographs for the Congressional Office of Technology Assessment and the Vietnam Veterans of American Foundation, in addition to over 40 articles written for limited-distribution newsletters and over 60 analytical reports prepared for the Defense Department. He is the author of Kursk: The Battle of Prokhorovka (Aberdeen Books, Sheridan, CO., 2015), America’s Modern Wars: Understanding Iraq, Afghanistan and Vietnam (Casemate Publishers, Philadelphia & Oxford, 2015), War by Numbers: Understanding Conventional Combat (Potomac Books, Lincoln, NE., 2017) , The Battle of Prokhorovka (Stackpole Books, Guilford, CT., 2019), The Battle for Kyiv (Frontline Books, Yorkshire, UK, 2023), Aces at Kursk (Air World, Yorkshire, UK, 2024), Hunting Falcon: The Story of WWI German Ace Hans-Joachim Buddecke (Air World, Yorkshire, UK, 2024) and The Siege of Mariupol (Frontline Books, Yorkshire, UK, 2024).
Mr. Lawrence lives in northern Virginia, near Washington, D.C., with his wife and son.

Articles: 1516

One comment

  1. Chris, seems like a good opportunity for you to start a relationship with you district’s Representative and your state’s two Senators. Start with a request for the information that you are seeking and then expand to seeking to exert influence!

Leave a Reply

Your email address will not be published. Required fields are marked *