Category Modeling, Simulation & Wargaming

Validation of Wargaming Simulation Models – Confusion !!

[Clinton Reilly has been a regular commentator on this blog. We present here a guest blog post from him]

Originally, I was heartened to read in your blog that the U.S. Congress is setting up a committee to oversee the validation of wargaming models, which were seen to be of doubtful validity. Validation is obviously a ‘good thing’ as it enhances models and so they produce more useful and reliable results. Your blog has put forward several articles to this effect.

I hastened to communicate this to a senior member of the Australian Defence Science and Technology (DST) group expecting an enthusiastic response, in anticipation of the Australian government following suit. However, much to my surprise, the said member addressed the MORS Modelling and Simulation Community of Practice (CoP) saying that this was a matter of some concern as the testing and validation may not be tailored to the objectives of the individual models. Members were asked to comment on this alarming prospect. There was no comment.

While this is a possibility, it seems to me that in such a rational field the logical argument that tests must be tailored to objectives would prevail. It seems hardly worth saying!

So, I replied with a more heartening email to CoP members that validation was only to be seen as a boon and should be welcomed and encouraged. Wargames would be improved. I also emailed the MORS Wargaming CoP with a similar message, again asking for comment on the posts in the MORS Modelling and Simulation CoP.

Now this is where the confusion sets in. In the weeks since the emails were sent no one has replied. There has been no direct comment to me either. Why I ask? In a military community where modelling and wargaming is a central activity – why has no one replied on either the validity of current models or the need for greater validation?

I am submitting this to your blog in the hope that someone in the worldwide wargaming community has an opinion. Is there a problem with extensive validation of existing models? Is a program of validation needed to improve a low level of validity?  Does anyone care about the standard of current models and their outputs?

Does anyone reading this blog have a comment?

Clinton Reilly

Managing Director

Computer Strategies Pty Ltd

Sydney, Australia

Status of Defense Act

A month ago, I flagged pages 253-254 of the report 116-48, supporting the National Defense Authorization Act for Fiscal Year 2020. This report is here: https://www.congress.gov/116/crpt/srpt48/CRPT-116srpt48.pdf

This kicker was the statement that “The committee is concerned that…these models…has not been adequately validated….using real world data….[and] are simplistic by comparison…” The entire four paragraphs are quoted in this blog post:

U.S. Senate on Model Validation

The current text of the actual Defense Act, dated 6/27/19 is here: https://www.congress.gov/116/bills/s1790/BILLS-116s1790es.pdf

Now, I don’t know how these two 609- and 1726-page documents connect, but I gather the requirements still exist to have a team “..to assess the quality of these models and make recommendations…not later than December 31, 2020.”

Does anyone know anything further about this effort?

Today – Speaking at Historicon in Lancaster, PA., Friday 12 July

I will be speaking at Historicon in Lancaster, PA., Friday 12 July, at 6 PM. Historicon is one of the three major annual wargaming conventions run by the Historical Miniatures Gaming Society (HMGS). It will be run from 10 July-14 July, 2019. Their website is here: https://www.hmgs.org/general/custom.asp?page=HconHome

As part of this large convention, they have organized a “War College.” This is an impressive effort that includes 18 lectures on Thursday, Friday and Saturday. I have the last lecture on Friday, from 6 – 7 PM. The speakers for this series include published authors Paul Westermeyer, Pete Panzeri, Steve R. Waddell and John Prados, among others. Lecture descriptions are here:                                                                               . https://cdn.ymaws.com/www.hmgs.org/resource/resmgr/historicon/hcon_19/pels/19_war_college_pel_6-19-2019.pdf

I will be doing a presentation similar to the one I did at the New York Military Affairs Symposium (NYMAS). It is based upon part of my book War by Numbers: Understanding Conventional Combat.

The Hierarchy of Combat

The second conceptual element in Trevor Dupuy’s theory of combat is his definition of the hierarchy of combat:

[F]ghting between armed forces—while always having the characteristics noted [in the definition of military combat], such as fear and planned violence—manifests itself in different fashions from different perspectives. In commonly accepted military terminology, there is a hierarchy of military combat, with war as its highest level, followed by campaign, battle, engagement, action, and duel.

A war is an armed conflict, or a state of belligerence, involving military combat between two factions, states, nations, or coalitions. Hostilities between the opponents may be initiated with or without a formal declaration by one or both parties that a state of war exists. A war is fought for particular political or economic purposes or reasons, or to resist an enemy’s efforts to impose domination. A war can be short, sometimes lasting a few days, but usually is lengthy, lasting for months, years, or even generations.

A campaign is a phase of a war involving a series of operations related in time and space and aimed toward achieving a single, specific, strategic objective or result in the war. A campaign may include a single battle, but more often it comprises a number of battles over a protracted period of time or a considerable distance, but within a single theater of operations or delimited area. A campaign may last only a few weeks, but usually lasts several months or even a year.

A battle is combat between major forces, each having opposing assigned or perceived operational missions, in which each side seeks to impose its will on the opponent by accomplishing its own mission, while preventing the opponent from achieving his. A battle starts when one side initiates mission-directed combat and ends when one side accomplishes its mission or when one or both sides fail to accomplish the mission(s). Battles are often parts of campaigns. Battles between large forces usually are made up of several engagements, and can last from a few days to several weeks. Naval battles tend to be short and—in modern times—decisive.

An engagement is combat between two forces, neither larger than a division nor smaller than a company, in which each has an assigned or perceived mission. An engagement begins when the attacking force initiates combat in pursuit of its mission and ends when the attacker has accomplished the mission, or ceases to try to accomplish the mission, or when one or both sides receive significant reinforcements, thus initiating a new engagement. An engagement is often part of a battle. An engagement normally lasts one or two days; it may be as brief as a few hours and is rarely longer than five days.

An action is combat between two forces, neither larger than a battalion nor smaller than a squad, in which each side has a tactical objective. An action begins when the attacking force initiates combat to gain its objective, and ends when the attacker wins the objective, or one or both forces withdraw, or both forces terminate combat. An action often is part of an engagement and sometimes is part of a battle. An action lasts for a few minutes or a few hours and never lasts more than one day.

A duel is combat between two individuals or between two mobile fighting machines, such as combat vehicles, combat helicopters, or combat aircraft, or between a mobile fighting machine and a counter-weapon. A duel begins when one side opens fire and ends when one side or both are unable to continue firing, or stop firing voluntarily. A duel is almost always part of an action. A duel lasts only a few minutes. [Dupuy, Understanding War, 64-66]

Reminder – Speaking at Historicon in Lancaster, PA., Friday 12 July

I will be speaking at Historicon in Lancaster, PA., Friday 12 July. Historicon is one of the three major annual wargaming conventions run by the Historical Miniatures Gaming Society (HMGS). It will be run from 10 July-14 July, 2019. Their website is here: https://www.hmgs.org/general/custom.asp?page=HconHome

As part of this large convention, they have organized a “War College.” This is an impressive effort that includes 18 lectures on Thursday, Friday and Saturday. I have the last lecture on Friday, from 6 – 7 PM. The speakers for this series include published authors Paul Westermeyer, Pete Panzeri, Steve R. Waddell and John Prados, among others. Lecture descriptions are here:                                                                               . https://cdn.ymaws.com/www.hmgs.org/resource/resmgr/historicon/hcon_19/pels/19_war_college_pel_6-19-2019.pdf

I will be doing a presentation similar to the one I did at the New York Military Affairs Symposium (NYMAS). It is based upon part of my book War by Numbers: Understanding Conventional Combat.

Trevor Dupuy’s Definition of Military Combat

Ernst Zimmer: “Das Lauenburgische Jäger-Bataillon Nr. 9 bei Gravelotte” [Wikipedia]
The first element in Trevor Dupuy’s theory of combat is his definition of military combat:

I define military combat as a violent, planned form of physical interaction (fighting) between two hostile opponents, where at least one party is an organized force, recognized by governmental or de facto authority, and one or both opposing parties hold one or more of the follow-on objectives: to seize control of territory or people; to prevent the opponent from seizing or controlling territory or people; to protect one’s own territory or people; to dominate, destroy, or incapacitate the opponent.

The impact of weapons creates an environment of lethality, danger, and fear in which achievement of the objectives by one party may require the opponent to choose among: continued resistance and resultant destruction; retreat and loss of territory, facilities, and people; surrender. Military combat begins in any interaction, or at any level of combat from duel to full-scale war, when weapons are first employed with hostile intent by one or both opponents.  Military combat ends for any interaction or level of combat when both sides have stopped fighting.

There are two key points in this definition that I wish to emphasize. Though there may be much in common between military combat and a brawl in a barroom, there are important differences. The opponents in military combat are to some degree organized, and both represent a government or quasi-governmental authority. There is one other essential difference: the all-pervasive influence of fear in a lethal environment. People have been killed in barroom brawls, but this is exceptional. In military combat there is the constant danger of death from lethal weapons employed by opponents with deadly intent. Fear is without question the most important characteristic of combat. [Dupuy, Understanding War, 63-64]

U.S. Senate and Model Validation – Comments

This is a follow-up to our blog post:

Have They Been Reading Our Blog?

This rather significant effort came out of the blue for us, and I gather a whole lot of others in the industry. The actual proposed law is here:

U.S. Senate on Model Validation

Some people, we gather, are a little nervous about this effort. On the other hand, Clinton Reilly, an occasional commenter on this blog and the Managing Director of Computer Strategies Pty Ltd, Sydney Australia, nicely responds to these concerns with the following post:

I would not be too concerned by the prospect of more rigorous validation. I am sure it represents a major opportunity to improve modelling practices and obtain the increased funding that will be required to support the effort.

One of the first steps will be the development of a set of methodologies that will be tailored to testing the types of models required. I am sure that there will be no straight jacketing or enforced uniformity as it is obvious the needs served by the models are many and varied and cannot be met by a “one size fits all” approach.

Provided modellers prepare themselves by developing an approach to validation that is required by their user community they will be in a good position to work with the investigating committee and secure the support and funding needed.

In the end, validation is not a “pass-fail” test to be feared, it is a methodology to improve the model and improve confidence in the model results, and to fully understand the strengths and weaknesses of a model. This is essential if you are going to be using the model for analytical uses, and practically essential even if you are using it for training.

So this is an opportunity not a threat. It is a much needed leap forward.

Let us begin work on developing an approach to validation that suits our individual modelling requirements so that we can present them to the review committee when it asks for input.

Now, my experience on this subject, which dates back to managing the Ardennes Campaign Simulation Data Base (ACSDB) in 1987, is that many of the U.S. Military Operations Research community will not see it as “…an opportunity, not a threat.” We shall see.

The Elements of Trevor Dupuy’s Theory of Combat

Trevor Dupuy’s combat models (the Quantified Judgement Model (QJM) and the Tactical Numerical Deterministic Model (TNDM)) are formal expressions of his theory of combat. Dupuy provided an extensive discussion of the basis for his theory in his books, particularly Understanding War: History and Theory of Combat (NOVA Publications, 1987). While many are familiar with his models, fewer are aware of the concepts that he based it upon. This will be the first in a series of posts looking at those elements.

As Dupuy explained,

As a starting point for an explanation of a scientific theory, it is useful to define fundamental terms, to state and explain critical assumptions, and to establish—or limit—the scope of the discussion that follows. The definitions and explanations that follow are generally consistent with usage in the military and analytical communities, and with definitions that have been formulated for its work by The Military Conflict Institute. However, I have in some instances modified or restated these to conform to my own ideas and usage. [Dupuy, Understanding Combat, 63]

The basic elements of his theory of combat are:

Definition of Military Combat
The Hierarchy of Combat
The Conceptual Components of Combat
The Scope of Theory
Definition of a Theory of Combat

These will each be discussed in future posts.

Reminder – Speaking at Historicon in Lancaster, PA., Friday 12 July

I will be speaking at Historicon in Lancaster, PA., Friday 12 July. Historicon is one of the three major annual wargaming conventions run by the Historical Miniatures Gaming Society (HMGS). It will be run from 10 July-14 July, 2019. Their website is here: https://www.hmgs.org/general/custom.asp?page=HconHome

As part of this large convention, they have organized a “War College.” This is an impressive effort that includes 18 lectures on Thursday, Friday and Saturday. I have the last lecture on Friday, from 6 – 7 PM. The speakers for this series include published authors Paul Westermeyer, Pete Panzeri, Steve R. Waddell and John Prados, among others. Lecture descriptions are here:                                                                               . https://cdn.ymaws.com/www.hmgs.org/resource/resmgr/historicon/hcon_19/pels/19_war_college_pel_6-19-2019.pdf

I will be doing a presentation similar to the one I did at the New York Military Affairs Symposium (NYMAS). It is based upon part of my book War by Numbers: Understanding Conventional Combat.

How Common are Casualty Estimates?

I do note that several of the talking heads on TV have mentioned that providing a casualty estimate is standard practice when briefing for a military action. This may be the case, I have never been to one of those briefings. But….while casualty estimates for a single proposed operation may be common, my experience is that they are not that common when it comes to evaluating wars.

There were certainly multiple estimates done in 1990 for the 1991 Gulf War both inside and outside the Government. A number of estimates were high and many have never been publicly released. I have yet to have seen any systematic analysis of the casualty estimates done for the Gulf War. Most of these were based upon combat models, so as such, serve as a validation test for these combat models. Wikipedia used to have a discussion on this issue, but it has since disappeared. Needless to say, the estimate derived from a combat model made before a war is probably something that would be very telling. The fact that DOD never conducted an analysis after the war of the various estimates is also very telling.

Now, we did do an estimate in fall of 1995 for a Bosnia peacekeeping operation (the Dayton conference ended with a peace agreement in November 1995). This was the only estimate done for this operation. According to rumor, the Joint Chiefs of Staff asked the Deputy Under Secretary of the Army for Operations Research (DUSA-OR) if they could do such an estimate, and were told that there was no methodology to estimate casualties in an Operation Other Than War (OOTW) (see page 309, America’s Modern Wars). So The Dupuy Institute ended up doing such an estimate, and we did it based upon two different methodologies (see Appendix II, America’s Modern Wars).

As far as I know, we were also the only people who did an estimate in 2004/2005 for the cost and duration of an extended guerilla war in Iraq. Some one else may have done some work on duration, as I kept hearing the statement made that guerilla wars lasted an average of ten years. I could never tell if that figure was pulled from our work or if someone else did some similar work. Regardless, I think we were the only people who did an analytically based casualty estimate of the war. See Chapter 1, America’s Modern Wars.

Now, when I was writing America’s Modern Wars, I decided to look into what estimates had been made in the first half of the 1960s for the war in Vietnam. I really could not find anything analytical, although my search was not exhaustive (meaning I am still waiting for someone to prove me wrong). But, it does not appear that any analytically based estimate was made during the 1960s for casualties and duration for what turned into America’s third bloodiest and second longest war. See pages 4, 29-30, America’s Modern Wars.

So, there appears to have been an estimate done for the casualties for a strike on Iraq (although I doubt it will be made public). On the other hand, it is doubtful if anyone has done an estimate for the full range of options, or addressing the Iranian counter-options, or examining an extended conflict with Iran. This would certainly have to address the range of counter-strikes and other options available to Iran if such a conflict further developed. For various reasons, the DOD seems hesitant to do these types of estimates.

Now, I happen to think our decision makers would be better served if they indeed did have some estimates of casualties and duration of these conflicts when they were contemplating their options. They were not done for the Vietnam War, only one estimate was done for Bosnia and I think only one estimate was done for the continuing war in Iraq. I am guessing, based upon that track record, no such estimate has been made for an extended conflict with Iran.