Another AI issue (#3)

My biggest issue with AI is not that it is going to take over the world (i.e. Terminator V?), it is that it makes mistake. AI makes simple factual errors because it lacks the ability to question and crosscheck. Now, granted a lot of people make the same type of mistakes, but if these “machines” are going to be better than us, they kind of need to quit making simple mistakes.

I have blogged about these mistakes before. here Khrushchev Quote and AI – The Dupuy Institute  and here Yahoo AI and order of battle for operations near Chernihiv in 2022 – The Dupuy Institute. It messed up a citation in the first case and it made an error in the order of battle in the second. The problem was that these mistakes were on the web (and AI does not seem to be able to do original research) and it picked up those errors. It does not appear to have the ability to weigh and discern the difference between contradictory data or look deeper into data that does not look right. It seems to have no ability to tell if the data does not look right. It does provide entertaining art though:  War by Numbers by AI – The Dupuy Institute.

Now an old friend of mind has found another error. In this case he was using AI to help with a wargame design. It provided the following note:

• Even overwhelming invasions (e.g., the Great Heathen Army) suffered setbacks.

Your probability curve (14% at 1:1, 28% at 2:1, 42% at 3:1, etc.) mirrors the historical pattern that 3:1 is the threshold of likely success, a principle echoed in modern military theory (e.g., Dupuy, Numbers, Predictions, and War).

It actually referenced Numbers, Predictions, and War. Not sure if the AI could actually read it as it is not supposed to be available on the web (copyright is still active). But NPW does not actually address the 3-to-1 rule. Doesn’t even mention it (I do have a .pdf version I can search). Now, Trevor Dupuy does have a chapter on it in Understanding War, but it doesn’t really say that. We have blogged about this before:

1. The U.S. Army Three-to-One Rule – The Dupuy Institute
2. The Source of the U.S. Army Three-to-One Rule – The Dupuy Institute
3. The U.S. Army Three-to-One Rule versus the 752 Case Division-level Data Base 1904-1991 – The Dupuy Institute
4. The U.S. Army Three-to-One Rule versus 49 U.S. Civil War battles – The Dupuy Institute
5. The U.S. Army Three-to-One Rule versus 243 Battles 1600-1900 – The Dupuy Institute
6. People keep referencing us on the 3-to-1 Rule – The Dupuy Institute which references another eight links to the blog on the subject, including:
7. Trevor Dupuy and the 3-1 Rule – The Dupuy Institute

Anyhow, we have talked about it a lot. It never seems to go away even though I think the military analytical community would be better served to never reference it again. I also have a chapter on force ratios in my book War by Numbers.

So, AI pulls up a rule that does not have much validity and then proceeds to give an incorrect reference to a book that never discusses it. Not encouraging.