The Problem with LLMs

Plausible answers to a question everyone knows is impossible to answer with one correct answer.. and so AI just makes up plausible answers without knowing the answer.. that is what it does in general, actually.. the problem is that the content of those plausible answers differ per session, not just per model..

The intent is purely to convince the user that it has answered correctly.. it is not possible to get it to always return the same answer every time, because it does not KNOW the answer, and the answer is also UNKNOWABLE.

In the future it would be useful to train these models to admit ignorance.. say ‘I don’t know’ - but that would run entirely contrary to current training and fine-tuning methods where models are directly incentivised to convince the user that their answer is correct!

Also it would not be possible to get a concrete ‘I don’t know’ - because it’s a moving platform not a static mapping from ‘question’ to ‘answer’. It is not like reading a book, more like watching the reflections from a running river.

However most people hailing it as ‘Oracle of Delphi’ have no idea of any of this.. and that is why AI stocks are still ridiculously overhyped.