Neither can you easily notice that when a human teacher/tutor does that. At least with math, you're able to try it yourself and see if it works. I've had many cases where following through on a misleading explanation by a teacher/book actually ended up leading to me better retaining the topic.

Or when you tell it you think the answer is something else it agrees with you and apologies that yes, now it can see, 2+2 does equal 5.

That's absolutely a bug that needs to be fixed, but I think it's possible. Maybe have the network which generates the answer be moderated by another network that assesses the truthiness of it.

It's just a matter of priorities for the company designing the models.

It might be possible, but nobody knows for sure, because these models are rather more mysterious than their architecture suggests.

> Maybe have the network which generates the answer be moderated by another network that assesses the truthiness of it.

Like a GAN? Sometimes you can do that, but it seems not always.

If this was simple and obvious, they'd have done it as soon as the first one was interesting-but-wrong.

Especially in a limited domain like grade school math, it seems entirely plausible that we can have models in very short order that ~never hallucinate. There's no external dependencies and the problem space is extremely well-defined and constrained. Much, much, much easier than making something like Chat-GPT never hallucinate

That is the entire purpose of LLMOps. Provide guardrails to prevent hallucination and ensure precise control of GenAI output.

How can you tell what's true or not?

You have to develop your own QA methods to ensure output is exactly what you want.