Hacker News new | past | comments | ask | show | jobs | submit login

I had a similar experience with regular o1 about integral that was divergent. It was adamant that it wasn't and would respond to any attempt at persuasion with variants of "its a standard integral" with a "subtle cancellation". When I asked for any source for this standard integral it produced references to support its argument that existed but didn't actually contain the integral. When I told it the references didn't have the result and backpedalled (gaslighting!) to "I never told you they were in there". When I pointed out that in fact it did it insisted this was just a "misunderstanding". It only relented when I told it Mathematica agreed the integral was divergent. It still insisted it never said that the books it pointed to contained this (false, non-sensical) result.

This was new behaviour for me to see in an LLM. Usually the problem is these things would just fold when you pushed back. I don't know which is better, but being this confidently wrong (and "lying" when confronted with it) is troubling.




> but being this confidently wrong (and "lying" when confronted with it) is troubling.

It works in politics, marketing, and self-promotion.

If you use the web as a training set, those categories dominate.


Maybe they also trained the model on Sam Altman. ;)


Please don't crack ones like that when I'm drinking my coffee.


I've also had it invent non-existent references.

> being this confidently wrong (and "lying" when confronted with it) is troubling.

I don't find it troubling. I like being reminded to distrust and confirm everything it offers.


The troubling part is that the references themselves existed -- one was an obscure Russian text that is difficult to find (but is exactly where you'd expect to find this kind of result, if it existed).




Consider applying for YC's Summer 2025 batch! Applications are open till May 13

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: