It's just how this sort of a model works in inference. You can set the temperature to zero, then you'll get mostly constant outputs (but not quite, since as far as I can tell nobody is working on determinization of such systems, and it is not really deterministic).
I'm going to have to dust off some ancient math... I think LLMs are based on neuronal networks which are based on regression models which are based on systems of differential equations for which last time I studied this stuff, we only had probabilistic solutions?
It's been a bunch of decades since I did anything even remotely related to these areas (and I was never good at math).
