If its not quite hot enough you can end up having a nice incubator for bacteria

Yes, but water coming from the cold water tap has spent less time sitting in that accumulated stuff.

Seems like the presence of the accumulated gunk in the cylinder means the hot water must be cleaner by definition? The accumulation being the net difference between what was dissolved in the incoming water and what is dissolved in the outgoing water.

With the exception of gunk that is from the lining of the cylinder itself of course.

There can't be more minerals coming out than what went in though. It's not like calcium grows. Maybe it might break off and come out as some concentrated chunks, but overall there would have to be less coming out for it to accumulate in the tank.

Yeah, so the gunk has less time to fall out of the cold water.