I will be a bit sloppy with my description here: There are two ways to "prove" thermodinamics. You can start from the axioms of probability and statistics *together* with an assumption that matter is made out of atoms or similar constituents, and you can derive, through statistical physics, most of thermodynamics. That is a perfectly reasonable and practical way to do it.
However, it is extremely valuable to also understand that the vast majority of thermodynamics (including our understanding of entropy) can be derived from completely independent axioms that have no relationship to information theory or the knowledge of discreteness of matter. You need to define temperature as an transitive observable (zeroth law of thermodynamics), you need to define heat as a type of energy (first law of thermodynamics), and you need some version of the second law (there are a few equivalent ones, but the easiest is that "heat flows only from high temperature to low temperature"). Lastly, you need equations of state as a starting point (instead of assumptions about the microscoping constituents of matter). Tadaaa, you have all that is necessary to derive all our knowledge of thermodynamics without knowing statistics or atoms.
Aesthetically this is incredibly pleasing and intellectually it is quite surprising that these two sets of axioms that look nothing alike produce the same self-consistent physics. To be productive in cutting edge research you usually need to be comfortable with both.
Edit: corrected associative -> transitive as pointed out below.
Sure enough, the principles of Carnot's thermodynamics and the premises of statistical mechanisc look quite differently. The thing is: since both form the same thermodynamics there must be a connection.
I submit: the qualification 'completely independent axioms' is incorrect. There is the observation: temperature is transitive. This transitive property is a statement of conservation (In terms of Carnot's thermodynamic it used to be thought of as conservation of Caloric.) The concept of Conservation of a quantity correlates with information.
We have that statistical mechanics subsumed Carnot's thermodynamics.
The laws of Carnot's thermodynamics are theorems of statistical mechanics. (Those theorems weren't necessarily stated explicitly. I'm saying the principles of statistical mechanics are sufficient to imply the laws of Carnot's thermodynamics.)
Statistical mechanics needs to assume the existence of atoms to give you the results of Carnot's. Carnot's thermodynamics does not. That makes it a bit less straightforward to say one strictly follows from the other. Otherwise the gist of what you say seems reasonable.
> it is quite surprising that these two sets of axioms that look nothing alike produce the same self-consistent physics
Why is that surprising? This sort of thing seems to be ubiquitous in our universe. F=ma is equivalent to the principle of least action. Turing machines are equivalent to the lambda calculus. Electrodynamics is equivalent to the existence of a finite reference velocity.
Also:
> You need to define temperature as an associative observable
Did you mean transitive? My understanding is that the required axiom is that if system A is in thermal equilibrium with system B, and B is in equilibrium with C, then A will be in equilibrium with C. That's transitivity, not associativity.
Indeed, this type of "surprises" is how you know you are probably on the right track with your choice of abstraction. But I still find the word "surprise" reasonable for the emotion I feel when I observe something like that. Very pleasing reassuring intellectually satisfying surprise. A "sufficiently smart armchair theorist" might discover these "surprises" in advance simply from consistency and beauty and simplicity considerations, but these armchair theorists are usually just a made up thing we imagine when teaching.
I corrected the wrong use of the word associative, thanks!
However, it is extremely valuable to also understand that the vast majority of thermodynamics (including our understanding of entropy) can be derived from completely independent axioms that have no relationship to information theory or the knowledge of discreteness of matter. You need to define temperature as an transitive observable (zeroth law of thermodynamics), you need to define heat as a type of energy (first law of thermodynamics), and you need some version of the second law (there are a few equivalent ones, but the easiest is that "heat flows only from high temperature to low temperature"). Lastly, you need equations of state as a starting point (instead of assumptions about the microscoping constituents of matter). Tadaaa, you have all that is necessary to derive all our knowledge of thermodynamics without knowing statistics or atoms.
Aesthetically this is incredibly pleasing and intellectually it is quite surprising that these two sets of axioms that look nothing alike produce the same self-consistent physics. To be productive in cutting edge research you usually need to be comfortable with both.
Edit: corrected associative -> transitive as pointed out below.