One one hand, the analogy is imperfect with billiard balls precisely because they are big collections of particles, not actual particles. It might help to imagine instead a collision of, say helium atoms.
On the other hand, if you see a "video" of two warm, vibrating billiard balls colliding, which then cool off and stop vibrating, it's still only a good guess that the video is in reverse. It's only highly improbable that the motion of the atoms in the billiard ball, air, table, etc cancel out to produce more order, not impossible. That improbability, AFAICT, is the linchpin of the Second Law of Thermodynamics.
On the other hand, if you see a "video" of two warm, vibrating billiard balls colliding, which then cool off and stop vibrating, it's still only a good guess that the video is in reverse. It's only highly improbable that the motion of the atoms in the billiard ball, air, table, etc cancel out to produce more order, not impossible. That improbability, AFAICT, is the linchpin of the Second Law of Thermodynamics.