Lossless compression _is_ a scorecard for lossy compression. Thanks to the ideas from arithmetic coding.

There’s no way I could reproduce a calculus textbook verbatim, but I can probably prove all the important theorems in it.

Even then, given half of any sentence in the book, I don’t rate my chances of reproducing the next half. That’s more a question of knowing the author’s style than knowing calculus itself.

Does arithmetic coding capture all of that?

Let’s say you come up with the exact same text given an initial seed (of you ate this and that that day). Then it really is just the arithmetic function of the difference, as it is deterministic.

Now, who will get closer to the algorithm with that added arithmetic coding? You, knowing the proofs, or a random guy that doesn’t even speak the language? Does it then measure intelligence all else being equals!

Yep

Or just require the results to be 99.99 percent accurate (within some edit distance)

If it's 99.99% accurate arithmetic coding would have next to no data stored.

Arithmetic coding is optimal in turning probabilistic data into lossless data. There's provably no way to do it more efficiently than arithmetic coding. The data it needs for corrections is smaller the better the predictions are.

So given this why even dwell on ways that add any form of subjectivity. Arithmetic coding is there. It's a simple algorithm.