There's a theory that these LLMs are learning these specialized internal structures anyways. If you could distill _those_, now that would be useful! (cheaper inference, ability to deploy on mobile etc.)