I was under the impression that there is no guarantee that the data will actually be there with ipfs? Not sure about swarm. It's possible that it could be somehow paid for when launching the contract. Anyway, the idea makes sense

However, in my experience, contracts don't usually have a lot of source code. I think storing them on chain wouldn't be much of a problem. Unfortunately though, that might incentivize minification and other self defeating measures.