I've always wanted to run an experiment where NIH or NSF split the proposals randomly 50/50. Group A gets peer reviewed and awarded the way the current system works. Group B gets peer reviewed and the top half of proposals gets put into a lottery and awarded at random. I am willing to bet the outcomes and impact of Group B will always be better than Group A.
Cindy I suspect you might be right - especially if we measure for breakthrough discoveries.
The problem you face running such a trial is those scientists that control the current system are not too interested in trying anything that might take away their control. They love to mis-quote Churchill and claim their is no better way than the current system, but they are none too keen to try any alternatives.
