There's no extensive benchmarks, but I did do a Flappy Bird pass@3 test just to show that a 1.58bit model does in fact work well!

The goal was to showcase that MoEs quantized down to 1.58bit without any further training does in fact work!

Right and congrats on all of that. But I need to know the evals to know whether this actually makes sense versus alternatives.