I started using zsh with oh-my-zsh as it made me realize how powerful, ergonomic and productive my shell could be. I was annoyed by the slowness. I thought of moving to barebones zsh with the few plugins. My colleague suggested that I used prezto. I have been using prezto ever since with my preferred plugins enabled. And, I have been happy with my setup. It has all the features I need without the slowness of oh-my-zsh. I ended up getting all the features of fish shell using plugins. I didn't want to use fish because it's syntax is not compatible with bash which I have to write at work.

Give prezto a try. You might like it.