I'd say RL is an advanced robotics topic. There are some books out there that cover introductory or general RL aspects... after that you'll have to dive into papers and specific code bases.
Might not be a bad idea to start with a Python-based, practical RL book and move from practical demonstrations towards theory.
Although not an RL book, I really like Data-Driven Science and Engineering by Brunton and Kutz:
The Robotics resources and texts by Peter Corke are good: https://petercorke.com/ -also has this good related course: https://robotacademy.net.au/
"Controls Engineering in the FIRST Robotics Competition" by Tyler Veness is free and a good short reference: https://controls-in-frc.link/
Also add the MIT Robotics Series books: https://mitpress.mit.edu/series/intelligent-robotics-and-aut...
Algorithms for Decision Making is free and awesome: https://algorithmsbook.com/
Also looks really good (MIT Press hardcopy): https://introduction-to-autonomous-robots.github.io/