I must be doing something wrong but I can't see how to select an element or button by the letter over it's highlight. Ctrl+Letter doesn't do anything for me. From the README, that seemed to be the way to do it?
If there are multiple highlighted elements on screen, Ctrl-Letter lets you select which element is highlighted green. The green element can then be activated/selected with Enter.
As others already suggested, the possibility of directly activating a specific element by shortcut would be great. Awesome app nonetheless!