This is exactly what we do at our team too -- we're developer centric and as developers, we love using Github and Github issues (+ pull-requests) for everything (including PM and HR)
One thing we do miss are certain features from Trello-like tools that give a nice overview of your issues and facilitate PM, so we built a Chrome/FF extension to fill in those gaps (http://zenhub.io)
One thing I wish Github would do is allow for a more granular permission structure. It would be fantastic if we could allow people without github accounts do things like submit issues, or allow certain users (e.g. non-technical staff) only have access to the wiki and issue tracker.
Right now the permissions are centered around what you can do with repositories, it's very developer-centric. I think there is a lot that GitHub could do here to expand their service to be more applicable to an entire organization.
One thing we do miss are certain features from Trello-like tools that give a nice overview of your issues and facilitate PM, so we built a Chrome/FF extension to fill in those gaps (http://zenhub.io)
Disclaimer: I work at Axiom Zen