Isn't the bible involved in the whole inauguration part? Besides, candidates routinely discuss their faith, and "In God We Trust" appears on the currency.
Edit: Hah, I just realized that congressional sessions open with prayer as well. Not sure what other countries does this?
The bible is involved at the behest of the inauguree; we've had christian presidents, and so they've largely sworn in on the bible. John Quincy Adams used a book of laws, and Coolidge didn't use a book at all (there are a couple others, but they were unintentional).
Likewise, the US prayer is non-denominational (it typically is monotheistic though). Ireland, Canada, South Africa, and the UK also have parliamentary prayers.
UK currency often features the letters "D.G.", which are the initials to a latin phrase meaning "by the grace of God", but other European currency references to God have ended with the switch to the Euro.
The US certainly has above average entanglement of religiosity and governance, but hardly in a sense that makes it a theocracy. Politicians talking about faith and God is a very different thing from, eg, the country being run by the pope.
Edit: Hah, I just realized that congressional sessions open with prayer as well. Not sure what other countries does this?