Links take you to a different view (page, whatever), without affecting any other state.
Buttons perform actions (that might also present a new panel).
The problem is bad UIs muddling the waters between the two (having a plain text "label" looking like a link that does "Delete everything in my account" for example).
This fits with the convention that, generally speaking, links generate a GET request and buttons generate a POST one —
javascript & the odd GET-form-for-searching aside. Of course, webapps have seriously muddied the water!
I suspect that most people would have trouble coming up with a definition for a UI button that couldn't just as easily describe web links.