C++ exceptions make a hell of a lot more sense when you realise you shouldn't be handling them as close as possible to the throw site by covering your code in try-catch blocks. That defeats the point of separating the error-handling code and the "business logic."
Exception-neutral code is generally clearer and more expressive, and in many cases it's smaller and faster because all the destructor code is already there and will run anyway whether you leave the scope due to return or throw, but you don't have the overhead of a try-catch (the catch might involve a runtime type-identification check.)
I also find that code that aims to be exception-neutral is more likely to be correct, partly just because it's easier to reason about clearer, expressive code. Compare the various implementations of an assignment operator in the notes from my Exception Safety talk.
I've said it before and I'll say it again: destructors are my favourite C++ language feature. Without destructors you have no RAII, no exception-neutral code and automatic deterministic resource cleanup is an order of magnitude harder.
And yes, mysql++ misuses exceptions for flow-control in some places. I think Warren's planning API changes for version 3.0 so the point's worth raising on the mailing list.