In Understanding User and Kernel Mode Jeff Atwood says that code that uses exceptions is slow because “exceptions imply kernel mode transitions”. From the ongoing debate in the comments it seems that this may be true for some languages running under .NET that use Window’s Structured Exception Handling but it’s misleading to state that it’s true in the general case.
There are all sorts of reason why exceptions might be slow - they’re just more code the CPU has to execute after all. I worry though about general claims that any idiom is slow. Search a selection of programming blogs for “X is slow” and I guarantee that in the comments someone will say “I never knew that. I’m going to avoid X in future”. I imagine that after a couple of years of cargo-culting these folks are working with a miniscule subset of their chosen language, circumscribed by all those “slow” things that they’re religiously avoiding.
Measure instead of guessing. Look for algorithmic complexity: if you can go from O(N^2) or worse to O(N log N), O(N) or O(1) you might gain far more than a few percent. But then again you might not - so measure.