It tells the compiler that you won't be calling any function which modify the exception, which may help to optimize the code. Probably doesn't make much of a difference, but the cost of doing it is very small too.
are you going to modify the exception? if not, it may as well be const. same reason you SHOULD use const anywhere else (I say SHOULD because it doesn't really make that much difference on the surface, might help compilers, and also help coders use your code properly and not do stuff they shouldn't)
exception handlers, may be platform specific, and may put exceptions in funny places because they aren't expecting them to change?
a reference so you can access the exception polymorphically
a const to increase performance, and tell the compiler you're not going to modify the object
The latter is not as much important as the former, but the only real reason to drop const would be to signal that you want to do changes to the exception (usually useful only if you want to rethrow it with added context into a higher level).
Exception objects live in their own memory space† so you don't have to worry about catching exceptions created in temporary expressions.
All you're doing is promising that you won't modify the exception object, but since exception objects should have an immutable interface‡, there is really nothing practical here.
However, it might make you feel warm and cosy when you read it — that's how it is for me!
† They have their own, special, thread-local stack. ‡Disclaimer: Boost.Exception breaks this in order to do funky stuff and add exception details, post-construction. But this is hackery!