EDIT (2015): If you are designing for a wide audience, recall that the outline is often a critical accessibility feature for users who navigate via keyboards or require more apparent visual feedback. If you remove the outline, make sure to define an alternative focus state that provides appropriate visual feedback to your users.