Use a DateFormat. In an internationalized application, use the format provide by getInstance. If you want to explicitly control the format, create a new SimpleDateFormat yourself.
For this particular question, the standard suggestion of java.text.SimpleDateFormat works, but has the unfortunate side effect that SimpleDateFormatis not thread-safe and can be the source of particularly nasty problems since it'll corrupt your output in multi-threaded scenarios, and you won't get any exceptions!
I would strongly recommend looking at Joda for anything like this. Why ? It's a much richer and more intuitive time/date library for Java than the current library (and the basis of the up-and-coming new standard Java date/time library, so you'll be learning a soon-to-be-standard API).
I am providing the modern answer. The Timestamp class is a hack on top of the already poorly designed java.util.Date class and is long outdated. I am assuming, though, that you are getting a Timestamp from a legacy API that you cannot afford to upgrade to java.time just now. When you do that, convert it to a modern Instant and do further processing from there.
Pick how long or short of a format you want by specifying FormatStyle.SHORT, .MEDIUM, .LONG or .FULL. Pick your own locale where I put Locale.GERMAN. And pick your desired time zone, for example ZoneId.of("Europe/Oslo"). A Timestamp is a point in time without time zone, so we need a time zone to be able to convert it into year, month, day, hour, minute, etc. If your Timestamp comes from a database value of type timestamp without time zone (generally not recommended, but unfortunately often seen), ZoneId.systemDefault() is likely to give you the correct result. Another and slightly simpler option in this case is instead to convert to a .MEDIUM0 using .MEDIUM1 and then format the .MEDIUM0 in the same way as I did with the .MEDIUM3.