Type the content (i.e., the "text" property field) inside a <html></html> tag. So you can use <br> or<P> to insert a newline.
For example:
String labelContent = "<html>Twinkle, twinkle, little star,<BR>How I wonder what you are.<BR>Up above the world so high,<BR>Like a diamond in the sky.</html>";
It will display as follows:
Twinkle, twinkle, little star,
How I wonder what you are.
Up above the world so high,
Like a diamond in the sky.
This is horrifying. All these answers suggesting adding to the start of the label text, and there is not one word in the Java 11 (or earlier) documentation for JLabel to suggest that the text of a label is handled differently if it happens to start with <html>. Who says that works everywhere and always will? And you can get big, big surprises wrapping arbitrary text in and handing it to an html layout engine.
I've upvoted the answer that suggests JTextArea. But I'll note that JTextArea isn't a drop-in replacement; by default it expands to fill rows, which is not how JLabel acts. I haven't come up with a solution to that yet.