Label has an important focus handling responsibility.Its purpose is to allow you to place a caption with an access key. It has a Target property, which indicates the target of the access key. Like this...
In the absence of the Target property, the Label control does nothing useful. You'll just hear a beep if you press the access key indicating 'unable to process request'
The WPF Textblock inherits from FrameworkElement instead of deriving from System.Windows.Control like the Label Control. This means that the Textblock is much more lightweight. The downside of using a textblock is no support for Access/Accerelator Keys and there is no link to other controls as target.
When you want to display text by itself use the TextBlock. The benefit is a light, performant way to display text.
When you want to associate text with another control like a TextBox use the Label control. The benefits are access keys and references to target control.
Label can be used as an alternative to TextBlock for situations where minimal text support is required such as the label for a control. Using Label can be advantageous because it requires even less resources (lighter weight) then a TextBlock.
The two biggest reasons for the confusion regarding TextBlocks and Labels are Windows Forms and common sense.
When you wanted to slap a small bit of text on your form in Windows Forms, you used a Label, so it follows (incorrectly) that you would do the same thing with a WPF Label.
Common sense would lead you to believe that a Label is lightweight and a TextBlock isn't, when the opposite is true.