a potential con of anything MSMQ based is the restriction on maximum message size. IIRC it is approximately 4MB, which you might easily run into if you're dealing with large files and storing the file content within the message.
I'd recommend staying away from hand-rolled solutions as there is a bunch of somewhat difficult stuff that needs to be gotten just right - like how transactions are handled, how exceptions cause rollbacks, how to stop rolling back endlessly (poison messages), how to integrate with long-running workflows so that the state management boundaries line up, and more.
You will probably want some kind of durable/transactional messaging infrastructure, so not using MSMQ you'd be left with Service Broker on the Microsoft platform, or some other alternative like ActiveMQ. MSMQ has the benefit of already being installed on all Windows machines, as opposed to Service Broker which isn't.
In terms of choosing between NServiceBus, Mass Transit, and Rhino Service Bus - this Stackoverflow answer comparing NServiceBus to MassTransit would be a good place to start..
In our 3.1 release, we're introducing NSB Studio - a set of Visual Studio integrated modeling tools that enable you to model your system at a higher level of abstraction and have much of the configuration and initialization of NServiceBus be done for you automatically. I'd say that this really tips the scales in favor of NServiceBus.
NServiceBus is a good product but beware of licensing problems. It has a tendency to change it licensing policy as the authors wishes. Take a look for example at old license information.
It could happen that in the middle of you project development you will find out that you have to pay lot's of money for NServiceBus.
Also free version have performance limitations.
MassTransit is absolutely free open source, it has no limitations, and is under Apache 2.0 license.
InfoQ to Ayende:You have previously written a service bus for .NET
yourself, namely the Rhino Service Bus. Should users of Rhino Service
Bus now reconsider and move to NServiceBus?
Ayende: I built Rhino Service Bus around 2008. I built it mostly
because I wasn't happy with the state of the other service buses at
the time. I have had different concerns and direction when building my
service bus, but that was 4 years ago. In that time, I think that
NServiceBus made great strides in becoming an easier to use product
and having a much better out of the box development story. If I was
starting out with service buses today, I strongly doubt that I would
be building my own.