EDIT: Just saw your RVM mention in the post. Definitely the way to go.
You're going to want to install RVM -- it's an amazing package that will let you manage different Rubys and different sets of gems on the same machine. You can switch back and forth with total ease.
Once you got everything get up, you can see all of your installed rubys at the command line with with rvm list, and switch with rvm use ruby-head, for example. RVM keeps the gems on each ruby separate, which should help with your question.
RVM allows you to manage different versions of Ruby and Gems. You can install a version of ruby using, for example
rvm install 1.9.2
You can then use it using:
rvm use 1.9.2
Use specific gems on a per project basis with gemsets.
If you want further namespacing you can set up gemsets; directories which will contain specific gems for a specific project.
rvm gemset create myproject
then you can use them like so:
rvm use 1.9.2@myproject
Automation
To automate the process of switching gems, pop .ruby-version and .ruby-gemset files in your project root. Pop the version of Ruby and name of the gemset you want to use inside them and RVM wil select the correct gemset when you cd into your project directory.
Installing gems into your gemset
Install your gems into your gemset in the usual way using bundler if you are using it:
bundle install
or just using the regular old:
gem install mygem
The gems will go in the right gemset.
RVM Alternatives
You might also want to check out rbenv, which does similar job.