I don't have much experience with macs, so not sure if this version is ok for your, but have a look at http://www.phil.uu.nl/~xges/ssh/
If this particular app doesn't work, that's what you're looking for anyways - ssh agent. On unix-like boxes, you'd want to start your whole window manager through that, to get the global effect, but it might not be possible in osx.
sshkeychain is one possibility. installs fine with macports using:
sudo port install sshkeychain
it uses the keychain to store passwords, and you may simply launch it at the start-up of your login session (using at the first launch the usual right-ght click in the dock's icon + "launch at startup")
Note that Apple's svn uses keychain to store passwords but not necessarily the svn binary you would build with macports.
After much exploration, I think I've found the answer to this issue completely. First, make sure you do ssh-add -K ~/.ssh/your_key_here. This adds the key to your keychain. Some places, I have read that this is enough, but I wasn't certain. This is also mac-specific, so if you need to do this on another unix flavor, you won't have this option necessarily.
For good measure, I edited the ~/.ssh/config file (you may have to create it) to point to all the keys I have. Mine has the following:
According to the man page for ssh_config, it will try these in order. I'm not sure if the first three default ones I have listed need to be there, but I have included them anyway.
Since macOS 10.12.2 you can use the UseKeychain option.
Read more here or look into man ssh_config.
UseKeychain
On macOS, specifies whether the system should search for passphrases in the user's keychain
when attempting to use a particular key. When the passphrase is provided by the user, this
option also specifies whether the passphrase should be stored into the keychain once it has
been verified to be correct. The argument must be ``yes'' or ``no''. The default is ``no''.
and edit your ssh config (~/.ssh/config) file to automatically load keys from the key chain to the ssh-agent (AddKeysToAgent yes option) and store passphrases in the keychain (UseKeychain yes option):