You can set the directory option to the location that you want vim to store its swap files, e.g.:
mkdir -p $HOME/.vim/swapfiles # this dir must exist vi does not create it
" $HOME/.vimrc
:set directory=$HOME/.vim/swapfiles//
I use trailing double path separators because, from the help docs:
For Unix and Win32, if a directory ends in two path separators "//" or "\\", the swap file name will be built from the complete path to the file with all path separators substituted to percent '%' signs. This will ensure file name uniqueness in the preserve directory.