git checkout $file reverts the state of the file $file to the last committed state. I think you can use git checkout SHA-1 -- $file to revert the file to the commit identified by SHA-1.
How many commits do you need to go back and select from? If it's just one, maybe take a branch just before it, checkout the file you committed and then use git add -p to add it how you wanted it. Then you can go back to where you were and checkout the file from your temp branch.
Other alternatives include going back and editing the commit with git rebase -i (marking the commit as edit, then doing a git reset HEAD^ and redoing the commit when dropped back into the shell).
If the changes you need to select from are spread over a series of commits, it may be better to extract them as patches (or a patch covering all of them) and hand-edit the patch, taking out the changes you want to keep, and feeding the residual into git apply --reverse.