It's a bug report on TortoiseGit that seems to be describing your exact problem.
Basically, it seems that TGitCache.exe shouldn't be doing that, and in the newest versions of TortoiseGit it doesn't. Perhaps you might try upgrading your TortoiseGit?
TGitCache.exe will disappear from the process list.
If you need it again, you can turn it on by selecting another option, like "Default". After a click in an explorer window TGitCache.exe will run again.
Open TortoiseGit settings and open Advanced page. There set CacheTryIcon to true and restart TGitCache.
Now you have a new TortoiseGit icon in the systray where you can select "Exit" to close TGitCache (it will restart automatically again). You can also select "Disable Status cache" which will set Status cache to None (see https://stackoverflow.com/a/16921151/3906760).