Not sure if this clears the credentials but I was able to simply change the credentials by running aws configure again and entering in new credentials. I was also able to make multiple profiles by running aws configure --newusername and I could run a future command with that user by appending --profile newusername. Good tutorial here: https://www.crybit.com/configure-iam-user-on-your-linux-machine/
Maybe just a Windows thing but if you edit the ~.\aws\config and ~.\aws\credentials files like is suggested by Aniket Thakur, you might run into an issue where aws configure complains about a profile you deleted not being available. This seems to be because the AWS_PROFILE environment variable is referencing that profile. Unset that variable and you should be back in business.
Tried to comment under his answer but I don't have enough points (on this account).