Sometimes a plugin can be an attractive solution even for a simple problem. In this case we're lucky as there is eunuch.vim by the almighty Tim Pope.
In its own words eunuch.vim provides
Vim sugar for the UNIX shell commands that need it the most. Delete or rename a buffer and the underlying file at the same time. Load a find or a locate into the quickfix list. And so on.
Perfect. It has what we need, plus some additional tools if we're on a UNIX system.
The command you are looking for is
:Remove!
Again, remap it if you need it a lot, e.g. :nnoremap <Leader>rm :Remove!<CR>.
I like being able to delete files from within vim, but I am also paranoid about accidentally deleting important work that, for one reason or another, is not yet under version control. I find it useful to combine the previous information from @glts and @accolade with this answer on how to use the confirm command to prompt before quitting vim.
Putting these together, I added a function to my ~/.vimrc, which prompts before deleting the file and closing the buffer, and mapped it to a key combination:
nnoremap <Leader>d. :call DeleteFileAndCloseBuffer()
fun! DeleteFileAndCloseBuffer()
let choice = confirm("Delete file and close buffer?", "&Do it!\n&Nonono", 1)
if choice == 1 | call delete(expand('%:p')) | q! | endif
endfun
If you are one keystroke less paranoid than I am, you can append <CR> to the first line
This may be an unpopular opinion but your file tree explorer (Netrw/NerdTree) is going to be the simplest and safest way to delete a file. Even if OP is not using NerdTree, the in-built plugin Netrw will work just as well.