Do not make the same mistake I was making. I was handling 404 errors and wanted to redirect with 404=filename in the querystring, i.e. mysite.com?404=nonExistentFile.txt.
QueryString Keys cannot begin with numbers. Changing from 404 to FileNotFound solved my issue, i.e. mysite.com?FileNotFound=nonExistentFile.txt.