If, running an Oracle HotSpot JDK 1.7.x, on a Linux platform where your locale suggests UTF-8 (e.g. LANG=en_US.utf8), if you don't set it on the command-line with -Dfile.encoding, the JDK will default file.encoding and the default Charset like this:
... suggesting the default is UTF-8 on such a platform.
Additionally, if java.nio.charset.Charset.defaultCharset() finds file.encoding not-set, it looks for java.nio.charset.Charset.forName("UTF-8"), suggesting it prefers that string, although it is well-aliased, so "UTF8" will also work fine.
If you run the same program on the same platform with java -Dfile.encoding=UTF8, without the hypen, it yields:
file.encoding: UTF8
defaultCharset: UTF-8
... noting that the default charset has been canonicalized from UTF8 to UTF-8.
This is not a direct answer, but very useful if you don't have access to how java starts: you can set the environment variable JAVA_TOOLS_OPTIONS to -Dfile.encoding="UTF-8" and every time the jvm starts it will pick up that option.