You can change the level to OFF which should get rid of all logging. According to the log4j website, valid levels in order of importance are TRACE, DEBUG, INFO, WARN, ERROR, FATAL. There is one undocumented level called OFF which is a higher level than FATAL, and turns off all logging.
You can also create an extra root logger to log nothing (level OFF), so that you can switch root loggers easily. Here's a post to get you started on that.
You might also want to read the Log4J FAQ, because I think turning off all logging may not help. It will certainly not speed up your app that much, because logging code is executed anyway, up to the point where log4j decides that it doesn't need to log this entry.