Hit ? and it should show you the command shortcuts for doing page up/page down etc.
By default git looks at the $GIT_PAGER, then $PAGER environment variable to determine the program to be used for showing you the output of diff, log, show etc.
On new (linux) systems the default $PAGER used (even when it is not set) is less (it used to be the less capable more).
In both less and more the ? key gives you a command shortcut list.
I'm running what I think is a stock git install on Windows, msysgit 1.8.3, and for me git diff runs something that claims to be less, version 444.
Instead of '?' for help, it wants 'h'.
Seems that '?' is the start of some kind of regular expression search.