EDIT: I'm going to leave this answer up as it will work, and it demonstrates a few calls that readers may not otherwise know about. However, Luke's answer is definitely nicer - go upvote it :)
You can do:
public static bool IsNullableEnum(this Type t)
{
return t.IsGenericType &&
t.GetGenericTypeDefinition() == typeof(Nullable<>) &&
t.GetGenericArguments()[0].IsEnum;
}