编译的 C # Lambda 表达式性能

考虑对集合进行以下简单操作:

static List<int> x = new List<int>() { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };
var result = x.Where(i => i % 2 == 0).Where(i => i > 5);

现在让我们使用表达式:

static void UsingLambda() {
Func<IEnumerable<int>, IEnumerable<int>> lambda = l => l.Where(i => i % 2 == 0).Where(i => i > 5);
var t0 = DateTime.Now.Ticks;
for (int j = 1; j < MAX; j++)
var sss = lambda(x).ToList();


var tn = DateTime.Now.Ticks;
Console.WriteLine("Using lambda: {0}", tn - t0);
}

但我想在运行中构建这个表达式,所以这里有一个新的测试:

static void UsingCompiledExpression() {
var f1 = (Expression<Func<IEnumerable<int>, IEnumerable<int>>>)(l => l.Where(i => i % 2 == 0));
var f2 = (Expression<Func<IEnumerable<int>, IEnumerable<int>>>)(l => l.Where(i => i > 5));
var argX = Expression.Parameter(typeof(IEnumerable<int>), "x");
var f3 = Expression.Invoke(f2, Expression.Invoke(f1, argX));
var f = Expression.Lambda<Func<IEnumerable<int>, IEnumerable<int>>>(f3, argX);


var c3 = f.Compile();


var t0 = DateTime.Now.Ticks;
for (int j = 1; j < MAX; j++)
var sss = c3(x).ToList();


var tn = DateTime.Now.Ticks;
Console.WriteLine("Using lambda compiled: {0}", tn - t0);
}

当然,它并不完全像上面那样,所以为了公平起见,我稍微修改了第一个:

static void UsingLambdaCombined() {
Func<IEnumerable<int>, IEnumerable<int>> f1 = l => l.Where(i => i % 2 == 0);
Func<IEnumerable<int>, IEnumerable<int>> f2 = l => l.Where(i => i > 5);
Func<IEnumerable<int>, IEnumerable<int>> lambdaCombined = l => f2(f1(l));
var t0 = DateTime.Now.Ticks;
for (int j = 1; j < MAX; j++)
var sss = lambdaCombined(x).ToList();


var tn = DateTime.Now.Ticks;
Console.WriteLine("Using lambda combined: {0}", tn - t0);
}

现在来看 MAX = 100000,VS2008,调试 ON 的结果:

Using lambda compiled: 23437500
Using lambda:           1250000
Using lambda combined:  1406250

关闭调试:

Using lambda compiled: 21718750
Using lambda:            937500
Using lambda combined:  1093750

令人惊讶的是,编译后的表达式大约比其他替代方法慢17倍,现在问题来了:

  1. 我是否在比较非等效表达式?
  2. 是否有一种机制使.NET“优化”已编译的表达式?
  3. 如何以编程方式表示相同的链调用 l.Where(i => i % 2 == 0).Where(i => i > 5);

更多统计数据。 VisualStudio2010,调试打开,优化关闭:

Using lambda:           1093974
Using lambda compiled: 15315636
Using lambda combined:   781410

调试,优化:

Using lambda:            781305
Using lambda compiled: 15469839
Using lambda combined:   468783

关闭调试,优化:

Using lambda:            625020
Using lambda compiled: 14687970
Using lambda combined:   468765

从 VS2008(C # 3)切换到 VS2010(C # 4) ,使得 UsingLambdaCombined比本地 lambda 更快。


好的,我已经找到了一种方法,可以将 lambda 编译的性能提高一个数量级以上。这里有一个提示; 在运行了分析器之后,92% 的时间都花在了:

System.Reflection.Emit.DynamicMethod.CreateDelegate(class System.Type, object)

嗯... ... 为什么它要在每次迭代中创建一个新的委托?我不确定,但是解决方案在另一篇文章中提出。

35762 次浏览

Compiled lambda performance over delegates may be slower because Compiled code at runtime may not be optimized however the code you wrote manually and that compiled via C# compiler is optimized.

Second, multiple lambda expressions means multiple anonymous methods, and calling each of them takes little extra time over evaluating a straight method. For example, calling

Console.WriteLine(x);

and

Action x => Console.WriteLine(x);
x(); // this means two different calls..

are different, and with second one little more overhead is required as from compiler's perspective, its actually two different calls. First calling x itself and then within that calling x's statement.

So your combined Lambda will certainly have little slow performance over single lambda expression.

And this is independent of what is executing inside, because you are still evaluating correct logic, but you are adding additional steps for compiler to perform.

Even after expression tree is compiled, it will not have optimization, and it will still preserve its little complex structure, evaluating and calling it may have extra validation, null check etc which might be slowing down performance of compiled lambda expressions.

Could it be that the inner lambdas are not being compiled?!? Here's a proof of concept:

static void UsingCompiledExpressionWithMethodCall() {
var where = typeof(Enumerable).GetMember("Where").First() as System.Reflection.MethodInfo;
where = where.MakeGenericMethod(typeof(int));
var l = Expression.Parameter(typeof(IEnumerable<int>), "l");
var arg0 = Expression.Parameter(typeof(int), "i");
var lambda0 = Expression.Lambda<Func<int, bool>>(
Expression.Equal(Expression.Modulo(arg0, Expression.Constant(2)),
Expression.Constant(0)), arg0).Compile();
var c1 = Expression.Call(where, l, Expression.Constant(lambda0));
var arg1 = Expression.Parameter(typeof(int), "i");
var lambda1 = Expression.Lambda<Func<int, bool>>(Expression.GreaterThan(arg1, Expression.Constant(5)), arg1).Compile();
var c2 = Expression.Call(where, c1, Expression.Constant(lambda1));


var f = Expression.Lambda<Func<IEnumerable<int>, IEnumerable<int>>>(c2, l);


var c3 = f.Compile();


var t0 = DateTime.Now.Ticks;
for (int j = 1; j < MAX; j++)
{
var sss = c3(x).ToList();
}


var tn = DateTime.Now.Ticks;
Console.WriteLine("Using lambda compiled with MethodCall: {0}", tn - t0);
}

And now the timings are:

Using lambda:                            625020
Using lambda compiled:                 14687970
Using lambda combined:                   468765
Using lambda compiled with MethodCall:   468765

Woot! Not only it is fast, it is faster than the native lambda. (Scratch head).


Of course the above code is simply too painful to write. Let's do some simple magic:

static void UsingCompiledConstantExpressions() {
var f1 = (Func<IEnumerable<int>, IEnumerable<int>>)(l => l.Where(i => i % 2 == 0));
var f2 = (Func<IEnumerable<int>, IEnumerable<int>>)(l => l.Where(i => i > 5));
var argX = Expression.Parameter(typeof(IEnumerable<int>), "x");
var f3 = Expression.Invoke(Expression.Constant(f2), Expression.Invoke(Expression.Constant(f1), argX));
var f = Expression.Lambda<Func<IEnumerable<int>, IEnumerable<int>>>(f3, argX);


var c3 = f.Compile();


var t0 = DateTime.Now.Ticks;
for (int j = 1; j < MAX; j++) {
var sss = c3(x).ToList();
}


var tn = DateTime.Now.Ticks;
Console.WriteLine("Using lambda compiled constant: {0}", tn - t0);
}

And some timings, VS2010, Optimizations ON, Debugging OFF:

Using lambda:                            781260
Using lambda compiled:                 14687970
Using lambda combined:                   468756
Using lambda compiled with MethodCall:   468756
Using lambda compiled constant:          468756

Now you could argue that I'm not generating the whole expression dynamically; just the chaining invocations. But in the above example I generate the whole expression. And the timings match. This is just a shortcut to write less code.


From my understanding, what is going on is that the .Compile() method does not propagate the compilations to inner lambdas, and thus the constant invocation of CreateDelegate. But to truly understand this, I would love to have a .NET guru comment a little about the internal stuff going on.

And why, oh why is this now faster than a native lambda!?

Recently I asked an almost identical question:

Performance of compiled-to-delegate Expression

The solution for me was that I shouldn't call Compile on the Expression, but that I should call CompileToMethod on it and compile the Expression to a static method in a dynamic assembly.

Like so:

var assemblyBuilder = AppDomain.CurrentDomain.DefineDynamicAssembly(
new AssemblyName("MyAssembly_" + Guid.NewGuid().ToString("N")),
AssemblyBuilderAccess.Run);


var moduleBuilder = assemblyBuilder.DefineDynamicModule("Module");


var typeBuilder = moduleBuilder.DefineType("MyType_" + Guid.NewGuid().ToString("N"),
TypeAttributes.Public));


var methodBuilder = typeBuilder.DefineMethod("MyMethod",
MethodAttributes.Public | MethodAttributes.Static);


expression.CompileToMethod(methodBuilder);


var resultingType = typeBuilder.CreateType();


var function = Delegate.CreateDelegate(expression.Type,
resultingType.GetMethod("MyMethod"));

It's not ideal however. I'm not quite certain to which types this applies exactly, but I think that types that are taken as parameters by the delegate, or returned by the delegate have to be public and non-generic. It has to be non-generic because generic types apparently access System.__Canon which is an internal type used by .NET under the hood for generic types and this violates the "has to be a public type rule).

For those types, you can use the apparently slower Compile. I detect them in the following way:

private static bool IsPublicType(Type t)
{


if ((!t.IsPublic && !t.IsNestedPublic) || t.IsGenericType)
{
return false;
}


int lastIndex = t.FullName.LastIndexOf('+');


if (lastIndex > 0)
{
var containgTypeName = t.FullName.Substring(0, lastIndex);


var containingType = Type.GetType(containgTypeName + "," + t.Assembly);


if (containingType != null)
{
return containingType.IsPublic;
}


return false;
}
else
{
return t.IsPublic;
}
}

But like I said, this isn't ideal and I would still like to know why compiling a method to a dynamic assembly is sometimes an order of magnitude faster. And I say sometimes because I've also seen cases where an Expression compiled with Compile is just as fast as a normal method. See my question for that.

Or if someone knows a way to bypass the "no non-public types" constraint with the dynamic assembly, that's welcome as well.

Your expressions are not equivalent and thus you get skewed results. I wrote a test bench to test this. The tests include the regular lambda call, the equivalent compiled expression, a hand made equivalent compiled expression, as well as composed versions. These should be more accurate numbers. Interestingly, I'm not seeing much variation between the plain and composed versions. And the compiled expressions are slower naturally but only by very little. You need a large enough input and iteration count to get some good numbers. It makes a difference.

As for your second question, I don't know how you'd be able to get more performance out of this so I can't help you there. It looks as good as it's going to get.

You'll find my answer to your third question in the HandMadeLambdaExpression() method. Not the easiest expression to build due to the extension methods, but doable.

using System;
using System.Collections.Generic;
using System.Linq;


using System.Diagnostics;
using System.Linq.Expressions;


namespace ExpressionBench
{
class Program
{
static void Main(string[] args)
{
var values = Enumerable.Range(0, 5000);
var lambda = GetLambda();
var lambdaExpression = GetLambdaExpression().Compile();
var handMadeLambdaExpression = GetHandMadeLambdaExpression().Compile();
var composed = GetComposed();
var composedExpression = GetComposedExpression().Compile();
var handMadeComposedExpression = GetHandMadeComposedExpression().Compile();


DoTest("Lambda", values, lambda);
DoTest("Lambda Expression", values, lambdaExpression);
DoTest("Hand Made Lambda Expression", values, handMadeLambdaExpression);
Console.WriteLine();
DoTest("Composed", values, composed);
DoTest("Composed Expression", values, composedExpression);
DoTest("Hand Made Composed Expression", values, handMadeComposedExpression);
}


static void DoTest<TInput, TOutput>(string name, TInput sequence, Func<TInput, TOutput> operation, int count = 1000000)
{
for (int _ = 0; _ < 1000; _++)
operation(sequence);
var sw = Stopwatch.StartNew();
for (int _ = 0; _ < count; _++)
operation(sequence);
sw.Stop();
Console.WriteLine("{0}:", name);
Console.WriteLine("  Elapsed: {0,10} {1,10} (ms)", sw.ElapsedTicks, sw.ElapsedMilliseconds);
Console.WriteLine("  Average: {0,10} {1,10} (ms)", decimal.Divide(sw.ElapsedTicks, count), decimal.Divide(sw.ElapsedMilliseconds, count));
}


static Func<IEnumerable<int>, IList<int>> GetLambda()
{
return v => v.Where(i => i % 2 == 0).Where(i => i > 5).ToList();
}


static Expression<Func<IEnumerable<int>, IList<int>>> GetLambdaExpression()
{
return v => v.Where(i => i % 2 == 0).Where(i => i > 5).ToList();
}


static Expression<Func<IEnumerable<int>, IList<int>>> GetHandMadeLambdaExpression()
{
var enumerableMethods = typeof(Enumerable).GetMethods();
var whereMethod = enumerableMethods
.Where(m => m.Name == "Where")
.Select(m => m.MakeGenericMethod(typeof(int)))
.Where(m => m.GetParameters()[1].ParameterType == typeof(Func<int, bool>))
.Single();
var toListMethod = enumerableMethods
.Where(m => m.Name == "ToList")
.Select(m => m.MakeGenericMethod(typeof(int)))
.Single();


// helpers to create the static method call expressions
Func<Expression, ParameterExpression, Func<ParameterExpression, Expression>, Expression> WhereExpression =
(instance, param, body) => Expression.Call(whereMethod, instance, Expression.Lambda(body(param), param));
Func<Expression, Expression> ToListExpression =
instance => Expression.Call(toListMethod, instance);


//return v => v.Where(i => i % 2 == 0).Where(i => i > 5).ToList();
var exprParam = Expression.Parameter(typeof(IEnumerable<int>), "v");
var expr0 = WhereExpression(exprParam,
Expression.Parameter(typeof(int), "i"),
i => Expression.Equal(Expression.Modulo(i, Expression.Constant(2)), Expression.Constant(0)));
var expr1 = WhereExpression(expr0,
Expression.Parameter(typeof(int), "i"),
i => Expression.GreaterThan(i, Expression.Constant(5)));
var exprBody = ToListExpression(expr1);
return Expression.Lambda<Func<IEnumerable<int>, IList<int>>>(exprBody, exprParam);
}


static Func<IEnumerable<int>, IList<int>> GetComposed()
{
Func<IEnumerable<int>, IEnumerable<int>> composed0 =
v => v.Where(i => i % 2 == 0);
Func<IEnumerable<int>, IEnumerable<int>> composed1 =
v => v.Where(i => i > 5);
Func<IEnumerable<int>, IList<int>> composed2 =
v => v.ToList();
return v => composed2(composed1(composed0(v)));
}


static Expression<Func<IEnumerable<int>, IList<int>>> GetComposedExpression()
{
Expression<Func<IEnumerable<int>, IEnumerable<int>>> composed0 =
v => v.Where(i => i % 2 == 0);
Expression<Func<IEnumerable<int>, IEnumerable<int>>> composed1 =
v => v.Where(i => i > 5);
Expression<Func<IEnumerable<int>, IList<int>>> composed2 =
v => v.ToList();
var exprParam = Expression.Parameter(typeof(IEnumerable<int>), "v");
var exprBody = Expression.Invoke(composed2, Expression.Invoke(composed1, Expression.Invoke(composed0, exprParam)));
return Expression.Lambda<Func<IEnumerable<int>, IList<int>>>(exprBody, exprParam);
}


static Expression<Func<IEnumerable<int>, IList<int>>> GetHandMadeComposedExpression()
{
var enumerableMethods = typeof(Enumerable).GetMethods();
var whereMethod = enumerableMethods
.Where(m => m.Name == "Where")
.Select(m => m.MakeGenericMethod(typeof(int)))
.Where(m => m.GetParameters()[1].ParameterType == typeof(Func<int, bool>))
.Single();
var toListMethod = enumerableMethods
.Where(m => m.Name == "ToList")
.Select(m => m.MakeGenericMethod(typeof(int)))
.Single();


Func<ParameterExpression, Func<ParameterExpression, Expression>, Expression> LambdaExpression =
(param, body) => Expression.Lambda(body(param), param);
Func<Expression, ParameterExpression, Func<ParameterExpression, Expression>, Expression> WhereExpression =
(instance, param, body) => Expression.Call(whereMethod, instance, Expression.Lambda(body(param), param));
Func<Expression, Expression> ToListExpression =
instance => Expression.Call(toListMethod, instance);


var composed0 = LambdaExpression(Expression.Parameter(typeof(IEnumerable<int>), "v"),
v => WhereExpression(
v,
Expression.Parameter(typeof(int), "i"),
i => Expression.Equal(Expression.Modulo(i, Expression.Constant(2)), Expression.Constant(0))));
var composed1 = LambdaExpression(Expression.Parameter(typeof(IEnumerable<int>), "v"),
v => WhereExpression(
v,
Expression.Parameter(typeof(int), "i"),
i => Expression.GreaterThan(i, Expression.Constant(5))));
var composed2 = LambdaExpression(Expression.Parameter(typeof(IEnumerable<int>), "v"),
v => ToListExpression(v));


var exprParam = Expression.Parameter(typeof(IEnumerable<int>), "v");
var exprBody = Expression.Invoke(composed2, Expression.Invoke(composed1, Expression.Invoke(composed0, exprParam)));
return Expression.Lambda<Func<IEnumerable<int>, IList<int>>>(exprBody, exprParam);
}
}
}

And the results on my machine:

Lambda:
Elapsed:  340971948     123230 (ms)
Average: 340.971948    0.12323 (ms)
Lambda Expression:
Elapsed:  357077202     129051 (ms)
Average: 357.077202   0.129051 (ms)
Hand Made Lambda Expression:
Elapsed:  345029281     124696 (ms)
Average: 345.029281   0.124696 (ms)


Composed:
Elapsed:  340409238     123027 (ms)
Average: 340.409238   0.123027 (ms)
Composed Expression:
Elapsed:  350800599     126782 (ms)
Average: 350.800599   0.126782 (ms)
Hand Made Composed Expression:
Elapsed:  352811359     127509 (ms)
Average: 352.811359   0.127509 (ms)