在 LINQ 中创建批处理

有人能建议一种方法来创建一定大小的批在 LINQ？

理想情况下，我希望能够在一定数量的可配置块中执行操作。

92059 次浏览
小开

您不需要编写任何代码。使用 莫林克批处理方法，将源序列批处理成大小不同的存储桶(MoreLINQ 可以作为 NuGet 包安装) :

int size = 10;
var batches = sequence.Batch(size);

实施方式如下:

public static IEnumerable<IEnumerable<TSource>> Batch<TSource>(
this IEnumerable<TSource> source, int size)
{
TSource[] bucket = null;
var count = 0;


foreach (var item in source)
{
if (bucket == null)
bucket = new TSource[size];


bucket[count++] = item;
if (count != size)
continue;


yield return bucket;


bucket = null;
count = 0;
}


if (bucket != null && count > 0)
yield return bucket.Take(count).ToArray();
}
小开 
public static class MyExtensions
{
public static IEnumerable<IEnumerable<T>> Batch<T>(this IEnumerable<T> items,
int maxItems)
{
return items.Select((item, inx) => new { item, inx })
.GroupBy(x => x.inx / maxItems)
.Select(g => g.Select(x => x.item));
}
}

用途如下:

List<int> list = new List<int>() { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 };


foreach(var batch in list.Batch(3))
{
Console.WriteLine(String.Join(",",batch));
}

产出:

0,1,2
3,4,5
6,7,8
9
小开

所有这些在大批处理或低内存空间的情况下性能都很糟糕。不得不自己编写那个将流水线(注意任何地方都没有项目积累) :

public static class BatchLinq {
public static IEnumerable<IEnumerable<T>> Batch<T>(this IEnumerable<T> source, int size) {
if (size <= 0)
throw new ArgumentOutOfRangeException("size", "Must be greater than zero.");


using (IEnumerator<T> enumerator = source.GetEnumerator())
while (enumerator.MoveNext())
yield return TakeIEnumerator(enumerator, size);
}


private static IEnumerable<T> TakeIEnumerator<T>(IEnumerator<T> source, int size) {
int i = 0;
do
yield return source.Current;
while (++i < size && source.MoveNext());
}
}

编辑: 此方法的已知问题是，在转移到下一批之前，必须对每个批进行枚举和完全枚举。例如，这种方法行不通:

//Select first item of every 100 items
Batch(list, 100).Select(b => b.First())
小开 
    static IEnumerable<IEnumerable<T>> TakeBatch<T>(IEnumerable<T> ts,int batchSize)
{
return from @group in ts.Select((x, i) => new { x, i }).ToLookup(xi => xi.i / batchSize)
select @group.Select(xi => xi.x);
}
小开

我很晚才加入，但我发现了更有趣的事。

所以我们可以在这里使用 Skip和 Take来获得更好的性能。

public static class MyExtensions
{
public static IEnumerable<IEnumerable<T>> Batch<T>(this IEnumerable<T> items, int maxItems)
{
return items.Select((item, index) => new { item, index })
.GroupBy(x => x.index / maxItems)
.Select(g => g.Select(x => x.item));
}


public static IEnumerable<T> Batch2<T>(this IEnumerable<T> items, int skip, int take)
{
return items.Skip(skip).Take(take);
}


}

接下来我检查了100000条记录。只是循环在 Batch的情况下需要更多的时间

控制台应用。

static void Main(string[] args)
{
List<string> Ids = GetData("First");
List<string> Ids2 = GetData("tsriF");


Stopwatch FirstWatch = new Stopwatch();
FirstWatch.Start();
foreach (var batch in Ids2.Batch(5000))
{
// Console.WriteLine("Batch Ouput:= " + string.Join(",", batch));
}
FirstWatch.Stop();
Console.WriteLine("Done Processing time taken:= "+ FirstWatch.Elapsed.ToString());




Stopwatch Second = new Stopwatch();


Second.Start();
int Length = Ids2.Count;
int StartIndex = 0;
int BatchSize = 5000;
while (Length > 0)
{
var SecBatch = Ids2.Batch2(StartIndex, BatchSize);
// Console.WriteLine("Second Batch Ouput:= " + string.Join(",", SecBatch));
Length = Length - BatchSize;
StartIndex += BatchSize;
}


Second.Stop();
Console.WriteLine("Done Processing time taken Second:= " + Second.Elapsed.ToString());
Console.ReadKey();
}


static List<string> GetData(string name)
{
List<string> Data = new List<string>();
for (int i = 0; i < 100000; i++)
{
Data.Add(string.Format("{0} {1}", name, i.ToString()));
}


return Data;
}

花费的时间是这样的。

首先-00.00.00.0708,00.00.00.0660

第二部分(采用和跳过第一部分)-00.00.00.0008,00.00.00.0008

小开

与 MoreLINQ 相同的方法，但是使用 List 而不是 Array。我还没有做过基准测试，但可读性对某些人来说更重要:

    public static IEnumerable<IEnumerable<T>> Batch<T>(this IEnumerable<T> source, int size)
{
List<T> batch = new List<T>();


foreach (var item in source)
{
batch.Add(item);


if (batch.Count >= size)
{
yield return batch;
batch.Clear();
}
}


if (batch.Count > 0)
{
yield return batch;
}
}
小开

如果您从定义为 IEnumerable<T>的 sequence开始，并且您知道它可以安全地枚举多次(例如，因为它是一个数组或列表) ，那么您可以使用这个简单的模式来批处理元素:

while (sequence.Any())
{
var batch = sequence.Take(10);
sequence = sequence.Skip(10);


// do whatever you need to do with each batch here
}
小开

这是一个完全懒惰的、低开销的 Batch 的单函数实现，它不进行任何累积。在 EricRoller 的帮助下，根据(并修复) Nick Whaley 的 解决方案。

迭代直接来自底层 IEnumable，因此必须严格按顺序枚举元素，并且不能多次访问。如果某些元素没有在内部循环中使用，它们将被丢弃(如果尝试通过保存的迭代器再次访问它们，将抛出 InvalidOperationException: Enumeration already finished.)。

您可以在 .NET 小提琴测试一个完整的样品。

public static class BatchLinq
{
public static IEnumerable<IEnumerable<T>> Batch<T>(this IEnumerable<T> source, int size)
{
if (size <= 0)
throw new ArgumentOutOfRangeException("size", "Must be greater than zero.");
using (var enumerator = source.GetEnumerator())
while (enumerator.MoveNext())
{
int i = 0;
// Batch is a local function closing over `i` and `enumerator` that
// executes the inner batch enumeration
IEnumerable<T> Batch()
{
do yield return enumerator.Current;
while (++i < size && enumerator.MoveNext());
}


yield return Batch();
while (++i < size && enumerator.MoveNext()); // discard skipped items
}
}
}
小开

我编写了一个自定义 IEnumable 实现，它不使用 linq，并保证对数据进行一次枚举。它还实现了所有这些，而不需要支持列表或数组，这些列表或数组会导致大型数据集上的内存爆炸。

下面是一些基本的测试:

    [Fact]
public void ShouldPartition()
{
var ints = new List<int> {0, 1, 2, 3, 4, 5, 6, 7, 8, 9};
var data = ints.PartitionByMaxGroupSize(3);
data.Count().Should().Be(4);


data.Skip(0).First().Count().Should().Be(3);
data.Skip(0).First().ToList()[0].Should().Be(0);
data.Skip(0).First().ToList()[1].Should().Be(1);
data.Skip(0).First().ToList()[2].Should().Be(2);


data.Skip(1).First().Count().Should().Be(3);
data.Skip(1).First().ToList()[0].Should().Be(3);
data.Skip(1).First().ToList()[1].Should().Be(4);
data.Skip(1).First().ToList()[2].Should().Be(5);


data.Skip(2).First().Count().Should().Be(3);
data.Skip(2).First().ToList()[0].Should().Be(6);
data.Skip(2).First().ToList()[1].Should().Be(7);
data.Skip(2).First().ToList()[2].Should().Be(8);


data.Skip(3).First().Count().Should().Be(1);
data.Skip(3).First().ToList()[0].Should().Be(9);
}

分区数据的扩展方法。

/// <summary>
/// A set of extension methods for <see cref="IEnumerable{T}"/>.
/// </summary>
public static class EnumerableExtender
{
/// <summary>
/// Splits an enumerable into chucks, by a maximum group size.
/// </summary>
/// <param name="source">The source to split</param>
/// <param name="maxSize">The maximum number of items per group.</param>
/// <typeparam name="T">The type of item to split</typeparam>
/// <returns>A list of lists of the original items.</returns>
public static IEnumerable<IEnumerable<T>> PartitionByMaxGroupSize<T>(this IEnumerable<T> source, int maxSize)
{
return new SplittingEnumerable<T>(source, maxSize);
}
}

这是实现类

    using System.Collections;
using System.Collections.Generic;


internal class SplittingEnumerable<T> : IEnumerable<IEnumerable<T>>
{
private readonly IEnumerable<T> backing;
private readonly int maxSize;
private bool hasCurrent;
private T lastItem;


public SplittingEnumerable(IEnumerable<T> backing, int maxSize)
{
this.backing = backing;
this.maxSize = maxSize;
}


public IEnumerator<IEnumerable<T>> GetEnumerator()
{
return new Enumerator(this, this.backing.GetEnumerator());
}


IEnumerator IEnumerable.GetEnumerator()
{
return this.GetEnumerator();
}


private class Enumerator : IEnumerator<IEnumerable<T>>
{
private readonly SplittingEnumerable<T> parent;
private readonly IEnumerator<T> backingEnumerator;
private NextEnumerable current;


public Enumerator(SplittingEnumerable<T> parent, IEnumerator<T> backingEnumerator)
{
this.parent = parent;
this.backingEnumerator = backingEnumerator;
this.parent.hasCurrent = this.backingEnumerator.MoveNext();
if (this.parent.hasCurrent)
{
this.parent.lastItem = this.backingEnumerator.Current;
}
}


public bool MoveNext()
{
if (this.current == null)
{
this.current = new NextEnumerable(this.parent, this.backingEnumerator);
return true;
}
else
{
if (!this.current.IsComplete)
{
using (var enumerator = this.current.GetEnumerator())
{
while (enumerator.MoveNext())
{
}
}
}
}


if (!this.parent.hasCurrent)
{
return false;
}


this.current = new NextEnumerable(this.parent, this.backingEnumerator);
return true;
}


public void Reset()
{
throw new System.NotImplementedException();
}


public IEnumerable<T> Current
{
get { return this.current; }
}


object IEnumerator.Current
{
get { return this.Current; }
}


public void Dispose()
{
}
}


private class NextEnumerable : IEnumerable<T>
{
private readonly SplittingEnumerable<T> splitter;
private readonly IEnumerator<T> backingEnumerator;
private int currentSize;


public NextEnumerable(SplittingEnumerable<T> splitter, IEnumerator<T> backingEnumerator)
{
this.splitter = splitter;
this.backingEnumerator = backingEnumerator;
}


public bool IsComplete { get; private set; }


public IEnumerator<T> GetEnumerator()
{
return new NextEnumerator(this.splitter, this, this.backingEnumerator);
}


IEnumerator IEnumerable.GetEnumerator()
{
return this.GetEnumerator();
}


private class NextEnumerator : IEnumerator<T>
{
private readonly SplittingEnumerable<T> splitter;
private readonly NextEnumerable parent;
private readonly IEnumerator<T> enumerator;
private T currentItem;


public NextEnumerator(SplittingEnumerable<T> splitter, NextEnumerable parent, IEnumerator<T> enumerator)
{
this.splitter = splitter;
this.parent = parent;
this.enumerator = enumerator;
}


public bool MoveNext()
{
this.parent.currentSize += 1;
this.currentItem = this.splitter.lastItem;
var hasCcurent = this.splitter.hasCurrent;


this.parent.IsComplete = this.parent.currentSize > this.splitter.maxSize;


if (this.parent.IsComplete)
{
return false;
}


if (hasCcurent)
{
var result = this.enumerator.MoveNext();


this.splitter.lastItem = this.enumerator.Current;
this.splitter.hasCurrent = result;
}


return hasCcurent;
}


public void Reset()
{
throw new System.NotImplementedException();
}


public T Current
{
get { return this.currentItem; }
}


object IEnumerator.Current
{
get { return this.Current; }
}


public void Dispose()
{
}
}
}
}
小开

因此，如果加上一顶功能帽子，这看起来微不足道... ... 但是在 C # 中，有一些显著的缺点。

你可能会把它看作是 IEnumable 的展开(在谷歌上搜索它，你可能会在一些 Haskell 文档中找到它，但是可能有一些 F # 的东西在使用 un摺，如果你知道 F # ，眯着眼睛看 Haskell 文档，它就会有意义)。

Unold 与摺(“聚合”)相关，除了不是通过输入 IEnumable 迭代，而是通过输出数据结构迭代(IEnumable 和 IObable 之间的类似关系，实际上我认为 IObable 确实实现了一个名为 generate... 的“ unold”)

无论如何 首先，你需要一个展开的方法，我认为这工程(不幸的是，它最终会爆炸堆栈的大“列表”... 你可以写这在 F # 使用产量安全！而不是连接) ;

    static IEnumerable<T> Unfold<T, U>(Func<U, IEnumerable<Tuple<U, T>>> f, U seed)
{
var maybeNewSeedAndElement = f(seed);


return maybeNewSeedAndElement.SelectMany(x => new[] { x.Item2 }.Concat(Unfold(f, x.Item1)));
}

这有点迟钝，因为 C # 没有实现函数式语言认为理所当然的一些东西... ... 但它基本上是取一个种子，然后生成 IEnumable 中下一个元素和下一个种子的“可能”答案(可能在 C # 中不存在，所以我们使用 IEnumable 来伪造它) ，并连接其余的答案(我不能保证“ O (n?)”复杂性)。

一旦你做到了这一点，然后;

    static IEnumerable<IEnumerable<T>> Batch<T>(IEnumerable<T> xs, int n)
{
return Unfold(ys =>
{
var head = ys.Take(n);
var tail = ys.Skip(n);
return head.Take(1).Select(_ => Tuple.Create(tail, head));
},
xs);
}

它看起来非常干净... ... 您将“ n”元素作为 IEnumable 中的“ next”元素，而“ tail”是未处理列表的其余部分。

如果 head 中没有任何东西... ... 那么您就结束了... ... 您返回“ Nothing”(但是伪造为一个空的 IEnumable >) ... ... 否则您返回 head 元素和 tail 来处理。

你也许可以使用 IObserver 来实现这一点，可能已经有一个类似于“ Batch”的方法，你也许可以使用它。

如果担心堆栈溢出的风险(可能应该) ，那么应该在 F # 中实现(可能还有一些 F # 库(FSharpX?)已经有了这个)。

(我只做了一些基本的测试，所以可能会有一些奇怪的 bug)。

小开

我知道每个人都使用复杂的系统来做这项工作，我真的不明白为什么。 Take 和 Skip 将允许所有这些操作使用 Func<TSource,Int32,TResult>转换函数的公共选择。 比如:

public IEnumerable<IEnumerable<T>> Buffer<T>(IEnumerable<T> source, int size)=>
source.Select((item, index) => source.Skip(size * index).Take(size)).TakeWhile(bucket => bucket.Any());
小开

另一种方法是使用 处方 缓冲操作员

//using System.Linq;
//using System.Reactive.Linq;
//using System.Reactive.Threading.Tasks;


var observableBatches = anAnumerable.ToObservable().Buffer(size);


var batches = aList.ToObservable().Buffer(size).ToList().ToTask().GetAwaiter().GetResult();
小开

只是另一行实现。它甚至可以在空列表中工作，在这种情况下，您将得到一个零大小的批处理集合。

var aList = Enumerable.Range(1, 100).ToList(); //a given list
var size = 9; //the wanted batch size
//number of batches are: (aList.Count() + size - 1) / size;


var batches = Enumerable.Range(0, (aList.Count() + size - 1) / size).Select(i => aList.GetRange( i * size, Math.Min(size, aList.Count() - i * size)));


Assert.True(batches.Count() == 12);
Assert.AreEqual(batches.ToList().ElementAt(0), new List<int>() { 1, 2, 3, 4, 5, 6, 7, 8, 9 });
Assert.AreEqual(batches.ToList().ElementAt(1), new List<int>() { 10, 11, 12, 13, 14, 15, 16, 17, 18 });
Assert.AreEqual(batches.ToList().ElementAt(11), new List<int>() { 100 });
小开

我想知道为什么从来没有人发布过一个老式的循环解决方案。这里有一个:

List<int> source = Enumerable.Range(1,23).ToList();
int batchsize = 10;
for (int i = 0; i < source.Count; i+= batchsize)
{
var batch = source.Skip(i).Take(batchsize);
}

这种简单性是可能的，因为 Take 方法:

... 枚举 source并生成元素，直到生成了 count元素或者 source不再包含更多元素。如果 count超过 source中的元素数，则返回 source的所有元素

免责声明:

在循环中使用 Skip 和 Take 意味着枚举数将被枚举多次。如果延迟枚举，这是危险的。它可能导致数据库查询、 Web 请求或文件读取的多次执行。这个示例显式地用于 List，因为 List 不会被延迟，所以问题不大。 这仍然是一个缓慢的解决方案，因为每次调用时跳过都会枚举集合。

这也可以用 GetRange方法解决，但是需要额外的计算来提取一个可能的休息批:

for (int i = 0; i < source.Count; i += batchsize)
{
int remaining = source.Count - i;
var batch = remaining > batchsize  ? source.GetRange(i, batchsize) : source.GetRange(i, remaining);
}

这里有第三种方法来处理这个问题，它适用于2个循环。这样可以确保集合只枚举1次！:

int batchsize = 10;
List<int> batch = new List<int>(batchsize);


for (int i = 0; i < source.Count; i += batchsize)
{
// calculated the remaining items to avoid an OutOfRangeException
batchsize = source.Count - i > batchsize ? batchsize : source.Count - i;
for (int j = i; j < i + batchsize; j++)
{
batch.Add(source[j]);
}
batch.Clear();
}
小开

下面是对 Nick Whaley 的(链接)和 infogulch 的(链接)惰性 Batch实现的一个尝试性改进。这个很严格。您要么按正确的顺序枚举批处理，要么得到一个异常。

public static IEnumerable<IEnumerable<TSource>> Batch<TSource>(
this IEnumerable<TSource> source, int size)
{
if (size <= 0) throw new ArgumentOutOfRangeException(nameof(size));
using (var enumerator = source.GetEnumerator())
{
int i = 0;
while (enumerator.MoveNext())
{
if (i % size != 0) throw new InvalidOperationException(
"The enumeration is out of order.");
i++;
yield return GetBatch();
}
IEnumerable<TSource> GetBatch()
{
while (true)
{
yield return enumerator.Current;
if (i % size == 0 || !enumerator.MoveNext()) break;
i++;
}
}
}
}

下面是 IList<T>类型源的延迟 Batch实现。这个函数不对枚举施加任何限制。这些批可以按任意顺序进行部分枚举，也可以多次枚举。不过，在枚举期间不修改集合的限制仍然存在。这是通过在生成任何块或元素之前对 enumerator.MoveNext()进行虚拟调用来实现的。不足之处在于枚举数没有处置，因为它不知道枚举什么时候结束。

public static IEnumerable<IEnumerable<TSource>> Batch<TSource>(
this IList<TSource> source, int size)
{
if (size <= 0) throw new ArgumentOutOfRangeException(nameof(size));
var enumerator = source.GetEnumerator();
for (int i = 0; i < source.Count; i += size)
{
enumerator.MoveNext();
yield return GetChunk(i, Math.Min(i + size, source.Count));
}
IEnumerable<TSource> GetChunk(int from, int toExclusive)
{
for (int j = from; j < toExclusive; j++)
{
enumerator.MoveNext();
yield return source[j];
}
}
}
小开

一个易于使用和理解的版本。

    public static List<List<T>> chunkList<T>(List<T> listToChunk, int batchSize)
{
List<List<T>> batches = new List<List<T>>();


if (listToChunk.Count == 0) return batches;


bool moreRecords = true;
int fromRecord = 0;
int countRange = 0;
if (listToChunk.Count >= batchSize)
{
countRange = batchSize;
}
else
{
countRange = listToChunk.Count;
}


while (moreRecords)
{
List<T> batch = listToChunk.GetRange(fromRecord, countRange);
batches.Add(batch);


if ((fromRecord + batchSize) >= listToChunk.Count)
{
moreRecords = false;
}


fromRecord = fromRecord + batch.Count;


if ((fromRecord + batchSize) > listToChunk.Count)
{
countRange = listToChunk.Count - fromRecord;
}
else
{
countRange = batchSize;
}
}
return batches;
}
小开
最佳答案

向.NET 6.0添加了一个 Enumerable.Chunk()扩展方法。

例如:

var list = new List<int> { 1, 2, 3, 4, 5, 6, 7 };


var chunks = list.Chunk(3);
// returns { { 1, 2, 3 }, { 4, 5, 6 }, { 7 } }

对于那些不能升级的，源代码可以在 GitHub 上找到。

小开

作为.NET 6中 LINQ 的一个新的助手方法，您可以将任何 IEnumable 分块到批处理中:

int chunkNumber = 1;
foreach (int[] chunk in Enumerable.Range(0, 9).Chunk(3))
{
Console.WriteLine($"Chunk {chunkNumber++}");
foreach (var item in chunk)
{
Console.WriteLine(item);
}
}
小开

这是我能想到的最干净的 Batch版本:

public static IEnumerable<IEnumerable<T>> Batch<T>(this IEnumerable<T> source, int count)
{
if (source == null) throw new System.ArgumentNullException("source");
if (count <= 0) throw new System.ArgumentOutOfRangeException("count");
using (var enumerator = source.GetEnumerator())
{
IEnumerable<T> BatchInner()
{
int counter = 0;
do
yield return enumerator.Current;
while (++counter < count && enumerator.MoveNext());
}
while (enumerator.MoveNext())
yield return BatchInner().ToArray();
}
}

使用以下代码:

Console.WriteLine(String.Join(Environment.NewLine,
Enumerable.Range(0, 20).Batch(8).Select(xs => String.Join(",", xs))));

我得到了:

0,1,2,3,4,5,6,7
8,9,10,11,12,13,14,15
16,17,18,19

值得注意的是，在“ &”的回答中，这段代码失败了:

var e = Enumerable.Range(0, 20).Batch(8).ToArray();


Console.WriteLine(String.Join(Environment.NewLine, e.Select(xs => String.Join(",", xs))));
Console.WriteLine();
Console.WriteLine(String.Join(Environment.NewLine, e.Select(xs => String.Join(",", xs))));

他们的回答是:

19
19
19


19
19
19

由于内部枚举不是作为数组计算的。

小开

执行批处理的另一种方法:

public static class Extensions
{
public static IEnumerable<TOut> Batch<T, TOut>(this IEnumerable<T> source, Func<T, T, TOut> func)
{
using (var enumerator = source.GetEnumerator())
{
while (true)
{
bool state;


state = enumerator.MoveNext(); if (!state) break; var v0 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v1 = enumerator.Current;


yield return func(v0, v1);
}
}
}


public static IEnumerable<TOut> Batch<T, TOut>(this IEnumerable<T> source, Func<T, T, T, TOut> func)
{
using (var enumerator = source.GetEnumerator())
{
while (true)
{
bool state;


state = enumerator.MoveNext(); if (!state) break; var v0 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v1 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v2 = enumerator.Current;


yield return func(v0, v1, v2);
}
}
}


public static IEnumerable<TOut> Batch<T, TOut>(this IEnumerable<T> source, Func<T, T, T, T, TOut> func)
{
using (var enumerator = source.GetEnumerator())
{
while (true)
{
bool state;


state = enumerator.MoveNext(); if (!state) break; var v0 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v1 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v2 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v3 = enumerator.Current;


yield return func(v0, v1, v2, v3);
}
}
}


public static IEnumerable<TOut> Batch<T, TOut>(this IEnumerable<T> source, Func<T, T, T, T, T, TOut> func)
{
using (var enumerator = source.GetEnumerator())
{
while (true)
{
bool state;


state = enumerator.MoveNext(); if (!state) break; var v0 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v1 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v2 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v3 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v4 = enumerator.Current;


yield return func(v0, v1, v2, v3, v4);
}
}
}


public static IEnumerable<TOut> Batch<T, TOut>(this IEnumerable<T> source, Func<T, T, T, T, T, T, TOut> func)
{
using (var enumerator = source.GetEnumerator())
{
while (true)
{
bool state;


state = enumerator.MoveNext(); if (!state) break; var v0 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v1 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v2 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v3 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v4 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v5 = enumerator.Current;


yield return func(v0, v1, v2, v3, v4, v5);
}
}
}


public static IEnumerable<TOut> Batch<T, TOut>(this IEnumerable<T> source, Func<T, T, T, T, T, T, T, TOut> func)
{
using (var enumerator = source.GetEnumerator())
{
while (true)
{
bool state;


state = enumerator.MoveNext(); if (!state) break; var v0 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v1 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v2 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v3 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v4 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v5 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v6 = enumerator.Current;


yield return func(v0, v1, v2, v3, v4, v5, v6);
}
}
}


public static IEnumerable<TOut> Batch<T, TOut>(this IEnumerable<T> source, Func<T, T, T, T, T, T, T, T, TOut> func)
{
using (var enumerator = source.GetEnumerator())
{
while (true)
{
bool state;


state = enumerator.MoveNext(); if (!state) break; var v0 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v1 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v2 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v3 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v4 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v5 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v6 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v7 = enumerator.Current;


yield return func(v0, v1, v2, v3, v4, v5, v6, v7);
}
}
}


public static IEnumerable<TOut> Batch<T, TOut>(this IEnumerable<T> source, Func<T, T, T, T, T, T, T, T, T, TOut> func)
{
using (var enumerator = source.GetEnumerator())
{
while (true)
{
bool state;


state = enumerator.MoveNext(); if (!state) break; var v0 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v1 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v2 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v3 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v4 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v5 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v6 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v7 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v8 = enumerator.Current;


yield return func(v0, v1, v2, v3, v4, v5, v6, v7, v8);
}
}
}


public static IEnumerable<TOut> Batch<T, TOut>(this IEnumerable<T> source, Func<T, T, T, T, T, T, T, T, T, T, TOut> func)
{
using (var enumerator = source.GetEnumerator())
{
while (true)
{
bool state;


state = enumerator.MoveNext(); if (!state) break; var v0 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v1 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v2 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v3 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v4 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v5 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v6 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v7 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v8 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v9 = enumerator.Current;


yield return func(v0, v1, v2, v3, v4, v5, v6, v7, v8, v9);
}
}
}


public static IEnumerable<TOut> Batch<T, TOut>(this IEnumerable<T> source, Func<T, T, T, T, T, T, T, T, T, T, T, TOut> func)
{
using (var enumerator = source.GetEnumerator())
{
while (true)
{
bool state;


state = enumerator.MoveNext(); if (!state) break; var v0 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v1 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v2 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v3 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v4 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v5 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v6 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v7 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v8 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v9 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v10 = enumerator.Current;


yield return func(v0, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10);
}
}
}


public static IEnumerable<TOut> Batch<T, TOut>(this IEnumerable<T> source, Func<T, T, T, T, T, T, T, T, T, T, T, T, TOut> func)
{
using (var enumerator = source.GetEnumerator())
{
while (true)
{
bool state;


state = enumerator.MoveNext(); if (!state) break; var v0 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v1 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v2 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v3 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v4 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v5 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v6 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v7 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v8 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v9 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v10 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v11 = enumerator.Current;


yield return func(v0, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11);
}
}
}


public static IEnumerable<TOut> Batch<T, TOut>(this IEnumerable<T> source, Func<T, T, T, T, T, T, T, T, T, T, T, T, T, TOut> func)
{
using (var enumerator = source.GetEnumerator())
{
while (true)
{
bool state;


state = enumerator.MoveNext(); if (!state) break; var v0 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v1 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v2 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v3 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v4 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v5 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v6 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v7 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v8 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v9 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v10 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v11 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v12 = enumerator.Current;


yield return func(v0, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12);
}
}
}


public static IEnumerable<TOut> Batch<T, TOut>(this IEnumerable<T> source, Func<T, T, T, T, T, T, T, T, T, T, T, T, T, T, TOut> func)
{
using (var enumerator = source.GetEnumerator())
{
while (true)
{
bool state;


state = enumerator.MoveNext(); if (!state) break; var v0 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v1 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v2 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v3 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v4 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v5 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v6 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v7 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v8 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v9 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v10 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v11 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v12 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v13 = enumerator.Current;


yield return func(v0, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13);
}
}
}


public static IEnumerable<TOut> Batch<T, TOut>(this IEnumerable<T> source, Func<T, T, T, T, T, T, T, T, T, T, T, T, T, T, T, TOut> func)
{
using (var enumerator = source.GetEnumerator())
{
while (true)
{
bool state;


state = enumerator.MoveNext(); if (!state) break; var v0 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v1 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v2 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v3 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v4 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v5 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v6 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v7 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v8 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v9 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v10 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v11 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v12 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v13 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v14 = enumerator.Current;


yield return func(v0, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14);
}
}
}


public static IEnumerable<TOut> Batch<T, TOut>(this IEnumerable<T> source, Func<T, T, T, T, T, T, T, T, T, T, T, T, T, T, T, T, TOut> func)
{
using (var enumerator = source.GetEnumerator())
{
while (true)
{
bool state;


state = enumerator.MoveNext(); if (!state) break; var v0 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v1 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v2 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v3 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v4 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v5 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v6 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v7 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v8 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v9 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v10 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v11 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v12 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v13 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v14 = enumerator.Current;
state = enumerator.MoveNext(); if (!state) break; var v15 = enumerator.Current;


yield return func(v0, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15);
}
}
}
}

下面是一个用法示例:

using System;
using System.Linq;




namespace TestProgram
{
class Program
{
static void Main(string[] args)
{
foreach (var item in Enumerable.Range(0, 12).ToArray().Batch((R, X1, Y1, X2, Y2) => (R, X1, Y1, X2, Y2)))
{
Console.WriteLine($"{item.R}, {item.X1}, {item.Y1}, {item.X2}, {item.Y2}");
}
}
}
}
小开

下面是一个通过 IAsyncEnumerable-https://learn.microsoft.com/en-us/dotnet/csharp/whats-new/tutorials/generate-consume-asynchronous-stream在 C # 中使用 Async 迭代的实现

public static class EnumerableExtensions
{
/// <summary>
/// Chunks a sequence into a sub-sequences each containing maxItemsPerChunk, except for the last
/// which will contain any items left over.
///
/// NOTE: this implements a streaming implementation via <seealso cref="IAsyncEnumerable{T}"/>.
/// </summary>
public static async IAsyncEnumerable<IEnumerable<T>> ChunkAsync<T>(this IAsyncEnumerable<T> sequence, int maxItemsPerChunk)
{
if (sequence == null) throw new ArgumentNullException(nameof(sequence));
if (maxItemsPerChunk <= 0)
{
throw new ArgumentOutOfRangeException(nameof(maxItemsPerChunk), $"{nameof(maxItemsPerChunk)} must be greater than 0");
}


var chunk = new List<T>(maxItemsPerChunk);
await foreach (var item in sequence)
{
chunk.Add(item);


if (chunk.Count == maxItemsPerChunk)
{
yield return chunk.ToArray();
chunk.Clear();
}
}


// return the "crumbs" that
// didn't make it into a full chunk
if (chunk.Count > 0)
{
yield return chunk.ToArray();
}
}


/// <summary>
/// Chunks a sequence into a sub-sequences each containing maxItemsPerChunk, except for the last
/// which will contain any items left over.
/// </summary>
public static IEnumerable<IEnumerable<T>> Chunk<T>(this IEnumerable<T> sequence, int maxItemsPerChunk)
{
if (sequence == null) throw new ArgumentNullException(nameof(sequence));
if (maxItemsPerChunk <= 0)
{
throw new ArgumentOutOfRangeException(nameof(maxItemsPerChunk), $"{nameof(maxItemsPerChunk)} must be greater than 0");
}


var chunk = new List<T>(maxItemsPerChunk);
foreach (var item in sequence)
{
chunk.Add(item);


if (chunk.Count == maxItemsPerChunk)
{
yield return chunk.ToArray();
chunk.Clear();
}
}


// return the "crumbs" that
// didn't make it into a full chunk
if (chunk.Count > 0)
{
yield return chunk.ToArray();
}
}
}

提交答案