5
\$\begingroup\$

So I have a method that looks like this:

   public static List<List<List<string>>> Split(List<List<string>> source, int chunksize)
   {
       return source
           .Select((x, i) => new { Index = i, Value = x })
           .GroupBy(x => x.Index / chunksize)
           .Select(x => x.Select(v => v.Value).ToList())
           .ToList();
   }

While this is nice and elegant and readable, unfortunately that groupby is killing performance because of the implicit sort. I need a way to rewrite this to be more performant, while still being reasonably elegant.

\$\endgroup\$

2 Answers 2

Reset to default
6
\$\begingroup\$

We can make the method more generic:

public static IEnumerable<IEnumerable<T>> Split<T>(IEnumerable<T> source, int chunkSize)
{

Then we can store the current chunk in a List<T>. Using the constructor that takes the initial capacity should be good for performance:

    var chunk = new List<T>(chunkSize);
    foreach (var item in source)
    {
        chunk.Add(item);
        if (chunk.Count == chunkSize)
        {
            yield return chunk;
            chunk.Clear();
        }
    }

Finally we need to return the final (non-empty) chunk

    if (chunk.Count > 0)
    {
        yield return chunk;
    }
}

Depending on your use case, you might want to make a copy of the list and yield return chunk.ToList() instead.

\$\endgroup\$
5
  • \$\begingroup\$ This implementation has a bug. You're doing yield return chunk which returns a reference to the list and then you're clearing the list immediately after. Remember, you're returning a reference. \$\endgroup\$ Commented Dec 10, 2014 at 2:57
  • \$\begingroup\$ @VincePanuccio it works fine if the chunks are processed as they are generated, e.g. var chunks = Split(new[] { 1, 2, 3, 4, 5 }, 2).Select(chunk => string.Join(",", chunk)); Console.WriteLine(string.Join("|", chunks)); prints 1,2|3,4|5 as expected. If that's not the case, then using ToList as I mentioned will solve the problem. \$\endgroup\$ Commented Dec 10, 2014 at 3:09
  • 2
    \$\begingroup\$ The problem with that approach is the consumer needs to know that. Always returning a copy will avoid any possibility of a bug. \$\endgroup\$ Commented Dec 10, 2014 at 11:34
  • \$\begingroup\$ I am not particularly good with Generic methods, how would I invoke this method when passing in a List<List<string>>? My attempts thus far compile, but give Null reference exceptions when run. \$\endgroup\$ Commented Dec 11, 2014 at 0:20
  • \$\begingroup\$ @AlexanderRyanBaggett you don't need to do anything special to call it with a List<List<string>>. There's an example you can play with here, output is at the bottom of the page. \$\endgroup\$ Commented Dec 11, 2014 at 0:28
1
\$\begingroup\$

From your question, it looks as though you have a list of lists which you need flattened and chunked.

If you have some time up your sleeve, it would be beneficial to make this method generic so it can handle any type and perhaps even turn it in to an extension method so it's available more easily.

void Main()
{
    var list1 = new List<string>(new[] { "1", "2" ,"3" });
    var list2 = new List<string>(new[] { "4", "5" ,"6" });
    var list3 = new List<string>(new[] { "7", "8" ,"9", "10" });

    List<List<string>> source = new List<List<string>>(new[] { list1, list2, list3 });

    Split(source, 3).Dump();
}

public static IEnumerable<IEnumerable<string>> Split(List<List<string>> source, int chunksize)
{
    var chunk = new List<string>(chunksize);

    foreach (var element in source.SelectMany (s => s))
    {
        if (chunksize == chunk.Count) {
            yield return chunk;
            chunk = new List<string>(chunksize);
        }
        chunk.Add(element);
    }

    yield return chunk;
}
\$\endgroup\$

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.