C# WebClient parallel call to API with Async/ Await

Question

I am new to Async/Await. I have to call api and sace result into a txt file. I used webclient.OpenRead and was able to save the file. I want to run my code with multiple inputs to API and I would like to run it in parallel.

For sequential operation I used WebClient.OpenRead() as code works.

class Program
{
    static void Main(string[] args)
    {
        var name = "test1";
        var filename = $"c:\\temp\\{name}.txt";
        var url = $"https://testapi?name eq '{name}')";
        using (var client = new WebClient())
        {
            using (Stream stream = client.OpenRead(url))
            using (StreamReader streamReader = new StreamReader(stream))
            using (JsonTextReader reader = new JsonTextReader(streamReader))
            {
                reader.SupportMultipleContent = true;
                var serializer = new JsonSerializer();
                var parsedData = serializer.Deserialize(reader).ToString();
                File.WriteAllText(filename, parsedData);
                Console.WriteLine($"Result save at {filename}");
            }
        }
    }
}

For parallel operation I tried to utilize WebClient.OpenReadAsync() but my code is not saving the file in c:\temp\

static void Main(string[] args)
        {

            var names = new string[] { "test1", "test2" };
            foreach (var name in names)
            {
                var filename = $"c:\\temp\\{name}.txt";
                 var url = $"https://testapi?name eq '{name}') ";
                var client = new WebClient();
                client.OpenReadCompleted += (sender, e) =>
                {
                    Stream reply = (Stream)e.Result;
                    StreamReader s;
                    s = new StreamReader(reply);
                    using (JsonTextReader reader = new JsonTextReader(s))
                    {
                        reader.SupportMultipleContent = true;
                        var serializer = new JsonSerializer();
                        var parsedData = serializer.Deserialize(reader).ToString();
                        File.WriteAllText(filename, parsedData);
                        Console.WriteLine($"Result save at {filename}");
                    }
                };
                client.OpenReadAsync(new Uri(url));
            }
        }

What is wrong with my code? Any pointers?

Michał Żołnieruk · Accepted Answer · 2017-09-19 12:39:54Z

Below is the code which will parallel your tasks. Notice that I don't use foreach loop, which in your case would mean sequential tasks execution. The important part is that first I create a list of tasks which should be executed, then I create a single task with Task.WhenAll by wrapping a list of tasks and then I await the end of the task with .Wait();

Also, read about .Wait(). and how it behaves. It effectively blocks the thread of the application, which can be not desired in more complicated applications.

static void Main(string[] args)
{
    var tasks = new[] {"test1", "test2"}.Select(SaveData);
    Task.WhenAll(tasks).Wait();
}

private static async Task SaveData(string fileName)
{
    var filename = $"c:\\temp\\{fileName}.txt";
    var url = $"https://testapi?name eq '{fileName}') ";
    var client = new WebClient();

    client.OpenReadCompleted += (sender, e) =>
    {
        var reply = e.Result;
        StreamReader s;
        s = new StreamReader(reply);
        using (var reader = new JsonTextReader(s))
        {
            reader.SupportMultipleContent = true;
            var serializer = new JsonSerializer();
            var parsedData = serializer.Deserialize(reader).ToString();
            File.WriteAllText(filename, parsedData);
            Console.WriteLine($"Result save at {fileName}");
        }
    };

    await client.OpenReadTaskAsync(new Uri(url));
}

thanks for your response. It works. Ocassionally, I get error from e.Result as {"The remote server returned an error: (500) Internal Server Error."}. This could be from the service.

Collectives™ on Stack Overflow

C# WebClient parallel call to API with Async/ Await

1 Answer 1

1 Comment

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Related