I would rewrite your code into a while (true) loop, that can only be exited using the return in your try or using a throw; inside a condition in your catch:

while (true)
{
    try
    {
        return await PutAsync(url, content, authenticationHeaderValue, mediaTypeWithQualityHeaderValue);
    }
    catch (Exception ex)
    {
        numberOfAttempts++;
        if (numberOfAttempts >= TotalNumberOfAttempts)
            throw;
    }
}

This way, you don't need ExceptionDispatchInfo or the useless (but required by the compiler) throw at the end.

I also changed catch (AggregateException ex) to catch all exceptions, because await usually doesn't throw AggregateException (unlike task.Wait() or task.Result).

I would rewrite your code into a while (true) loop, that can only be exited using the return in your try or using a throw; inside a condition in your catch:

while (true)
{
    try
    {
        return await PutAsync(url, content, authenticationHeaderValue, mediaTypeWithQualityHeaderValue);
    }
    catch
    {
        numberOfAttempts++;
        if (numberOfAttempts >= TotalNumberOfAttempts)
            throw;
    }
}

This way, you don't need ExceptionDispatchInfo or the useless (but required by the compiler) throw at the end.

I also changed catch (AggregateException ex) to catch all exceptions, because await usually doesn't throw AggregateException (unlike task.Wait() or task.Result).