Retrying Curl X amounts of Time until Successful Return (Script/Functions)

Question

I'm under no illusion that this is at all the best way or even a good way of doing this. It's simply a working solution. Tips, opinions, suggestions, insults, death threats, let's hear it.

<?php

require_once '../include/include.php';
include FUNCTIONS.DS.'functions.php';
$getPage = CurlPage($URL);
$getPage1httpCode = $getPage['httpCode'];

if ($getPage1httpCode != 200)
    {
    sleep(rand(5, 10));
    echo '<h3 style="color:white;">Attempt #2</h3>';
    $getPage2 = CurlPage($URL);
    $getPage1httpCode2 = $getPage2['httpCode'];
    if ($getPage1httpCode2 != 200)
        {
        sleep(rand(5, 10));
        echo '<h3 style="color:white;">Attempt #3</h3>';
        $getPage3 = CurlPage($URL);
        $getPage1httpCode3 = $getPage3['httpCode'];
        if ($getPage1httpCode3 != 200)
            {
            sleep(rand(5, 10));
            echo '<h3 style="color:white;">Attempt #4</h3>';
            $getPage4 = CurlPage($URL);
            $getPage1httpCode4 = $getPage4['httpCode'];
            if ($getPage1httpCode4 != 200)
                {
                sleep(rand(5, 10));
                echo '<h3 style="color:white;">Attempt #5</h3>';
                $getPage5 = CurlPage($URL);
                $getPage1httpCode5 = $getPage5['httpCode'];
                if ($getPage1httpCode5 != 200)
                    {
                    sleep(rand(5, 10));
                    echo '<h3 style="color:white;">Failed.</h3>';
                    var_dump('Fatal Error');
                    die();
                    }
                }
            }
        }
    }

if     ($getPage           == 200 || $getPage1httpCode == 200 || $getPage1httpCode2 == 200 || $getPage1httpCode3 == 200 || $getPage1httpCode4 == 200 || $getPage1httpCode5 == 200)               {

if     ($getPage           == 200) { $HTML             =  $getPage['Data'];            echo '<h3 style="color:green;>httpCode1 Success: 200</h3>'; }
elseif ($getPage1httpCode  == 200) { $HTML             =  $getPage1httpCode['Data'];   echo '<h3 style="color:green;>httpCode2 Success: 200</h3>'; }
elseif ($getPage1httpCode2 == 200) { $HTML             =  $getPage1httpCode2['Data'];  echo '<h3 style="color:green;>httpCode3 Success: 200</h3>'; }
elseif ($getPage1httpCode3 == 200) { $HTML             =  $getPage1httpCode3['Data'];  echo '<h3 style="color:green;>httpCode4 Success: 200</h3>'; }
elseif ($getPage1httpCode4 == 200) { $HTML             =  $getPage1httpCode4['Data'];  echo '<h3 style="color:green;>httpCode5 Success: 200</h3>'; }
elseif ($getPage1httpCode5 == 200) { $HTML             =  $getPage1httpCode5['Data'];  echo '<h3 style="color:green;>httpCode6 Success: 200</h3>'; }
else   { var_dump('Fatal Error'); die(); }

if (isHTML($HTML)) {

$saveHTMLfileName = ROOT_DIR.DS.'pages'.DS.date('m-d-Y_hia').'.html';   
if (!file_exists($saveHTMLfileName))     { file_put_contents($saveHTMLfileName, $HTML);   }

}

TruncateCookieFile();

$conn = NULL;

Associated Functions // include FUNCTIONS.DS.'functions.php';

function CurlPage($URL)
    {
    $Cookie = ROOT_DIR . DS . '_misc' . DS . 'cookie.txt';
    if (!file_exists($Cookie))
        {
        $OpenCookieFile = fopen($Cookie, "w");
        fclose($OpenCookieFile);
        }

    $Host = 'example.com';
    $acceptLanguage = 'en-US';
    $Curl = curl_init();
    curl_setopt($Curl, CURLOPT_URL, $URL);
    curl_setopt($Curl, CURLOPT_CONNECTTIMEOUT, 30);
    curl_setopt($Curl, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($Curl, CURLOPT_FOLLOWLOCATION, true);
    curl_setopt($Curl, CURLOPT_SSL_VERIFYPEER, false);
    curl_setopt($Curl, CURLOPT_SSL_VERIFYHOST, false);
    curl_setopt($Curl, CURLOPT_ENCODING, 1);
    curl_setopt($Curl, CURLOPT_COOKIESESSION, 1);
    curl_setopt($Curl, CURLOPT_COOKIEJAR, $Cookie);
    curl_setopt($Curl, CURLOPT_COOKIEFILE, $Cookie);
    curl_setopt($Curl, CURLOPT_HTTPHEADER, array(
        'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
        'Accept-Language:' . $acceptLanguage . ',en;q=0.9',
        'Cache-Control: max-age=0',
        'Connection: keep-alive',
        'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:28.0) Gecko/20100101 Firefox/28.0',
        'Host: ' . $Host
    ));
    $Content = curl_exec($Curl);
    $httpCode = curl_getinfo($Curl, CURLINFO_HTTP_CODE);
    curl_close($curl);
    if ($httpCode == 200)
        {
        return array(
            'httpCode' => $httpCode,
            'Data' => $Content
        );
        }
    elseif ($httpCode != 200)
        {
        return array(
            'httpCode' => 0,
            'Data' => 0
        );
        }
    }



function TruncateCookieFile() {
    $Cookie = ROOT_DIR . DS . '_misc' . DS . 'cookie.txt';
        $openCookieFile = @fopen($Cookie, "r+");

        if ($openCookieFile !== false)
        {
            ftruncate($openCookieFile, 0);
            fclose($openCookieFile);
        }
}



function isHTML( $str ) { return preg_match( "/\/[a-z]*>/i", $str ) != 0; }

I'm especially turned off by my loop-from-hell HTTP Code 200 Validator. I know I can add a validator on the HTML being returned, matching content-size to file-size. Of course add a more detailed and verbose error logging system. But these are just new features, irrelevant to the question.

I want to know how much my code sucks and how it can be improved.

John Conde · Accepted Answer · 2019-02-01 17:06:40Z

You can put your logic in a while loop that repeats indefinitely (we'll break out of it later). In that loop make your HTTP request and parse the response. If it is successful get the HTML returned by it and break out of the loop. If it fails pause, increment your retry attempts, and then check to see if you have reached your maximum amount of retries. If so, die().

while (true) {
    $getPage = CurlPage($URL);
    $getPage1httpCode = $getPage['httpCode'];
    if ($getPage1httpCode === 200) {
        $HTML = $getPage['Data'];
        break;
    }
    sleep(rand(5, 10));
    $attempts++;
    if ($attempts === 5) {
        die('Fatal Error');
    }
}

I didn't run this code but it should demonstrate how you can simply this code and avoid code repetition and the arrowhead anti-pattern.

Variable Naming

When naming variables don't use all uppercase unless you are trying to represent a constant. $HTML should be $html. (And if you are trying to have a variable act as a constant you should use define() to make it an actual constant but that doesn't apply here).

Variables should also start with a lower case letter. So $Cookie becomes $cookie.

Follow PSR coding standards

The PSR coding standards exist to ensure a high level of technical interoperability between shared PHP code. They also ensure conformity for projects with multiple developers.

PSR-2 says that:

Opening braces for control structures MUST go on the same line, and closing braces MUST go on the next line after the body.

if ($getPage1httpCode != 200)
{

becomes:

if ($getPage1httpCode != 200) {

When doing comparisons use === whenever possible

Unlike == which compares values only, === compares both values and type. This strict comparison helps to avoid error, and attacks, that occur when PHP encounters a comparison of two variables of different types it will coerce one of the variables into the type of the other variable in order to do the comparison.

For example

1 == '1'  // true
1 === '1' // false

How much does this matter? It depends. If you get into a situation where you are getting numbers as strings but you are trying to use them as numbers, for something like sorting, you can get unexpected results if your check only checks value instead of type. And those of us who remember phpBB remember when it was subject to a slew of high profile vulnerabilities many of which were resolved simply by using a stricter comparison. So, yes, it matters.

Very informative and insightful, thank you. Looking forward to doing some clean up with these techniques in mind. The while loop is quite brilliant I was hoping there was a viable solution to loop X amount of times with a variable. Haven't tested yet either, will soon, but see no reason why it wouldn't work. Thanks again! — Brian Bruman
– Brian Bruman, Commented Feb 1, 2019 at 21:51
You can configure to loop a variable number of times. Just remove the hard coded 5 and replace it with a variable that you assign a value to elsewhere. — John Conde
– John Conde, Commented Feb 2, 2019 at 3:06
Yes, that's why I was especially thankful. Tried it out by the way - works great. Actually never used an infinite loop before.. very cool solution. — Brian Bruman
– Brian Bruman, Commented Feb 2, 2019 at 6:28

Stack Exchange Network

Retrying Curl X amounts of Time until Successful Return (Script/Functions)

1 Answer 1

You must log in to answer this question.

Hot Network Questions

Retrying Curl X amounts of Time until Successful Return (Script/Functions)

1 Answer 1

You must log in to answer this question.

Related

Hot Network Questions