Ajax call inside each loop

Question

I found many posts with this topic. But the solutions I found is not much unsuitable for me. Some experts advised to change code structure, but I am not sure how can I do that.

What I want:
1) Get a list of movie from SQL database
2) Fetch information from a website for each movie

Problem I face: PHP MAX_TIMEOUT occurs.

Solution I thought: call async req for each movie, separately

Bottleneck: Too many async requests

Can you please advice how to implement that (if possible only JS, not jquery please)?

Some solutions from web:

1) Use ASYNC = FALSE.... I don't want to use SYNC req, pointless of using Ajax then
2) Collect all data, then make Ajax call once ... well, I did that first .. but it is a long script (fetching movie info from web), so ultimately causing PHP MAX_TIMEOUT
3) increase PHP MAX_TIMEOUT ... not feasible, I don't know how much to increase.

JS

function loadData(mArray){
    mArray = [{"movieid":"1","title":"10 Things I Hate About You"},{"movieid":"2","title":"100 Girls"}]; // TO SIMLYFY, I PUT THIS CODE HERE .. NORMALLY I GET THIS ARRAY USING ANOTHER AJAX CALL
    for (var i = 0; i < mArray.length; i++) { 
        var obj = mArray[i];
        webAjaxcall(obj["mid"],obj["title"]);  // DEFINITELY NOT A GOOD IDEA
    }
    return true;
}

function webAjaxcall(mid,title){
    var xmlhttp=new XMLHttpRequest();
    xmlhttp.onreadystatechange=function(){
        if (xmlhttp.readyState==4 && xmlhttp.status==200){
            //DO SOMETHING
        }
    }
    xmlhttp.open("POST","file2.php",true);
    xmlhttp.setRequestHeader("Content-type","application/x-www-form-urlencoded");
    params = "title="+title+"&mid="+mid;
    xmlhttp.send(params);
}

Just in case anybody wants to know how I populate the JS array:

FILE 1

$sql = "SELECT `movieid`,`title` FROM movielist";
    $result = mysql_query($sql) or die(mysql_error());
    while($row=mysql_fetch_assoc($result)){
    $output[] = $row;
    }
    exit(json_encode($output));

FILE 2

$json=file_get_contents("http://www.website.com/?t=".rawurlencode($moviename));
$info=json_decode($json);
DO SOMETHING

AJAX TO GET MOVIELIST

var xmlhttp=new XMLHttpRequest();
var myarr;
xmlhttp.onreadystatechange=function(){
    if (xmlhttp.readyState==4 && xmlhttp.status==200){
        myarr = xmlhttp.responseText;
        loadData(JSON.parse(myarr));
    }
}
xmlhttp.open("POST","file1.php",true);
xmlhttp.setRequestHeader("Content-type","application/x-www-form-urlencoded");
params = "fname=<?php echo $ses_id;?>";
xmlhttp.send(params);

when you are calling your loadData function? on window load or something else??? — Manwal
– Manwal, Commented Dec 23, 2013 at 7:34

Community · Accepted Answer · 2017-05-23 12:28:41Z

Note: ASYNC = FALSE means synchronous, which means everything is going to happen in sequence, one call waiting for the previous, and ultimately results in blocking code.

Solution / Opinion: assuming that the site (or API) where you're pulling data can't handle multiple results in a single request, the only way you'll be able to handle this volume of looping ajax requests is to cache the ajax results directly in your SQL db:

::pseudo-architecture::

Let's assume the following PHP files:

index.php

Displays your results
Handles loop logic to display your movies using a single SQL query
Results that are not cached display a "loading" indicator
Write a $(document).ready() function that loops through all the "not-cached" movies, asynchronously calls get.php with appropriate GET parameters for each entry that wasn't already cached. This way it doesn't affect the page load time, as it occurs after the page has already loaded.

::pseudocode::

for movie in movies
    if object has cached data and date retrieved is less than [some time ago]
        return data from SQL db
    else
        display a "caching in progress" notification for that title
        send GET request to get.php

Note: you might need to queue/delay your requests to get.php depending on how powerful your server is, lest you get 1000 separate threads running at once.

get.php

Example url: http://www.yoursite.com/get.php?name=superman&year=1980
Retrieves $_GET parameters sent from your index.php's ajax loop and forwards a request to your 3rd party API/website.
You want this to be asynchronous, throwing a connection-close ASAP (see how to do that)
Think of this like an API "proxy"

::pseudocode::

send 200 ok status code and connection-close header
get $_GET parameters
retrieve API data for your movie by sending $_GET parameters
cache to your SQL db once data is returned

Ultimately, the page loads in realtime, and in order to see the new data, you'd need to refresh the page (or if you want to get REALLY fancy, do something with WebSockets that notifies the client).

thanks r3mus .. 1) u r right, that's why I don't want to use SYNC req ... 2) I edited and try to make it simpler, if it happens in actual. However, main concern is I have let say 1000 movies to collect data and I have to do it one by one .... 3) the API does not handle multiple result in a single request. the solution u proposed is little difficult for me, but I will try to understand and implement. Thanks. ...
1) OK, so you'd actually want async = true ;) 2) Still complicated, but given the solution, I don't think your example needs to be more clear 3) I'll see if I can expand the pseudo to make it more clear.

Collectives™ on Stack Overflow

Ajax call inside each loop

1 Answer 1

2 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Linked

Related