How to write node.js Async inside for loop?

Question

I started build nodejs app to scrape data from a site that has pagination. I write for loop to get all page and push all url to store in an array. Inside that for loop I write async.eachSeries to run all url one by one and after it finish running I want to print out success message. But I don't know how to put that together to make it works properly. Here is my code

for(var pageNo = 1; pageNo <=200; pageNo++){
  siterequest('https://www...../en/jobs/?by=&in=&page='+pageNo, function(err, res, body){
    if(!err && res.statusCode ==200){
        var $ = cheerio.load(body);
        var links = [];
        $('.headline3.job-name-title > strong > a').each(function(i, elem){
            links.push('https://www......com.kh/'+$(elem).attr('href'));
        });
        async.eachSeries(links, function(uri, next){
            console.log('i will go this '+uri);
            next();
        }, function(callback){
            console.log('I did it');
        });
      };
  });
};

The code above did not work for me. Please help me! Thanks in advance

remove semicolon at the end of for loop

Ravi Hirani
– Ravi Hirani

2016-02-12 06:30:58 +00:00
Commented Feb 12, 2016 at 6:30 — Ravi Hirani
– Ravi Hirani, Commented Feb 12, 2016 at 6:30

Oleksandr T. · Accepted Answer · 2016-02-12 06:40:44Z

As you are using async.js you can try async.times instead of for loop

async.times(200, function (paneNo, nextPage) {
  siterequest('https://www...../en/jobs/?by=&in=&page=' + (pageNo + 1), 
    function (err, res, body) {
      if (!err && res.statusCode ==200) {
        var $ = cheerio.load(body);
        var links = [];

        $('.headline3.job-name-title > strong > a').each(function (i, elem) {
          links.push('https://www......com.kh/'+$(elem).attr('href'));
        });

        async.eachSeries(links, function (uri, next) {
          console.log('i will go this '+ uri);
          next();
        }, function(callback) {
          nextPage();
        });
      } else {
        nextPage(err);
      }
  });
}, function(err) {
  console.log('Done!');
});

Collectives™ on Stack Overflow

How to write node.js Async inside for loop?

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related