Using Javascript node.js how do I parallel process For loops?

Question

I only started to learn javascript 2 days ago so I'm pretty new. I've written code which is optimal but takes 20 minutes to run. I was wondering if there's a simple way to parallel process with for loops e.g.

for (x=0; x<5; x++){ processor 1 do ...

for (x=5; x<10; x++){ processor 2 do ...

Answer 1

Since the OP wants to process the loop in parallel, the async.each() function from the async library is the ideal way to go.

I've had faster execution times using async.each compared to forEach in nodejs.

Answer 2

web workers can run your code in parallel, but without sharing memory/variables etc - basically you pass input parameters to the worker, it works and gives you back the result.

http://www.html5rocks.com/en/tutorials/workers/basics/

You can find nodejs implementations of this, example

https://www.npmjs.com/package/webworker-threads

OR, depending on how your code is written, if you're waiting on a lot of asynchronous functions, you can always rewrite your code to run faster (eg using event queuess instead of for loops - just beware of dependencies, order of execution, etc)

Answer 3

You might want to take a look at the async.js project, especially the parallel function.

Important quote about it :

parallel is about kicking-off I/O tasks in parallel, not about parallel execution of code. If your tasks do not use any timers or perform any I/O, they will actually be executed in series. Any synchronous setup sections for each task will happen one after the other. JavaScript remains single-threaded.

Example :

async.parallel([
    function(callback){
        setTimeout(function(){
            callback(null, 'one');
        }, 200);
    },
    function(callback){
        setTimeout(function(){
            callback(null, 'two');
        }, 100);
    }
],
// optional callback
function(err, results){
    // the results array will equal ['one','two'] even though
    // the second function had a shorter timeout.
});

Answer 4

To run code in parallel or want to make requests in parallel you can use Promise.all or Promise.settled.

Make all the queries in parallel (asynchronously). Resulting in each query firing at the same time.

let promise1 = new Promise((resolve) => setTimeout(() => resolve('any-value'), 3000);
let responses = await Promise.all([promise1, promise2, promise3, ...])
for(let response of responses) {
  // format responses
  
  // respond to client
}

For more examples check out this article