Nested queries in Node JS / MongoDB

Question

My userlist table in mongo is set up like this:

email: [email protected], uniqueUrl:ABC, referredUrl: ...

I have the following code where I query all of the users in my database, and for each of those users, find out how many other's users' referredUrl's equal the current user's unique url:

exports.users = function(db) {
  return function(req, res) {
    db.collection('userlist').find().toArray(function (err, items) {
      for(var i = 0; i<= items.length; i++) {
        var user = items[i];
        console.log(user.email);
        db.collection('userlist').find({referredUrl:user.uniqueUrl}).count(function(err,count) {
          console.log(count);
        });
      }
    });
  };
};

Right now I'm first logging the user's email, then the count associated with the user. So the console should look as such:

[email protected]
1
[email protected]
3
[email protected]
2

Instead, it looks like this:

[email protected]
[email protected]
[email protected]
1
3
2

What's going on? Why is the nested query only returning after the first query completes?

Neil Lunn · Accepted Answer · 2014-03-19 08:40:33Z

Welcome to asynchronous programming and callbacks.

What you are expecting to happen is that everything works in a linear order, but that is not how node works. The whole subject is a little too broad for here, but could do with some reading up on.

Luckily the methods invoked by the driver all key of process.nextTick, which gives you something to look up and search on. But there is a simple way to remedy the code due to the natural way that things are queued.

db.collection('userlist').find().toArray(function(err,items) {
  var processing = function(user) {
    db.collection('userlist').find({ referredUrl: user.uniqueUrl })
      .count(function(err,count) {
        console.log( user.email );
        console.log( count );
    });
  };
  for( var i = 0; i < items.length; i++) {
    var user = items[i];
    processing(user);
  }
});

Now of course that is really an oversimplified way of explaining this, but understand here that you are passing parameters through to your repeating .find() and then doing all the output there.

As said, fortunately some of the work is done for you in the API functions and the event stack is maintained as you added the calls. But mostly, now the output calls are made together and are not occurring within different sets of events.

For a detailed explanation of event loops and callbacks, I'm sure there a much better ones out there than I could write here.

Is there a way to get a callback from process(user) of when it completes?

Shnkc · Accepted Answer · 2014-03-19 07:46:39Z

Callbacks are asynchronous in node.js. So, your count function (function(err,count) { console.log(count); }) is not executed immediately after console.log(user.email);. Therefore, the output is normal, nothing wrong with it. What the wrong is the coding style. You shouldn't call callbacks consecutively to get same result when you call functions in same manner in python (in single thread). To get desired result, you should do all work in single callback. But before doing that, I recommend you to understand how callbacks work in nodejs. This will significantly help your coding in nodejs

Collectives™ on Stack Overflow

Nested queries in Node JS / MongoDB

2 Answers 2

1 Comment

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Related