Async confusion in node.js

Question

I'm in deep trouble trying to understand how to make my code asynchronous in Node.js land. Please see my code below, in two varieties.

Ok so here's my first try - I have three functions here. A processing function (iim), a file copy function (fse.copy), and an archive function (da).

I need da to happen after iim, and iim to happen after fse.copy.

This first approach results in archive happening, but it's empty because iim never appears to happen.

  da(randomString, function(err) {
    if (err) {
      log.error(err);
    } else {
      fse.copy(temp_path, new_location + file_name, function(err) {
        if (err) {
          log.error(err);
        } else {
          log.info("File saved to " + new_location + file_name);
          var sourceImage = new_location + file_name;
          log.debug(sourceImage);
          log.debug(randomString);

          iim(sourceImage, randomString, function(err) {
            if (err) {
              log.error(err);
            }
          });
        }
      });
    }
  });

The next block is an alternate approach which results in the da happening before iim is finished.

  fse.copy(temp_path, new_location + file_name, function(err) {
    if (err) {
      log.error(err);
    } else {
      log.info("File saved to " + new_location + file_name);
      var sourceImage = new_location + file_name;
      log.debug(sourceImage);
      log.debug(randomString);

      iim(sourceImage, randomString, function(err) {
        if (err) {
          log.error(err);
        }
      });
      da(randomString, function(err) {
        if (err) {
          log.error(err);
        }
      });
    }
  });

Answer 1

Here's what I'd recommend -- in your question you say you need to essentially run three functions in series -- correct? Run function A, then function B, and lastly, run function C.

The simplest way to do this is using the asyncjs library.

Here's an example:

var async = require('async');

async.series([
  function a(cb) {
    // do stuff
    cb();
  },
  function b(cb) {
    // do stuff
    cb();
  },
  function c(cb) {
    // do stuff
    cb();
  },
], function() {
  // this will run once all three functions above have finished
});

Now, let's say that each of those functions needs to return data to the next function. SO imagine that function B needs input from function A to run. How do you accomplish that? Using async.waterfall!

var async = require('async');

async.waterfall([
  function a(cb) {
    // do stuff
    cb(null, 'value');
  },
  function b(val, cb) {
    // do stuff with val
    cb(null, 'woot');
  },
  function c(val, cb) {
    // do stuff with val
    cb(null);
  },
], function() {
  // this will run once all three functions above have finished
});

Not bad right?

Hope this helps!

EDIT: Here's a code block showing your code above refactored using asyncjs:

async.waterfall([
  function(cb) {
    fse.copy(temp_path, new_location + file_name, function(err) {
      if (err) {
        log.error(err);
      } else {
        log.info("File saved to " + new_location + file_name);
        var sourceImage = new_location + file_name;
        log.debug(sourceImage);
        log.debug(randomString);
      }
      console.log('Finished running fs.copy');
      cb(null, sourceImage, randomString);
    });
  },
  function(sourceImage, randomString, cb) {
    iim(sourceImage, randomString, function(err) {
      if (err) {
        log.error(err);
      }
      console.log('Finished running iim');
      cb(null, randomString);
    });
  },
  function(randomString, cb) {
    da(randomString, function(err) {
      if (err) {
        log.error(err);
      }
      console.log('Finished running da');
      cb();
    });
  }
], function() {
  console.log('All done!');
});

Answer 2

So you can either put da into the callback for iim (right now it's not) from your second example:

fse.copy(temp_path, new_location + file_name, function(err) {
    if (err) {
      log.error(err);
    } else {
      log.info("File saved to " + new_location + file_name);
      var sourceImage = new_location + file_name;
      log.debug(sourceImage);
      log.debug(randomString);

      iim(sourceImage, randomString, function(err) {
        if (err) {
          log.error(err);
          return;
        }
        da(randomString, function(err) {
          if (err) {
            log.error(err);
          }
        });

      });
    }
  });

That said, callback depth can be flattened with the use of a library like async (https://github.com/caolan/async)