How to avoid Multiple Step function execution due to an async call?

Question

I've got a Lambda that triggers a step function. I tried to upgrade it from node.js 10 to nod.je 12 and now it starts two Step functions (within two to three ms). Looks as it fires twice because I'm missing an async concept effect here. Can anyone explain this to me, please?

Old code (working)

'use strict';
var AWS = require('aws-sdk');
const s3 = new AWS.S3 ();
exports.handler = (event, context, callback) => {
    event.SourceBucket = "bucket-A";
    event.DestBucket = "bucket-B";
    event.uriBase = decodeURIComponent(event.Records[0].s3.object.key);
    event.uriBase = event.uriBase.substr(0,event.uriBase.lastIndexOf("/"));
    event.uriBaseDel = event.uriBase + "/images";
    event.Move = false;

    //Get all img keys
        const params = {
          Bucket: event.SourceBucket,
          Prefix: event.uriBase + "/images"
        };
        s3.listObjectsV2(params, function(err, data){
          if(err){console.log(err)}else{
            var result = data;  
            event.Keys = result.Contents.map(function(element){return(element.Key);});
                  // Start Step Function with Params
            const stepfunctions = new AWS.StepFunctions();
            const paramsb = {
              stateMachineArn: 'my step function arn',
              input: JSON.stringify(event),
            };

            stepfunctions.startExecution(paramsb, function(err, data) {
              if (err) {callback(err, data);} // an error occurred
              else     {callback(null, data);} // successful response
            });            
          }
        });
};

New code (executes the Step function double)

'use strict';
var AWS = require('aws-sdk');
const s3 = new AWS.S3 ();
exports.handler = async (event) => {
    event.SourceBucket = "bucket-A";
    event.DestBucket = "bucket-B";
    event.uriBase = decodeURIComponent(event.Records[0].s3.object.key);
    event.uriBase = event.uriBase.substr(0,event.uriBase.lastIndexOf("/"));
    event.uriBaseDel = event.uriBase + "/images";
    event.Move = false;
    //Get all img keys
        const params = {
          Bucket: event.SourceBucket,
          Prefix: event.uriBase + "/images"
        };
        var result = await s3.listObjectsV2(params).promise();
        event.Keys = result.Contents.map(function(element){return(element.Key);});

      const stepfunctions = new AWS.StepFunctions();

      const params2 = {
        stateMachineArn: 'my step function arn',
        input: JSON.stringify(event)
      };

      stepfunctions.startExecution(params2, function(err, data) {
        if (err) {return(err, data);} // an error occurred
        else     {return(null, data);} // successful response
      });
};

Your original code did not call its callback before the startExecution was done. Your new code ignores the result of startExecution, and immediately resolves the promise returned by the async function. Why this would lead to a double execution I cannot say. — Bergi
– Bergi, Commented Apr 22, 2020 at 16:13

ericbado · Accepted Answer · 2020-10-20 23:05:43Z

I have experienced the same issue, even if I returned the Promise.

  return stepfunctions.startExecution(params, function(err, data) {
    if (err) {return(err, data);} // an error occurred
    else     {return(null, data);} // successful response
  }).promise();

The fix is actually do not supply the callback:

return stepfunctions.startExecution(params).promise();

According to the API doc, the callback is:

Called when a response from the service is returned. If a callback is not supplied, you must call AWS.Request.send() on the returned request object to initiate the request.

which makes me think I have to provide a callback if I don't call send().

On the other hand, AWS Lambda will automatically await Promises in async handlers. https://docs.aws.amazon.com/lambda/latest/dg/nodejs-handler.html

For libraries that return a promise, you can return that promise directly to the runtime.

I am not sure what causes the double invocations but if the callback is the deal breaker, it makes me think the callback makes the handler wait and since the promise is returned, Lambda will run this code again? No idea, curious to know though.

Collectives™ on Stack Overflow

How to avoid Multiple Step function execution due to an async call?

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related