Using queues to handle asynchronous callbacks

Question

The application I'm working on has to fire off multiple requests to an external API. It will receive the results of those requests asynchronously, but then I need to handle each response in a synchronous way. My research tells me there are multiple ways of doing this, but I am considering this idea and wondering if it will work:

Every time I make a call to the external API, its response is entered into the queue as soon as it asynchronously comes back:

function genericFetchFunction(url, callback) {
    makeAsyncRequest(url, (result) => {
        addToHandlingQueue({
            data: result,
            callback
        });
    });
}

The queue handler pushes the response onto the queue, and initiates the queue firing process.

const responseQueue = [];

function addToHandlingQueue(response) {
    responseQueue.push(response);
    if (responseQueue.length == 1) {
        fireQueue();
    }
}

function fireQueue() {
    let item = responseQueue.shift();
    item.callback(item.data);
    if (responseQueue.length > 0) {
        fireQueue();
    }
}

Would this code work how I expect? Will all results get into the queue and their callbacks fired in sequence? If not - why not?

EDIT: The use case for this is that the callback for these responses is itself going to initiate a process that should be only handled synchronously rather than asynchronously (an update to a stored state).

Seems overly complex imo. If you simply want to send off n requests at once, then process them in order once they're all done, all you need is promise.all. If you want to instead send them one at a time, a recursive function will suffice. — Kevin B
– Kevin B, Commented Aug 27, 2018 at 18:44
@Marisha No, a response never comes in while something is still processing. JS first finishes the current run before firing the next asynchronous callback. — Bergi
– Bergi, Commented Aug 27, 2018 at 18:52
You've not been punished. Perhaps your question has. Someone did not think the question was up to the SO standards and downvoted. There has been no rush to do so, and the question as it stands looks ok to me. (If you haven't read How to ask in the help center, you probably should.) — Scott Sauyet
– Scott Sauyet, Commented Aug 27, 2018 at 19:02
Take a look at async waterfall and parallel techniques. It has all been done before and really simple to reproduce once you work with it, no need for a library to depend one though it’s quite small and stable. — dbf
– dbf, Commented Aug 27, 2018 at 19:09
You can gain reputation by answering questions as well. Probably much more quickly, in fact. — Scott Sauyet
– Scott Sauyet, Commented Aug 27, 2018 at 19:12

Marisha · Accepted Answer · 2018-08-27 19:19:35Z

2

For any who come across this question and are suffering from my same misconceptions, it seems this question is derived from a misunderstanding of how asynchronous processing is handled in Javascript. Multiple threads are not true "processing threads" in a hardware sense - rather just queued chunks of code that are iterated over until completion of all chunks. So in my example, nothing new can ever be added to the queue while the queue is still being processed, because that processing is all happening within the same event loop.

I found a good explanation of it in this article: How Javascript Works (the 'Dissecting the Event Loop' section) and there is additional information on the Concurrency Model and Event Loop MDN page.

answered Aug 27, 2018 at 19:19

Marisha

19614 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Scott Sauyet Over a year ago

Note that one of the comments in the question pointed you toward Promise.all as a way to handle the requirements you seemed to need. I would concur. If you want a sequential set of processes to run after a collection of asynchronous ones completes, then it would help to create Promise returns from your asynchronous calls, wrap the returned Promises in an array, then call Promise.all on the array. You'll get back a new Promise, and when (if?) that's resolved, you will be given an array containing those results in order. You should be able to build what you want off of that.

Collectives™ on Stack Overflow

Using queues to handle asynchronous callbacks

1 Answer 1

1 Comment

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Related