nodejs async/promise hell

Question

I have the following function:

const bulkPreprocess = (files) => {

let bulkOps = []

files.map(doc => {
    parse(doc).then(content => {
        const sent = sentiment(content)
        bulkOps.push(sentiment)
        bulkOps.push({anotherobject})
    })
})
  return bulkOps
}

Which is called by a main function like so:

module.exports = (req, res) => {
    //parses post request with file uploads
    const form = new multiparty.Form()

    form.parse(req, (err, fields, allFiles) => {
        //called more than once
        const files = allFiles['files']
        let processed = bulkPreprocess(files).then(bulk => {
            console.log(bulk.length)  
            addToES(bulk)
        })
    })

    res.json({ success: true })
}

My problem is that since bulkPreprocess calls the parse function (which is async), I can't get it to wait until all the files are parsed before addToES is called. The parse function itself calls another function which is async (which is why I had to make it async).

The whole flow is like this:

Main -> bulkPreprocess -> (Parse -> parseDoc) -> return value from bulkPre -> addToES

I tried changing all the functions to async/await, I tried returning a promise inside the map function in bulkPreprocess. I tried callbacks. Nothing is working out.

Any advice?

Answer 1

You need to return Promise from bulkPreprocess which resolves after your async parse so you need Promise.all to wait for all parse calls are finished

Edit: Now it's push objects to bulkOps and resolve with it after all parse are finished

const bulkPreprocess = (files) => {
    let bulkOps = [];
    return Promise.all(files.map(doc => {
        return parse(doc).then(content => {
            const sent = sentiment(content);
            bulkOps.push(sentiment);
            bulkOps.push({anotherobject});
        });
    })).then(() => bulkOps);
};