How to debug a hanging promise, not resolving a Postgres query using node-postgres?

Question

I'm doing some basic CRUD operations on Postgres DB with Node app using 'pg' (node-postgres). Seems to have some issue making this particular query - the whole app hangs, like the Promise never resolves. The execution stops at this point and doesn't throw error.

The whole thing worked before I changed the DB structure a little bit and refactored the code (made it more modular, etc.). The database is up and connected (using Cloud9's Ubuntu 14.04 LTS container), other query checks successfuly whether it exists. This failing query ('select * from inventory ...') maually works fine - when I type it in the psql CLI, I get result of 0 rows, as it should be. So my best guess is, something in the async/await logic is just not right.

// This part executes without issues
const initDB = async () => {
    try {
        await client.connect();
        console.log('Client connected to db ...');

        const checkTable = await client.query("SELECT EXISTS (SELECT 1 FROM information_schema.tables WHERE table_name = 'inventory')");
        if (!checkTable.rows[0].exists) {
            const table = await client.query('CREATE TABLE inventory (prod_id serial PRIMARY KEY, product_name VARCHAR (50), category TEXT, description TEXT, price REAL, width INT, depth INT, height INT, checksum TEXT, history REAL ARRAY)');
        }
    } catch (err) {
         console.log("Error initializing db.", err);   
    }
};

// This part fails
Promise.all([getProductData(productsUrlBase)]).then(async (values) => {
       // checksum = ...
       try {
          console.log('This logs on the console');
          const check = await client.query('SELECT * FROM inventory WHERE checksum=$1', [checksum]);
          console.log(check.rows, 'if this is empty, INSERT in DB!'); // Never logs on the console. App doesn't throw error, either.
    } catch (err) {
        console.log('Query error - checking / inserting items: ', err);
    }   
});

Postgres log file:
2019-04-19 12:18:53 UTC LOG:  database system was interrupted; last known up at 2019-04-19 08:39:56 UTC
2019-04-19 12:18:54 UTC LOG:  database system was not properly shut down; automatic recovery in progress
2019-04-19 12:18:54 UTC LOG:  record with zero length at 0/17A3BF8
2019-04-19 12:18:54 UTC LOG:  redo is not required
2019-04-19 12:18:54 UTC LOG:  MultiXact member wraparound protections are now enabled
2019-04-19 12:18:54 UTC LOG:  database system is ready to accept connections
2019-04-19 12:18:54 UTC LOG:  autovacuum launcher started
2019-04-19 12:18:54 UTC LOG:  incomplete startup packet

I expected to get Error at least, but nothing happens. You can see the Postgres server logs - to me it doesn't seem to have any significant problems there.

Here's the full GitHub repo. The problematic part is in ./data-handlers/update.js

did you try to return or await the Promise.all function ? github.com/kboeff/price-checker/blob/master/data-handlers/… — AnC
– AnC, Commented Apr 22, 2019 at 13:19
Tried, but didn't help. In fact, no queries work inside the wrapper function updateDB(). It is called in app.js like this for testing purposes: (async() => { await initDB(); updateDB(productsUrlBase, categories[0]); })(); A query inside this IIFE works fine, like this: (async() => { await initDB(); let testQuery = client.query('SELECT * FROM inventory').then(rows => { console.log(rows); // This logs the DB info. }); updateDB(productsUrlBase, categories[0]); })(); — kbo
– kbo, Commented Apr 22, 2019 at 14:01
And did you try to return or await the promise here? github.com/kboeff/price-checker/blob/master/app.js#L63 and use return in update.js line 12 — AnC
– AnC, Commented Apr 23, 2019 at 9:26

AnC · Accepted Answer · 2019-04-23 20:55:10Z

There a few little issues in your code but besides that, well done.

1) return or await the promises properly:
- https://github.com/kboeff/price-checker/blob/master/app.js#L63
and
- https://github.com/kboeff/price-checker/blob/master/data-handlers/update.js#L12

2) connect node-postgres via await client.connect(), read the docs again it's worth it

3) wrong for loop => https://github.com/kboeff/price-checker/blob/master/data-handlers/update.js#L18

4) and this part is still inside the for loop which aims for traversing on object properties but should be in the "for each record" loop https://github.com/kboeff/price-checker/blob/master/data-handlers/update.js#L28-L53

5) missing value placeholder in query $9 https://github.com/kboeff/price-checker/blob/master/data-handlers/update.js#L36

And are you sure you want to fire all these queries at once? Maybe in synchron would be better here. While talking about performance, maybe you should use the postgres type UUID for your checksum operation.

here is a working example using md5 for checksum:

const { Client } = require('pg');
const priceAlert = require('./price-alert');
const getProductData = require('./get-product-data');
const crypto = require('crypto');

const client = new Client;

const updateDB = async (productsUrlBase, category) => {
    // connect to db
    await client.connect();
    const records = await getProductData(productsUrlBase);

    for (const record of records) {

        let checksum = '';

        for (let key in record){
            if (record.hasOwnProperty(key)) {
                if (key !== 'price') {
                    checksum += record[key] || '';
                }
            }
        }

        try {
            const md5Checksum = crypto.createHash('md5').update(checksum).digest("hex");
            const check = await client.query('SELECT * FROM inventory WHERE checksum=$1', [md5Checksum]);
            console.log(check, 'if this is empty, INSERT in DB!'); // DEBUG
            // Record not found in db, add.
            let arrLit = '{' + record.price +'}';

            if (check.rows.length === 0) {
                let rows = await client.query("INSERT INTO inventory(product_name, category, description, price, width, depth, height, checksum, history) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)", [record.name, category, record.description, record.price, record.width, record.depth, record.height, md5Checksum, arrLit]);
                console.log(rows, 'INSERTED to DB.'); // DEBUG
            } else {
                // compare prices, signal for changes!
                let item = check.rows[0];
                // console.log(item);
                if (item.price !== record.price) {
                    priceAlert(item, record.price);
                    let formattedPrice = parseFloat(Math.round(record.price * 100) / 100).toFixed(2);
                    let txt = "UPDATE inventory SET price=" + formattedPrice + ", history = history||ARRAY[cast(" + formattedPrice +"as real)] WHERE checksum='" + checksum + "::uuid'";
                    let updatePrice = await client.query(txt);
                    console.log(updatePrice);

                }
            }
        } catch (err) {
            console.log('Query error - checking / inserting items: ', typeof record.price, err);
        }
    }

};

module.exports = updateDB;

The column inventory.checksum must be type UUID for this example to work
alter table inventory add column checksum uuid;

PS: Maybe also add process.exit(); for example at app.js line 64

Thanks a lot for the detailed answer. :) I'll try to fix all these issues and come back with some feedback.

Collectives™ on Stack Overflow

How to debug a hanging promise, not resolving a Postgres query using node-postgres?

1 Answer 1

1 Comment

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Related