How to import the most recent CSV file into Postgres using python?

Question

New daily csv file is constantly generated and written into a directory. The task is to automatically import that new csv everyday into the postgres table (so the PG table will be appended with one new record everyday). I have a python script that is good for importing ALL csv files. How can I just import the most recent csv?

import glob
import psycopg2

file_names = glob.glob('path/to/directory/*.csv')

con = psycopg2.connect(database="XXXX", user="XXXX", password="XXXX", host="XXXX")

for file_name in file_names:
    with open(file_name, 'r') as file_in:
        next(file_in)
        with con.cursor() as cur:
            cur.copy_from(file_in, "tbl_name", columns=('objectid', 'starttime', 'endtime', 'comments'), sep=",")
        con.commit()

con.close()

Ftisiot · Accepted Answer · 2021-04-21 07:25:39Z

1

I believe your problem is purely to find the latest file in a folder. If so, there is here another thread solving exactly that question

answered Apr 21, 2021 at 7:25

Ftisiot

1,8981 gold badge9 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to import the most recent CSV file into Postgres using python?

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related