Here are a few things you can do to improve the performance of the data loading:
convert the JSON file to CSV and use
LOAD DATAto load from a file (sample). It is probably the fastest way to do what you are trying to do.use
.executemany()instead of.execute():datajson = json.load(testFile) insert_query = """ INSERT INTO cstore (AccountNo) VALUES (%(AccountNo)s) """ cursor.executemany(insert_query, datajson)look into disabling/removing existing indexes during the insertion and then re-creating them after the insertion is done
make sure you are not doing the data-load "over the internet" and there is no network-latency and bandwidth impact, be "closer" to the database server
loading the JSON file is probably not a bottleneck, but you may also look into faster JSON parsers like
ujsonI remember we also had some data-loading-performance-related success with the ultra-fast
umysqlPython database driver, but it looks like the package has not been maintained for quite a while