A Firebase Uploader CLI

Question

I built this code as part of a test project that was given to me by a company. They had given me a .csv file that contained over 9000 records and wanted me to build a program that would upload it to Firebase.

Firebase has limits on how much data can be uploaded at one time, so uploading a large .json file would not have been possible (I know this cause I tried).

Tried my luck with promises also- not sure if I did that right, but the program worked lol.

The code below works just fine, but I feel as though I could do better- perhaps make it cleaner? I was working with Typescript for the first time. So if anyone has any tips on this please let me know. Typescript and I are not friends.

Tell me your thoughts please!

#!/usr/bin/env node

import csv from 'csvtojson';
import program from 'commander';
import * as fs from 'fs';
import * as path from 'path';
import * as firebase from 'firebase';
import firebaseConfig from './config';
import { resolve, parse } from 'path';
const ShardingToCsv = require('sharding-to-csv').ShardingToCsv;
const filehound = require('filehound');

// Set up Arguments in Command Line
program
  .version('1.0.0')
  .option('-s, --src <path>', 'Path to .csv file')
  .parse(process.argv);

// Initialize Firebase Config
firebase.initializeApp(firebaseConfig);

// Set DB for Referenced to firebase database
const db = firebase.database();

// If No DB Connection- let me know.
if (!db) {
  console.log('No Database Detected');
}

// Because the .CSV file is to large to parse and upload, break it into 10 different pieces.
const shardFiles = () => {
  // Set a Variable for the source path that the user inputs in
  const srcPath = program.src;

  // If no source path is entered, give this error message
  if (!srcPath) {
    console.log('Please provide valid path to .csv File');
  }

  // a Promise!
  return new Promise((resolve, reject) => {
    console.log('starting sharding process....');
    // Shard Larger .csv Files to Smaller versions for Firebase upload
    var sharding = new ShardingToCsv(srcPath, {
      encoding: 'iso-8859-1',
      maxFileSize: 1000000,
    }).shard();
    sharding.on('error', (err: any) => console.log(err), reject);
    sharding.on('completed', () => {
      console.log('Done Sharding, not to be confused with sharting!');
      // resolve promise
      resolve(sharding);
    });
  });
};

const fromDir = async () => {
  // Await the sharding of the files
  await shardFiles();

  // Actions to create .json files for each shard.

  // variable for shard directory
  // Change this Directory per your application

  const dir = './utils';
  // Read Shard Directory
  const files: any = fs.readdirSync(dir);
  // Interate over the Files
  for (const file of files) {
    const rawData: any = file;
    try {
      // Parse .csv shards into json files using csvtojson
      await csv()
        .fromFile(`utils/${rawData}`)
        .then((data) => {
          fs.writeFile(
            `./utils/${file}.json`,
            JSON.stringify(data, null, 4),
            (err) => {
              if (err) {
                throw err;
              }
              console.log(`created json file with ${file} name`);
            }
          );
        });
    } catch (err) {
      console.log(err);
    }
  }
};

// New Function to upload to database
const uploadToDatabase = async () => {
  // await creation of .json files
  await fromDir();

  filehound
    .create()
    .paths('./utils')
    .ext('.json')
    .find((err: any, jsonFiles: any) => {
      if (err) {
        return console.log(err);
      }
      const readFile = jsonFiles.map((jsonFile: any) => {
        const dbItem = jsonFile;
        const item: any = fs.readFileSync(dbItem);
        const parsedItem = JSON.parse(item);
        // console.log(parsedItem);

        firebase.database().ref('funds').set({ parsedItem });
        console.log('uploaded to database!');
      });
    });
};

uploadToDatabase();